I spotted the problem in /var/log/secure:
Jul 1 22:08:44 madhatter sshd[25996]: Authentication refused: bad ownership or modes for file /home/josh/.ssh/authorized_keys
I had the wrong permissions on my .ssh directory and my authorized_keys file.
chmod 755 ~/.sshand
chmod 644 ~/.ssh/authorized_keysdid the trick.