One of two things is going on.
Your GPS is, in all probability, using the WGS-84 Dataum, while the vast majority of topo maps use NAD-27. There can be pretty major differences in position data.

the other issue might be the format of your lat/long display.

40.56734 degrees is not the same is 40 56.734 which is 40 degress 56.734 minutes.

But you should really be learning and using UTM anyway, even for Geocaching. It's a better, faster and easier system to use and it's well supported by Geocache. These problems don't exist in UTM at all, and for geocaching UTM just plain kicks butt.