Yeah, Geo-Caching is kind of the high technology version of orienteering.
There is a lot more to navigation than just relying on a GPS reciever though.
You still need the maps because the GPS will give you direct courses unless you set your own waypoints.
The GPS will expect you to walk over a lake or a cliff instead of around it when it sets your course.
I keep a moderately good sighting compass with me all of the time. Silva makes a nice little folding one called the huntsman or the guide.
There are tons of inexpensive military lensatics around too.
A compass lets you just use the GPS for occasional position fixes, which is what it is really good at.
This means your batteries will last much longer.
Orienteering and the little button compasses are not as good for me because they have no way of sighting them.
Being able to take a back bearing on a landmark can be really helpful in locating your position on a map.
http://geology.isu.edu/geostac/Field_Exercise/topomaps/bearing.htm