I guess it depends on the GPS in question and the precision of the navigation required.

http://www.youtube.com/watch?v=CJyt3RFycsE

Although mapping GPS devices are constantly improving, ensuring that the correct GPS Datum and Grid Mag variation are programmed properly, even the most expensive mapping GPS systems don't generate a truly accurate bearing to follow unless they have an electronic compass built in (again these are only usually accurate to +- 2 degrees. GPS firmware can give a reasonable indication once you are on the move using velocity filtering but if you are under cover or the number of satellites begins to vary (affect HDOP) this can effect your positioning making your position jump around (i.e. even up to +- 100m) giving inaccurate compass bearings to a nearby waypoint when standing still. I basically don't trust GPS compass information to say with confidence as to the direction to move off in to a way point (unless a built in electronic compass is in use rather than pure GPS)- there is just to many things that need to be right to point you in the correct direction not including the fact that GPS information is slow as indicated in the video.

An experienced orienter would have finished the course using map and compass by the time the waypoints had been programmed into some of these GPSs.

EDIT;


As a little experiment I took my GPS (a Garmin Camo) and set my position as a waypoint called 001 (I was using the GPS indoors so had no WAAS PRN and could only find 4 PRNS to generate a fix, which simulated heavy foilage coverage. I then programmed the GPS for another waypoint 002 directly North which was 200 metres away then turned off the GPS for 5-10 minutes.

Switching the GPS back on and allowing the GPS to again aquire a fix showed that the waypoint 002 was now showing a bearing of 332 degrees and was 182 metres away rather than being directly North @ 0 degrees 200 metres away i.e. a 28 degree error. A significant azimuth error.

GPS receivers also are subject to errors just as a map and compass work are for the the unaware, the problem is that the technology is seen by many as being 'fool proof' and leads folks into a false sense of security leading to potentially disastrous results.