GPS doesn't know what direction it is pointing when it is not moving, or is moving very slowly. Thus a supplemental compass might prove useful.
Not always true. More modern GPSr have multi-axis compasses and give accurate direction even when motionless. My Garmin 62s for example. I do have to remember to recalibrate the compass after every battery change tho. But no matter what I always keep at least one magnetic compass with me.