by my calculations, being off by just one degree, and traveling just one mile, you'll be off course by about 92 feet (roughly 8.7 feet off per every 500 feet traveled)
That would be true if the error is not random;even if that were the case one could rather easily observe the objective and arrive at the correct spot. Usually you are dealing with a swinging needle that goes on either side of the true azimuth. In practice, one does pretty well to be within two degrees.
As Hikermor mentioned. There is some deviation in a compass needle - especially when taking a bearing while on the move.
Unless someone is out in extreme weather such as fog or blinding snow where you cannot see your surroundings at all, there is no need to "dead stick" (our old school slang for needing a 100% accurate compass reading.) So being out 1-4 degrees over a mile with decent sight visibility of your surroundings is well within acceptable limits for 99% of hikers, hunters, boaters. And most times, you find the trail, road or boat dock you were navigating to.