A GPS handheld unit calculates its position based on the time it takes to receive the satellites signal. This "time" is extremely precise. Buildings & mountains can deflect the radio signal affecting the time to reception giving you a bad reading. In such a case moving around will correct the reading.
One interesting thing happened about 6 years ago. I actually got an incorrect reading just north of Ouray, Colorado when standing next to a large cliff. I triple checked my readings and wrote them down, and as I moved away from the cliff, about a mile or two, it corrected itself.
So what you experienced there is the GPS behaving normaly. You guys should really read up on the subject if you're going to trust your life to a GPS !!!
This is a good place to start :
About GPS