That's right,
There's basically two surefire ways to destroy a permanent magnet. You can cook it with excessive heat (say, over 200 degrees F), or you can beat on it physically (like with a hammer). Since either would destroy the casing before you even got to the magnet inside, normal care considerations for the compass are all you need.
Metal objects brought into the vacinity of a magnet will cause it to deflect towards the metal, but it doesn't do permanent damage. Remove the metal tool, and the compass will go back to pointing at magnetic north... no problem as long as you're aware of it (like dont try to take a reading while you're holding a chunk of iron in your hand).
Since I can say way more about this than probably anyone wants to hear, I'll shut up now. <img src="images/graemlins/tongue.gif" alt="" /> But I'll try to answer any additional questions.