No, I haven't had a similar experience, but then I don't often camp alone. It's usually a family outing. Perhaps the group discourages some of the aggressive people. We have met up with our share of drunken/annoying people at public campgounds, which is why we look for the more remote areas, as other people have already indicated in this thread.
There is a time to oppose evil and there is a time to avoid it. I think you made the right decision and acted on it very appropriately.
Frank2135
_________________________
All we can do is all we can do.