Hey Flashman,
It is terrific that your personal situational awareness was acute enough to inform you that something was not right and you took steps to avoid a conflict. Thousands of years of human evolution has provided our species with a refined sense of intuition and excellent ability of reasoning. It just takes the "Survivor Personality" to listen to these clues and react appropriately. This topic is well explained in Lawrence Gonzalales book, "Deep Survival", a very worthwhile read.
My experiences have been very similar to LED's; in easy access locations (e.g. campgrounds, urban areas) where drugs and alcohol were being abused I have had any trouble with other people.
Most of my adventures are now in remote locations and the only other people you meet are other avid outdoor enthusiests.
Good topic,
Mike