I wear mine without incident, in New York, at the airport, wherever. Long as it fits the context, no one seems to even notice, let alone care. Don't wear it desert style unless you are in a dust storm. Wearing it like that in public will get you noticed in a bad way. Kinda like wearing a bandana over your nose and mouth like a desperado, or a Nixon head mask.
Now if I was wearing it with an egal or had on other middle eastern gear, that might start getting me noticed some also.
If you just wear it like a scarf and have it ready to deploy in other ways as the need arises, then you shouldn't expect any trouble.
_________________________
The ultimate result of shielding men from the effects of folly is to fill the world with fools.
-- Herbert Spencer, English Philosopher (1820-1903)