I grew up in an unsavory neighborhood in NYC. The only time I was ever (successfully) mugged, I was about 12, and stood out way more than I should have for where I was.
Anecdotally, blending in has allowed me to avoid a LOT of trouble in big cities.
I take the opposite approach when off the beaten path, I've never wanted to not stand out (excepting youthful hijinks long ago).