As one of those 'elderly men with concealed guns' I can testify that nobody pays any attention to me or my fanny pack.
If I were a criminal, I would pay plenty of attention to you (and pick an easier target)!
But then, I like to fancy myself smarter than the average criminal. The common everyday dumb ones, yeah, they'd be clueless. Which is a bonus for armed good guys!