I wear cargo pants and cargo shorts much of the time. Sometimes they're less comfortable (when I'm strapped in my airline seat for example), and then I wear a photographer's vest which I can take off and put under the seat in front or leave on if it's chilly on the plane. My vest is basically another piece of carry-on luggage, but I don't get dinged for it at the gate.