If it's for the summer, I vote for cotton, too. I wear cargo pants and t-shirts a lot, and I have cargo shirts, too - either should fill your need for extra pockets. I suggest a hat, as well; not a cap.

I worked summers in Texas at a place that made railroad cross-ties, and long pants, long sleeves, and hats worked very well. I preferred chambray shirts, as they were light and breathable, but soaked up the sweat for easy evaporation.

I wear cargo pants from Royal Robbins (now 511, I think) and BDU pants in khaki or navy. I know you don't want military gear, but nobody has ever commented on the BDUs, and I think I'm getting away with it. (Might be because I'm 60, though.) Both pants are made for scrambling in bad terrain (Royal was a rock climber and designed his pants for that), and they have lasted me years (remember my age, though - I may not scramble as much as you). Cargo pants and a long-sleeved t-shirt project an everyday image so long as you aren't blousing your pants in combat boots.

I wear my cargo pants and shirts to Burning Man every year - often around a hundred during the day, cool at night. Layers are your friend. So are hats and bandanas.

My suggestion is not to buy all one make of pants and shirts. REI sells many cargo pants that just don't wear well in the real world. They're made of some fabric that won't let you sweat through, and the fabric is too thin to last. I'm sure they're made for people who travel in planes and cars, but not on the ground.