"When in doubt, leave it out!" is another way of putting it, at least if one is traveling by foot. I try to force myself to follow that rule. No matter how fit you are, when you carry less you can travel further and faster. Climbers have always known that carrying too much emergency bivvy gear on a day climb often insures that you will need to bivvy! You want to try to find the Goldilocks point: Not too much, not too little....just the right amount of gear!
That being said, it is sometimes worth it to carry a back up of a few really essential items. In the bottom of my pack I always have a small lightweight folding knife and some extra firemaking gear, and a few other odds and ends. In winter I always carry an extra very small headlamp. Because of the unstable and sometimes severe climate around these parts I always carry a lot more clothing than I think I will need.
On advantage of traveling lighter is that one can take other stuff just for fun. For example, in Alaska there is always the opportunity for really neat wildlife sightings. Hence I often carry a good pair of binoculars. They are a bit heavy but are just the thing when you spot a familly of bears on the next ridge over, and want to stretch out on the tundra and watch them for awhile.
_________________________
"Toto, I've a feeling we're not in Kansas any more."
-Dorothy, in The Wizard of Oz