Martin,
Eh, I think I may have more junk, er, things on me than that EXCEPT: the keys. And the keys have for sure been the culprit in the past for me - I had the same problem with worn pockets etc that you describe. I simply do not carry all those keys on my person anymore. One set of keys to my personal vehicle in one pocket and one set of keys to my work vehicle in the other pocket and that's it. I also dispensed with all manner of fobs and simply have a small split ring to keep each set together (although the remote for my personal vehicle adds bulk, it is lightweight and has a pocket-friendly contour).
I have a large set of house keys, including for parents houses, keys to city hall, keys to Scout buildings and lockers, keys to various locked containers at work "with" me, but I do not carry them on my person.
Since ditching the keys, I have completely eliminated the worn pockets problem. Ergo, the number of keys I carried were the culprit.
If stripping down your pocket keys doesn't help, then I have one last word for you: suspenders <grin>.
Tom