I agree with your approach on this. I have my stashes of key gear in my car, home, desk, briefcase, and a survival vest of contents if I had to walk from my car, so an EDC kit is not necessary for me. Use of a PSK can mean something was not planned for properly but does not mean that one would not feel comfortable carrying a kit of some sort.
You have to weigh the pros and cons as they impact you and do what makes you feel comfortable.