As others have said, I think goals of camping, especially backpacking, and preparedness are essentially the same: to live & survive comfortably in the absence of modern amenities and support systems.

It seems natural that the gear & skills for one would also be beneficial for the other even if, as others noted, there isn't full overlap.

If I was trying to placate negative people though, I think I tend more towards modelling my preps after the recommendations of the Red Cross or a government initiative & would communicate the fact. For some that could shift their perception from paranoid wingnut to responsible citizen following the advice of experts.