Even being in NYC, I bet there are places to cache stuff. For example, lots of buildings have access to the roof, and many of those roofs may have sheds or something for repairs. Throwing in a bucket of food may well blend in, and no one will look twice at it. If you get involved in rehabbing areas (like those folks that go and make empty lots into gardens), you might be able to drop a little goody bag in as you plant a tree or something, since you've got a hole already dug. Abandoned cars still have trunks. Probably not the best area for visiting, but still a thought (even if you have to trudge into Jersey ) Those fake rocks that people store their keys in may be useful too, or the fake cleaning supply-cans that are occasionally found.

If you don't feel good about 5-gal buckets, what about ammo canisters from surplus stores? Not all are metal, if you're worried about scroungers with tools.

I'd add some TP. Maybe some baby wipes too. Bottle of soap or alcohol gel. More firestarting tools, some space blankets or something similar. Extra batteries for whatever you're carrying for light/radio/gps. Ammo, if you're carrying something for food procurement (or extra fishing line, hooks, etc). All of these are fairly small and shouldn't impact the space too much.

For food, add some dried fruit, nuts, maybe a bottle of vitamins and some pain killers (advil, tylenol, etc). I'd even add some junk food/goodies, just for pure comfort value. Consider some hot sauce or seasoning packets, since beans and rice gets real old, real quick.