This is key for me, I don't really consider something to be an EDC item unless it's actually on my body.

This is especially true of the context we are talking about here; a true emergency during the work day isn't necessarily going to happen when I'm within arms reach of my bag. I might be out at the coffee shop, in a meeting room, on a different floor, running a quick errand or any other of a multitude of reasons that I may not be at my desk.

That's not to say I don't have a backpack that supplements my EDC which is available when I'm at my desk, but I realize that there is a decent chance that if something bad happens I may not have access to it.

Based on this, I think the initial list of flashlight, whistle, knife (or multi-tool) and bandanna (or, my preference, a Buff) is a good set of things to have on your person at all times. To this I'd add your keys and either money or transit fare to ensure that if you had to evacuate your workplace before going back to your desk that you could still get home easily.