Izzy has pretty much got it. In order of worst to best, as far as common solutions go, you're pretty much looking at the following:

Bare mouth
Cloth covered mouth (bandanna or t-shirt)
Wet Cloth covered mouth or maybe a basic paper mask
N95-N100 Mask/Respirator (the ones with one-way valves are best)
R95-R100 Respirator
P95-P100 Respirator
Full face P100 Respirator
SCUBA

For sustained use in a situation like 9/11, I would really want a full P class respirator with a lot of spare cartridges. They're annoying to wear, they're bulky, and they're expensive; but they offer a level of protection that is reasonably well suited to the airborne hazards present in that situation.

For EDC carry though, typical N95 and N100 level masks are readily available, reasonably priced, small, lightweight, and easy to carry. They offer a reasonable level of protection if you're just trying to get out of a dangerous area, but it's not what I would want to wear if I was staying at the scene long term (like a first responder would). The big downside to these masks, though, is that they offer no protection of the eyes, so they have to be paired with some form of eye protection.

For me, that's not such a big deal, as I already wear prescription glasses with polycarbonate (safety) lenses, which affords me some eye protection. Therefore, I just toss a few N95's in my bags and what-not. However, if you're making a dedicated kit, where you would include both safety goggles and dust protection of some sort, it might make the most sense to just go with a full face unit and a few spare canisters. It's expensive, but even basic R95 full-face units offer a significant amount of protection for both the lungs and the eyes.