Potassium iodide(KI) pills will only protect your thyroid gland, and only from radioactive iodine isotopes. It won't protect the rest of your body, like "the crown jewels", if you are directly exposed to radiation energy or are exposed to other radioactive isotopes. These radioactive iodine isotopes can be released from an accident at a nuclear power plant, but are not likely to be found in much quantities in a "dirty bomb".
As we've seen with Hurricane Katrina, evacuating a large metropolitan area quickly is not really practical, but generally speaking, during a radiological event, evacuation is the best defense if you have the time since it protects your whole body from exposure, not just your thyroid gland. But each emergency is different, so sheltering in place may be better, depending on the circumstances.
If you don't live or work near a nuclear power plant, then I personally would not bother with KI pills since it's highly unlikely you'd ever be exposed to radioactive iodine. If you do live near a plant, then as Frankie suggests, I would get in touch with the utility company or authorities for more advice. They may already have stockpiles of KI pills in your neighborhood.