I thought about this a while ago and think the explanation is this. The amount of torque the earth's magnetic field creates by pulling on the needle is proportional to the needle's length squared (R cross F integrated over the length of the needle, for you physics geeks) for a given type of magnetic material. So with a short needle you don't have much torque available. This torque must be enough to overcome the friction in the bearing. But the bearing friction is basically constant for a given type, or at worst proportional to the mass of the needle (i.e. its length).

That means to make a reliable small compass, you must either (a) get more torque by using an ultra-strong (i.e. expensive) magnetic material for the needle, and/or (b) use an ultra-low-friction precision (i.e. expensive) bearing to lower the amount of torque you need. It's far cheaper and simpler to just use a longer needle (lots of torque) and a cheap bearing.

If you're utterly cramped for space in a PSK, the idea of magnetizing a sewing needle sounds workable. You could put a tiny permanent magnet (nicely sticks to the side of your Altoids tin) in the kit, and rub the sewing needle against it (10-20 strokes all in the same direction) to magnetize it. Another idea is pry open a big cheap compass and remove the needle from it and just put the needle (permanent magnet) in the kit, suspending it from a thread or floating it to get a reading. I haven't tried either of these methods, so don't blame me if they don't work. I'd think either method would be a big pain if it's windy.

Rather than search endlessly for an ultra-small-diameter compass, maybe it's better to look for a larger diameter but ultra-thin compass. A compass could be made very thin and still have plenty of torque.