Matt - great post. You inspired me to give it a whirl with all the firestarters I've acquired in the last couple years, for "just in case." And my results? DISMAL. It's now very clear that it's the skills, not the gear. I'm humbly asking for some tips to make it go more successfully in the future.
I grew up in a rural area and feel rural at heart, but have been a city dweller since college. For years I lit fires in our fireplaces/Franklin stoves with matches and newspaper, and I think that's led to a bit of my overconfidence. I've never been a Boy Scout, and the camping I've done (not very recently, unfortunately) used a multi-gas stove.

I had to do my test in more controlled circumstances than Matt's - rooftop, in a small charcoal grill (to keep flames from the tarpaper roof). For tinder, I had some wood chips, vaseline-soaked cotton, "lightning bug" sawdust tabs, and fire ribbon.
My firestarters included: a strike-force, a magnesium/flint bar, a mini flint/magnesium firestarter, and a mini Bic.
Getting sparks was not a problem - all items drew sparks, some more successfully than others. The strike force seemed to consistently give the most volume/longest throw on sparks. Also, a variety of items worked as a striker - the small piece of hacksaw blade that came with the mini firestarter, the back of a carbon steel Mora, the back of an ATS-34 blade, the back of an AUS-8 stainless blade, and a P-38 can opener.
The problem was getting the sparks to catch. Since purchasing these items, I've tested the striker just to see how well the sparks fly, but not used them to start fires before (or, sadly, yet).

A couple things I noticed that helped during the test - rather than using the flat of the back of a blade, it improved spark output and throw to tilt it and use the corner. Also, the more pressure put on the striker, the more impressive the spark output.

I tried both pushing the striker forward, and holding the striker in place/pulling the flint back (I read that suggestion here on ETS) - neither seemed to make a difference, although the pull was a bit more awkward for me. My results tended to be - several sparks along the length of the flint, one or two leaping the inch or two the the tinder, and nothing catching. The Strike Force would put out a slightly higher volume of sparks/slightly longer throw (I think because the handle made it easier to grip firmly) - but still nothing wanted to catch - even a nice dollop of fire ribbon.
Same with the magnesium bits I scraped off. First, they didn't seem to come off very neatly - very irregular-sized shavings, and they tended to go all over, not into a neat pile. How big a pile do people usually create before trying to spark them? How thick/tightly packed is your pile? How big are your shavings (mine tended to look like 1/16-inch long or shorter bits of waste from drilling through metal - no long whittled bits).

I've been watching the Survivor All-Stars show (guilty pleasure), and when they gave them magnesium firestarters, I was sitting there criticizing the way they were hacking at them and getting feeble results. I'll clap a stopper on my armchair criticisms now <img src="images/graemlins/crazy.gif" alt="" />

So, my list of questions:
--For the vaseline/cotton balls, how much vaseline? I read other posts, and softened the vaseline in the jar in the microwave and dipped the balls in - and they were sopping with vaseline. I've had it in film canisters, and yesterday when I tried to fish it out, I couldn't get a ball to come out - wisps of cotton suspended in vaseline came out. I'd read that I should fluff/shred it a bit, but it wasn't that easy to handle.
--How close are you holding your flints to the tinder when you strike? How far should I expect to throw sparks to get a successful fire, and what kind of volume of sparks should I expect to see to know I'm doing it correctly?
--Any vital details I'm missing on angle/pressure/how quickly to strike?
--I have trouble not bumping/jolting the tinder (or the tiny charcoal grill) if I'm doing a series of strikes.
--Any hints about shaving the magnesium/piling it/keeping a cohesive pile/how much to use?

This has been a great wake-up call for practice. If it had been one of the firestarters that I tried and it didn't work, I'd flatter myself that it was a Janet Jackson-style equipment malfunction, but I can't blame the tools for this one. I shudder to think about being in an emergency, trying to hold onto the mini flint with numb fingers and NEEDING a fire.
One comment on the fire ribbon - very impressive stuff. I think it would be good if they made very tiny tubes of it - like lip balm sized tubes.
Oh, and the mini-Bic? Lit everything quickly and easily. I'll continue to EDC a Windmill butane lighter and a mini Bic.

OK folks, please help the awkward ETS member.
Thanks
Dave