One question: did you secure the ferro rod in any way other than the tube it slides into? It looks like you might have a para cord loop going through the handle. I would be inclined to do that to prevent the ferro rod from falling out, as long as the ferro rod leash was long enough.
Yes, actually the Primus firesteels use a small diameter shock-cord which seems to be the perfect solution. I just shortened it a bit (cut off the cord-end & tied the ends instead) and it now works perfectly. Here's a close up of it which hopefully helps:

The nice thing about the shock-cord is that it stretches into place and feels nice & secure when holding the firesteel. I've had it out in the woods a bunch of times since the original modifications and have had no problems at all with the set-up.