I did exactly the same thing for my kids, 5 and 7. Put the whistles on breakaway lanyards.
Funny thing, my 5-year-old son is like the tortoise: slow to moderate paced, but he never rests and is a merciless taskmaster. He's slow so one of the adults would take turns walking with him as he fell behind the main group, but eventually the faster party would stop to take a rest or drink or whatever, and my son would catch up and walk through the group blowing his whistle to get everyone moving again. Hysterical!