If the schmutz is on the "hook" side of the Velcro, and it's not too sticky, try using an old clean toothbrush (and short strokes) to clean it out.