Good going thinking of the orange vest as a flag. Survival is often a matter of using what you have in unusual ways. Making the most of what you have on hand.
Within the various groups concerned with survival there is a subset of those who take a more militarized approach. For them everything has to be 'tactical', subdued or camouflaged in some way. The preferred mode is to not be seen.
There are a few situations where being less visually obvious might be an advantage. Given the numbers of recreational campers who flood into the woods it is always better if their gear is drab. It doesn't take many orange and bright blue tents on a mountainside to convert a beautiful vista into visual pollution.
But sometimes you want to be seen. You don't want to be mistaken for a deer during hunting season and if you need to be found, and/or rescued being easily spotted is going to help.
The old military parkas had a OD green side and a Naval Distress Orange, day-glow orange, reverse. Sometimes you want to blend and sometimes you don't.
A standard hi-visibility vest could come in handy if you have to evacuate by walking down a road. Getting run over by a exhausted or inattentive driver that doesn't see you until too late isn't going to help you any. A bright vest might make the difference between escape or being a casualty.
Those day-glow vest are pretty light and they fold up well. Choosing you clothing, other gear with an eye toward when you do and don't want to stand out will help.
You might also consider one or two of these:
http://www.bestglide.com/VS17_Signal_Panel_Marker.htmlAt 24" by 70" it would serve as an emergency ground cloth or sun shelter. Two connected would make a decent shelter and one that would be both durable and easy to spot.
That might be a bit bulky for smaller kits where size and weight are at a premium. Smaller kits can sometimes benefit from inclusion of some surveyor's flagging tape:
http://en.wikipedia.org/wiki/Flagging_(tape)
It comes in a wide variety of colors and widths from 1" to 6". The most common rolls are 1" wide and better than 500' long wound on a cardboard core is about the size of a hockey puck. Shorter lengths can be wound up without the core and can give you a hundred feet or more in a compact and light bundle.
These can be used to mark trails, your travel route or longer pieces tied to or between anything handy to make a highly visible signal that moves in the wind.