The study of the impact threat and the environmental consequences of impact is one focus of my research, and I can tell you that I do not lose sleep at night worrying about it. That is not to say that I do not think we should study the issue and be prepared to deal with an impending threat. We (that means all of us) have to decide, given the facts we have on the near-Earth object population and what we know (and have yet to learn) about the physical properties of these objects, how to prioritize our preparation and response to this threat given other threats we have to deal with (hurricanes, chemical spills, terrorism, etc.).

Take a look at a white paper that we (Chapman, Durda, and Gold) wrote on this subject to see where we stand:

http://www.boulder.swri.edu/clark/neowp.html

Dan