It's true that the numbers on H1N1 are not particularly scary compared to seasonal flu, but we haven't even been through a full flu season with H1N1 yet. I'm not saying this to be alarmist, but the potential impact from H1N1 does seem worse than seasonal flu, though. It has traits like:

• It's more infectious than seasonal flu. Even if H1N1 stays mild for the rest of the flu season, it will likely cause more hospitalizations and deaths than seasonal flu simply because more people will get infected than usual, even if any single person's risk of hospitalization or death is lower than with seasonal flu.
• People stay infectious longer with H1N1 which also increases everyone's chance of catching it (I cited the article in Art's H1N1 thread)
• The age distribution of deaths or cases with severe complications is more like past pandemic influenza's rather than seasonal flu. Instead of killing the elderly and very young children, it is hitting older children and adults younger than 50 the hardest. If H1N1 infects enough people, history will be recounting how H1N1 struck the young.
• H1N1 causes lung damage which seasonal flu cannot inflict. This story describes one expert's experience after 90 autopsies of H1N1 deaths and likens the damage he has seen with avian flu, and this article describes the mechanism of why H1N1 can directly attack deep lung tissue that seasonal flu cannot do. If we're lucky, H1N1 will not develop a stronger affinity for lung receptors later on. If it does, then it will be sending a lot more younger adults to the ICU.
• The fact that H1N1 stuck around during the summer months and is causing fresh outbreaks in August and September is pretty unheard of for the flu, and suggests the possibility of a long, tough flu season ahead.

Overall, we're lucky because not a whole lot has happened. At least so far. The typical flu season is not usually even considered open until at least October and most years, the peak months are January and February.

And even for people who don't catch H1N1 directly, it could still affect them. For example, maybe you need to go to the ER for something else like a car accident, but all the ones closest to you could be swamped with flu patients and they turn you away. It's already happening right now in places with fresh outbreaks. (Not that getting a flu shot can prevent car accidents. Wish we could develop a vaccine for that!)