I'm going to take the opposite approach of most others who have posted. I think your primary concern, first and foremost, should be stopping power. You don't get something for nothing. The only way to accomplish low recoil is by taking away power.
I respectfully disagree. It doesn't matter how powerful the shot was that missed the target. And IMHO, the difference in real-world stopping power between .45, .40, 10mm, 9mm, 357sig, etc. is not as great as many would make it out to be. Certain calibers may be better at a given task (i.e., 357sig may be better for barrier penetration, .45 may be better if using non-expanding ammo, etc.), but if you are not expecting to run into one of these more specialized conditions routinely, these calibers are more similar than distinct in capabilities. Unless you need barrier penetration, 357sig is reportedly no better than 9mm although it is definitely "more powerful", but it will kick a lot harder and cost you a lot more for the ammo.
Recoil can be reduced in other ways besides reduction of power. My .45 is easier and more fun to shoot (less obnoxious feeling recoil) than my .380. Light and small handguns will have more perceived recoil than than a larger heavier one (unless this statement is taken to ridiculous extremes, like comparing a .22 to a .357mag)