Alkaline batteries have the advantage of lower initial cost, but are ultimately more expensive than rechargeables. Lithium batteries, because of their higher energy density, are usually cheaper as well, depending upon how good a price you get for the alkalines (buy the Costco brand for the ultimate low price).

I would like to quote a recent Consumers Reports study (Dec 2009) on this topic, but I can't get my hands on it right now.

I still keep some alkaline batteries around and they are OK for something that will be used regularly with prompt battery replacement, like a GPS receiver, but I use them less and less.

When you factor in the cost of the items they damage through leakage, alkaline batteries are surprisingly pricey.