Benjammin, you might try Nido whole powdered milk. Like you, I really prefer milk that has some milkfat. I never had the Milkman brand, so can't compare the two. This stuff has a particular taste that I happen to love.
Downsides: it's quite expensive, and has shorter shelf life than nonfat milk.
Indian and Mexican grocery stores carry it. I watch for the occasional good price and/or free shipping offer on Amazon.
By the way, the Nido made in Mexico tends to be somewhat cheaper than Nido from Holland or England. I don't know if there's any difference between these products ...