Indeed, the driver is the most critical aspect. Rally racers usually use front wheel or all wheel drive cars, there are some special races for trucks, but most are cars, small ones. while they are not crossing huge obstacles, they do negotiate some very rough and slick surfaces. Driver skill makes all the difference. I, like widget, have seen so many large 4x4s in the ditch, upside down or impossibly stuck, and it is all because the driver thought that the truck would do everything for them. Overconfidence in your abilities, or the abilities of your vehicle, is what gets most in trouble.

You need the right vehicle for your circumstances and the objective you need to accomplish. If you bug out route is paved roads and you are not carrying much, a small front wheel drive can do it for cheap and using less fuel. If you have to cross sand dunes, you might need something else. that is where planning comes in. In other posts there have been discussions about planning your routes, distances to travel and obstacle to overcome. If you are not going to do the analysis and planning, then go for the 4x4 with a lift kit, just learn how to drive it and make sure you are carrying plenty of fuel.