"...you can leave the country on short notice..."

Other than going thru Canada a few times a couple of years ago (to and from Alaska), we are not international travelers. As I understand it, you need a visa in addition to a passport in order to enter most foreign countries, and you don't get one of those instantly. Is that correct, or am I misinformed yet again???