Yes, that's the crucial factor. Especially if you accept the speed of light restriction (and if you don't, you aren't really doing science any more).

In addition, there's a big difference between life and intelligent life. Life appeared on this planet as soon as conditions made it possible, but life capable of, eg, building a radio telescope for communication only appear very recently, and we have no guarantee it will not go extinct within a thousand years of achieving that. So the chances of getting two species, within 200 light years of each other, both capable of communicating/travelling across that gap, and both appearing at the same time, may be vanishingly small.