An Olympic sized pool is 50 metres each way so it would take 16 complete laps to swim a mile. One of the senior swimming instructors here who also coaches Triathletes once told me that on average, people who swim lengths on a regular basis and depending on swimming style, can do 8-10 laps non-stop in a pool of this size which is just over 1/2 mile.

That said, there are many more variables out on open water that could be well beyond your control to swim this distance or shorter such as wind, current, water and ambient temperatures, clothes you are wearing etc.

If someone finds themselves in a situation, hopefully they have a PFD on as it would make a big difference even if they decided to stay put and wait for help or decide to swim to shore.