SPOT FAILS UNDER PRESSURE

Saturday July 5, 2008 the SPOT - FINDMESPOT rescue device failed under very visible conditions.

Kent Couch of COUCHBALLOONS.COM flew a lawn chair with helium balloons from Bend, Oregon to Cambridge, Idaho.

One of the sponsors, SPOT, provided tracking information for publicity and rescue. According to Mark Knowles, event coordinator, an hour into the flight, the FINDMESPOT servers failed support.

Kent landed safely no help to SPOT. Ground crews in Idaho were promised a mashup between FINDMESPOT and GOOGLEMAPS. The anticipated recording of the flight over the Snake River and into Idaho was lost.

Excuses are many. One offered is that the servers were hit too hard by the National media exposure. Maybe SPOT is not as serious about there products capability as they advertise. Had Kent needed rescue, that excuse would have been of limited comfort.

Had James Kim been depending on the SPOT product, his death would not have boded well for SPOT

Disappointing. I was going to consider SPOT for the more mundane aspects of their service (tracking, etc). If it can't even do that...

I wonder if Les Stroud is going to regret being their celebrity sponsor. Every ad I see has him looking all rugged but with a SPOT on his belt LOL.

Anybody flies in a lawn chair with helium balloons deserves SPOT.

Hopefully they will find out why it didn’t work and work out the bugs. The idea of one of these devices is great.

Failure is only failure if you quit. It can be used as a learning experience that points out problem areas and lack of planning that can then be refined for future product refinement or service improvements.


No one was killed and with work this product & service could go on to save lives.

Les probably has an out clause or two in his contracts with SPOT, but either way no one died as a result of the products failure so there's nothing to regret... Unless he stays on and something bad happens.

I find it interesting (read as ridiculous) that SPOT would let an idiot in a lawn chair strapped to some balloons show their product so prominently on his site.

That would be enough for me to want to jump ship right there.

Regarding the SPOT as a rescue device in this particular situation: If something went wrong and the balloonatic needed to send a call for help, what could anyone do except track him until he finally lands or find out where he fell out of the lawn chair?

Temporary glitch or not in SPOT's performance, Doug laid out enough reasons for me to purchase something like the ACR Microfix over the SPOT should I finally take the leap and buy an item like this.

Sorry, it begs the obvious answer... shoot the balloons one at a time.

There. I said it. Sorry. But I did enjoy the mental picture and think it would be the most appropriate considering the nature of the entire event.

That's what the first guy who did it did. This stunt has been pulled many times. This is the first time I've heard it getting corporate sponsors :P

FWIW, there is a difference between the failure of back end systems and the failure of the user facing web stuff (although, if poorly designed, they can become related).

These online media events tend to be a learning experience for all companies. They often just don't understand the scaling issues very well the first time out.

I noticed the same kind of thing with SPOT and the Primal Quest coverage.

My guess is that this will help them build a better system and hopefully they will learn that they must keep the emergency use and the web tracking separate, if they already do not.

-john

besides SPOT,does anybody know if the guy had a parachute and basic survival gear with him on his flight?..or at least a few P.B.J sandwitches and water..vince g. 11b inf..

He had a parachute, and even used it on his previous attempt after shooting out too many baloons.

"...Anybody flies in a lawn chair with helium balloons deserves SPOT..."


Well said...

As an interesting aside, I just saw a commercial for the ACR Microfix on the Discovery Channel.

"...find out where he fell out of the lawn chair..."

The last one of these lawn chair idiots that I heard about was actually smart enough to wear a parachute. Don't know if this particular idiot did or not. But if he did, he had to bail, and landed in a boonies, SPOT would/should have been able to pinpoint him so that a rescue chopper (which would hopefully bill him bigtime) could find him. But since it went belly up, he could have been in for a long hike out...

I work in the web development industry, and I know about server load issues. Basically, you can see an exponential increase in traffic even from a moderate level of media and blogger coverage. One of my clients is a major financial services firm and we have had a few experiences where a national commercial ran and just melted the servers down. Building highly scalable systems is highly challenging.
That's why I'd never trust anything via the Internet for life safety. Anything public is highly susceptible to spike loading by accident - or in the case of a DDOS attack, by malice.
The FINDMESPOT servers could be deliberately targeted for such an attack, and there's not much that can be done. The same can not be said for networks that are not public.

New meaning to armchair adventurer.

http://news.bbc.co.uk/2/hi/americas/7491841.stm

The 'slashdot effect' is its own phenomenon. As soon as a story is posted (its like Digg.com but for geeks - www.slashdot.org), and if the story is associated to a web site thats on poor equipment... in a matter of seconds its down.

I haven't followed this much but I do recall that the weight budget for his "lawn chair" was over 300 lbs ... that's quite a lawn chair..

The biggest question in my mind is the sheer number of balloons. I think I read he was going to get 4 lb of lift from each balloon and had a total flight weight of 600 lbs, or 150 balloons. How are those tethered such that none are ruptured by another's tether rubbing against it?

When he shoots a balloon, shooting upwards would hit many balloons stacked vertically, also resulting in many tethers rubbing against balloons as the remaining balloons adjusted. Shooting to the side might not be practical either for that reason and for balance reasons.

In any case it sounds safer than the Swiss guy who strapped on a jet engine and flew over the Alps outside of the intervals when the lawn chair is too low for parachute bail-out.

It's possible the deal with the slashdot problem, *if* you're willing to spend vast sums of money. I doubt SPOT has the budget for it, so they need to separate their business, marketing, and operations networks and internet presences, and be prepared to handle operations via FAX or phone in case of a DoS attack to reassure customers. That said, this was probably one of those cases where the cleaning lady went to unplug the vacuum cleaner and yanked out an Ethernet switch power cord instead...

This is the catch 22 of the dev industry.

If the server stays up for the big event, then, you can wonder if they are being as careful about quality when they aren't on TV. Maybe they use resources for that one guy that they wouldn't ordinarily use, making the big demo a fraud.

On the other hand, if the server fails on the big event, that doesn't necessarily mean it would fail for you on an average day when you didn't have an unusually large number of people trying to track you.

Certainly, its egg on the face, but I feel their pain.

I hope they work it out... its all good experience, if they can hang on until the next iteration...

This is John Dark with SPOT.

I saw this thread about Couch Balloons, and thought readers might be interested in some of the facts. We have posted a note about Couch Balloons over on our new Blog.

http://findmespot.wordpress.com/2008/07/15/spot-successfully-tracks-kent-couch-at-couchballoonscom/

It also includes a document that lets developers know how to integrate SPOT into their site, which seems to have been a sticky point for our friends at couch balloons. We have a separate workaround if a site will get more than 500 views in 15 minutes, but users need to contact us so we can work directly with the developer.

Hi John,

Nice to see you here. Maybe you could talk a little about the web tracking during Primal Quest?

A number of times I visited the site it seemed unresponsive.

I'd be interested to know:

1) What you might be doing to address bursts of traffic, esp. during events. While I understand events are a special, artificial situation, it seems that a similar type of situation might arise during a normal emergency if the public caught wind of it (think James Kim case).

2) How the emergency aspects of your system are affected (or not) by the overloading of the web presence.

Thanks!

-john

Hi JohnN,

John Dark from SPOT here.

Thanks for asking about Primal Quest. It was an awesome event, and we were very fortunate to be a part of it. Wade Nelson does a nice writeup over at Hardcore Outdoor: http://hardcoreoutdoor.com/2008/07/04/primal-quest-2008-montana.aspx

Regarding the race tracking software, here are the answers to your questions:
Responsiveness -- there are two factors here. The first day, the server hosting the race tracking database was running slowly, causing between 20 and 30 minute delay in updates. The team swapped servers over the first several hours and after that updates came in within 2 minutes from the burst of data from SPOT to the browser.
Separately, there was a ton of data driving these pages, I found that the first load was slow due to file size, but that hitting refresh was very fast. The folks at Primal Quest were definitely satisfied. Not only did they use SPOT for race tracking, but for critical communications with their medical staff and support teams, including two 911 team rescues where the button was pushed, and another dozen teams that they pulled out of dangerously high water by simply tracking their points.

1& 2) Regarding how we handle emergencies: We have multiple redundancies between us and three GEOS locations (primarily Houston), with redundant 9-1-1 monitoring at three of our Gateway management centers, so even if there was no data appearing within the SPOT web service or on Shared pages, the emergency service continues to work normally. Separately, for events, we can use web cloud services such as Amazon, in addition to our own robust servers, to handle the high volume of page views.

John Dark from SPOT here.

For a more current example of how SPOT can be used to publicly track someone, check out how this team is using SPOT in a search for Steve Fossett's location. They use SPOT for public tracking, and as an archive of search locations. You can track them as they search in real time and it's very interesting to see the ground that they're covering. We have posted a note and link to it over at our blog: http://findmespot.wordpress.com/2008/07/17/spot-used-by-teams-searching-for-steve-fossett/

John Dark from SPOT here.

Here's another example of how SPOT is used for tracking and checking in. You can also see how the EAA is using SPOT to track some of the historic planes flying up to Oshkosh for AirVenture 2008: (available until the event in a couple of weeks) http://findmespot.wordpress.com/2008/07/...otor-jw-french/