So flickr had a hiccup yesterday. Well, truth be told, it was a major problem on their side: the image caches ran amok and delivered the wrong pics – not a few of them a bit on the more adult oriented side (as a sidenote, this proves what we all knew anyway: The Internet Is All About Porn). To the emotional outcry from lotsa lotsa users came the fact that the problem was not resolved by restarting the flaky cache server(s) but instead resurfaced once again. So finally, after quite a few hours of downtime (and I bet beet red engineers working overtime to find the bug and fix it) the system is back up.
So that's the exposition, which just about gives you an idea of the dimension of this thingy. It didn't? Well, then let me summarize: It Was BIG. However, flickr not only took down their site but pointed to their blog – in which Eric Costello did keep the users informed (if only tersely, but this is better than just a few lame marketing lines stating that all is beautiful and the system is just being enhanced yaddayaddayadda). When it was apparent that flickr would solve the problem he sat down and wrote a decent explanation of the problem – in a way to satisfy both non-technical users and the somewhat tech-savvy ones. He explains the issue without emotional overtures nor does he play it down:
To be clear, we regard this as a serious problem, but it is something that goes away as soon as we restart the malfunctioning servers (tonight we found that the servers were going insane again shortly after restarting, but we have isolated the problem and believe we have a permanent fix).
And finally, he concludes with:
We shamefacedly apologize for the inconvenience and the scare. We understand that it probably seems very, very strange and we know that many people got the impression that their photos were lost forever. But they should all be back now, safe and sound. And everyone who works on Flickr's engineering and technical operations teams are working double time to ensure that it never happens again. Thanks for your understanding and patience!
Folks, this is one of the best pieces of crisis management I have ever seen! It states the problem; it states the solution; it takes the blame where necessary and it gives a promise to the future. Now, if we could set this as mandatory teaching for all companies worldwide I would feel so much better.
Now I feel better about my glitches upgrading to WordPress 2.0.2. Just kidding. I think this is a great story.
I'll just assert one caveat, though, directed not so much to the Flickr incident as to the notion that good communication can fix everything.
Transparency and visibility are not the whole story, as important as they may be.
I recently fell back into Don Tapscott's super book from way back in 2003, The Naked Corporation: How the Age of Transparency will Revolutionize Business. (By the way, it's rated 5 stars by its Amazon peer reviewers. Don is – rightly – a cyber guru to many Fortune 500 businesses.) In it he says:
“From the marketing perspective, the message is clear. If you are going to be naked, you had better be buff.”
I love this. And as Don shows through examples, “Opening the kimono, especially when you're not superbly buff, presents risks…”
It's a great metaphor: transparency is bringing about a whole new way of doing business, in which businesses will want – and be required – to “get in shape”. So under the change in communications is a much bigger change.