Google Gmail Outage Underscores Relevance of Offline Backup

Slide Show

Top 10 Pitfalls of Traditional Data Backup Methods

Common pitfalls that can severely affect a company's processes and bottom line.

One corner of Google's vast cloud skipped a beat over the weekend, resulting in the deletion of e-mail messages belonging to some Gmail users. Estimates of affected users varies, with Google quick to emphasize that the affected accounts range from an initial 0.08 percent to a later revised 0.02 percent of all Gmail users. Nobody has a concrete figure at this point (and Google will probably never say), though with an estimated 170 million users, I think it is fair to say that affected Gmail users could only number in the tens of thousands.


The root of the issue appears to be a software bug in a recently released storage software update that inadvertently corrupted and deleted e-mails belonging to a number of Gmail users. While Google says that the presence of backup meant that no e-mail was lost, there are a couple of lessons that we can learn from a quick examination of this episode.


The Insidious Danger of Data Corruption


Google has always prided itself for the multiple copies of data it keeps mirrored across multiple data centers. Such an architecture deserves praise in our post-9/11 world, where geographical spread and multiple redundancies are considered necessary for data survival. However, it is all too clear from the Gmail outage that hyper-redundancy offers practically zero protection against data corruption-and could ironically end up replicating bad files over good copies even faster. Not that this is the case with Google here, but the same vulnerability to data corruption applies also to outright sabotage.


The Value of Offline Backup


For all the hype surrounding cloud storage, the only real defense against data corruption or sabotage is the presence of an offline storage tier. As it is, SMBs should not automatically disregard the use of tape backup, which offers inherent offline backup capabilities. And as I've highlighted in my recent blog "Re-examine Your SMB Backup Strategy," the use of tape also instills a certain level of traceability that can serve as a potential guard against deliberate data modifications.


In fact, as I highlighted in an earlier post on the relevance of tape in SMBs, Google itself is the largest consumer of tape in the world today, using some 50,000 LTO (Linear Tape-Open) tape cartridges every quarter. Still, offline backup does have its disadvantages. In a blog update on the official Gmail Blog, Ben Treynor, VP of engineering and site reliability czar, had this to say:

But restoring data from them also takes longer than transferring your requests to another data center, which is why it's taken us hours to get the email back instead of milliseconds.

Restoration work over at Gmail continues at the time of this writing. In the meantime, I would love to hear from readers who deploy any form of offline backup in their small- and mid-sized business.