More

    Downtime Report: Top Ten Outages in 2013

    2013 has seen some massive outages. And given our heavy reliance on technology today, there is more at stake than ever before. Outages affect not only internal users, but a company’s customers and partners – and impact revenue, credibility, trust, reputation and productivity.

    With all of this in mind, Neverfail wanted to put together its own list of the year’s top outages – placing them on a scale based on overall impact of their downtime. The criteria used to assess the scale ranges across multiple factors:

    Expansive reach – Businesses increasingly depend on the cloud for applications and access to data, so there’s more at stake today than before. In today’s interconnected world, an outage can have a rippling effect across a company’s user base, the country and even the globe.

    Damaged reputation – No one is perfect. But as customers become increasingly dependent upon the cloud for applications and access to their data, perfection is exactly what those customers demand. So, big outages draw a lot of media attention and can quickly put a company under attack. Not to mention the fact that user forums and social media platforms like Twitter have become the automatic and all-encompassing soap box for all irritated customers to expound on their complaints. Companies that rely on other cloud platforms to provide their own products and services may also see their reputation adversely affected if the cloud provider has an outage.

    Lost revenue – It is nearly impossible to determine the exact cost of downtime, since so much depends on the organization, the industry, the number of people impacted, etc. For example: A Standish study estimated that credit card applications lose around $2.6 million for every hour of downtime, whereas this year’s 49-minute Amazon.com outage reportedly cost the online retail website nearly $5 million in deferred revenue.

    It also appears that the cost of downtime is increasing. According to Gartner, in 2005 organizations lost $42,000 every hour of downtime. In 2011, it was estimated that IT downtime costs $26.5 billion in lost revenue each year and another study suggested that the average cost of data center downtime across industries is approximately $5,600 per minute.

    By that estimate, the top 10 outages equal a whopping $31,214,400 in lost revenue – and that only accounts for the providers themselves, not their end customers. Ouch!

    This slideshow summarizes the top 10 outages identified by Neverfail. We actually reviewed over 30 major outages; we’ll publish that list and some additional analysis next month.

    Downtime Report: Top Ten Outages in 2013 - slide 1

    Click through for the top 10 online outages of 2013, as identified by Neverfail.

    Downtime Report: Top Ten Outages in 2013 - slide 2

    Microsoft’s Windows Azure

    • Date: October 30, 2013
    • Duration: Over 20 hours
    • Failure: A sub-component of the system failed worldwide.
    • Impact: Every single Azure region was affected (including West U.S., West Europe, Southeast Asia, South Central U.S., North Europe, North Central U.S., East Asia, and East U.S.).

    Downtime Report: Top Ten Outages in 2013 - slide 3

    Google

    • Date: August 16, 2013
    • Duration: Less than 5 minutes
    • Failure: All of its services went down.
    • Fallout: The volume of global Internet traffic plunged by about 40 percent.

    Downtime Report: Top Ten Outages in 2013 - slide 4

    Amazon Web Services

    • Date: Sept. 13, 2013
    • Duration: Under 3 hours
    • Failure: Connectivity issues affected a single availability zone, disrupting a notable portion of Internet activity.
    • Reminder: If you rely heavily on the cloud for your infrastructure, have a failover plan.

    Downtime Report: Top Ten Outages in 2013 - slide 5

    NASDAQ

    • Date: August 22, 2013
    • Duration:  3 hours
    • Failure: A software bug, followed by inadequate built-in redundancy capabilities, triggered a massive trading halt in the U.S.
    • Impact: With all the exchanges dependent on one another, this outage had impact rippling across the globe.

    Downtime Report: Top Ten Outages in 2013 - slide 6

    OTC Markets Group Inc.

    • Date: November 7, 2013
    • Duration: Over 5 hours
    • Failure: A network failure due to a “lack of current quotation information,” prompted a complete shutdown in trading of over-the-counter stocks in the U.S.
    • Impact: The shutdown happened on one of the biggest trading sessions this year as Twitter Inc.’s shares debuted. While the disruption only paused less significant equities such as Fannie Mae and Freddie Mac, it tested investors’ nerves following a series of technical mishaps since August and exacerbated concerns about problems in the electronic infrastructure underpinning U.S. exchanges.

    Downtime Report: Top Ten Outages in 2013 - slide 7

    HealthCare.gov

    • Date: October 27-28, 2013
    • Duration: 16+ hours
    • Failure: A service outage at a Verizon Terremark data center caused downtime for HealthCare.gov, the trouble-plagued online insurance marketplace created by the Affordable Care Act.
    • Impact: With all of America watching the progress of the trouble-plagued online insurance marketplace created by the Affordable Care Act, a data center outage only added more fuel to the flame and perhaps made the public question where to point the finger of blame.

    Downtime Report: Top Ten Outages in 2013 - slide 8

    Amazon.com

    • Date: January 31, 2013
    • Duration: 49 minutes
    • Failure: Internal issues caused the Amazon.com home page to go down, displaying an error message.
    • Impact: The outage demonstrated the extremely high value of uptime to services such as Amazon. Analysts calculated that one hour of interrupted service may have translated to $5 million in lost revenue.

    Downtime Report: Top Ten Outages in 2013 - slide 9

    Microsoft – Hotmail and Outlook.com

    • Date: March 13, 2013
    • Duration: Nearly 16 hours
    • Failure: A firmware update caused the company’s servers to overheat; Hotmail and Outlook.com both suffered a loss of service.
    • Impact: Microsoft admitted that it required some human intervention to bring the services back online, thus delaying the restoration attempt further. Microsoft’s online service reputation took a big hit.

    Downtime Report: Top Ten Outages in 2013 - slide 10

    Google Drive

    • Date: March 18-20, 2013
    • Duration: 17 hours total
    • Failure: A glitch in the company’s network control software, which caused latency and recovery problems. Users faced slow load times or full-on timeouts while trying to access their Drive documents and files.
    • Impact: As much as one-third of the customer base was impacted, leading to a virtual hue-and-cry across the Internet.

    Downtime Report: Top Ten Outages in 2013 - slide 11

    Google’s Gmail

    • Date: September 23, 2013
    • Duration: 12 hours
    • Failure: Prolonged slow download times were triggered by a dual network failure.
    • Impact: The outage affected 29 percent of users. For 1.5 percent of Gmail messages, the delay in downloading large attachments was up to two hours. While its impact may not have been catastrophic, the outage at Gmail is a potential cause for concern, especially as businesses are turning to Google and other providers to run cloud-based email and SaaS.

    Downtime Report: Top Ten Outages in 2013 - slide 12

    Yahoo Mail

    While the index measures through the end of November, the very recent Yahoo Mail outage deserves a considerable honorable mention:

    • Date: December 9-13, 2013
    • Duration: Almost 4 days
    • Failure: A specific hardware problem in one of the company’s storage systems caused the prolonged partial email outage for users.
    • Impact: The multiday email outage impacted countless individuals and the many small businesses that rely on the service. Not only did the outage cast a dark shadow over the once-mighty Internet player, but the company was also majorly criticized for the way it handled its damage control, particularly its negligence in informing its users about the problems.

    Latest Articles