Yesterday’s Gmail outage that’s reported to have delayed 29.1% of all email sent between the hours of 9am and 7pm ET was due to a duel network error, which caused two separate failures.
Whilst it’s not known how many users were affected, as Google doesn’t provide daily email volume data, it’s a good bet that it was a fair few, as the service does have more than 425m active users. According to the search giants, the failure was picked up quickly by its engineers but it took them around 7 hours to "repurpose additional capacity" and deal with the backlog.
The errors slowed mail down, rather than going down completely and at first this led to an average delay of 2.6s. However, the delay in sending and receiving rose later on, with some reporting that email still hadn’t arrived after more than two hours of being sent.
Reliability site engineering manager Sabrina Farmer said that the occurrence was rare, commenting: "The message delivery delays were triggered by a dual network failure."
"This is a very rare event in which two separate, redundant network paths both stop working at the same time. The two network failures were unrelated, but in combination they reduced Gmail's capacity to deliver messages to users, and beginning at 5.54am PST messages started piling up."
It’s thought that many users also experienced difficulties with downloading attachments. Google experienced a "more significant glitch” in August which took down search, Talk and Drive and this has led many to question the wisdom of using a set of businesses processes that rely on a free cloud service.
Microsoft suffered a similar issue earlier on this year when its Outlook service went down due to overheating at one of its data centres.
"We apologize for the duration of today's event; we're aware that prompt email delivery is an important part of the Gmail experience, and today's experience fell far short of our standards. We have analysed (sic) the data on user impact and are providing a preliminary assessment of what occurred,” Google said on its Gmail service status page.
The news reinforces the idea that businesses using free cloud services are leaving themselves open to risk. Many employees around the world use free storage services such as Drive and Dropbox and whilst these are very good for personal use, it has to be questioned if they are professional grade solutions.