I heard from a few people that Gmail stopped working yesterday for a while. Personally, I don't recall having experienced any problems or interruptions, but others reported issues from California to New York, so I wanted to get to the bottom of it.
Historically, Gmail has been pretty stable and reliable, and they have numerous overlapping mail servers for redundancy's sake, so how could a widespread 100-minute outage happen? Two words: routine maintenance.
They took a small number of mail servers out of service for maintenance around 12:30pm Pacific time yesterday, which directed the traffic those machines would normally accept to other routers/servers. Apparently the traffic was enough that the backup systems became overloaded, and diverted that traffic to other fallback hardware. This chain reaction continued until the entire system became so busy that it couldn't answer any more requests and voila, an outage ensues. However, this primarily affected users accessing the service via a Web interface. Explains Google engineering VP Ben Treynor, "IMAP/POP access and mail processing continued to work normally because these requests don’t use the same routers."
It appears Google/Gmail have, at least for a moment, become victims of their own growing popularity. The engineering team is taking the outage "very seriously," and is working to ensure that this sort of thing doesn't happen again.
Article comments
1 - El Bicho
Can't wait for your follow-up piece: "The BC Video Section Outage Explained"
2 - Brian aka Guppusmaximus
@2 - Umm, I'm pretty sure most businesses still use MS Outlook which can be set-up to utilize Gmail. If not Outlook, then those people already know about Thunderbird & have, probably, already made the transition.
BUT, Just like Mark I,too, didn't experience any downtime.