TIL in 1997, a Reply All storm took down all of Microsoft's internal Exchange system
TIL in 1997, a Reply All storm took down all of Microsoft's internal Exchange system
It started off with an employee sending an email to a distribution list called "Bedlam DL3" asking to be taken off the list. With 13,000 recipients and everyone replying all with, "Me too!" and other messages, it was estimated that over 15 million messages were sent through the system in an hour. This crashed the MTA service due to a recipient limit. Each time the MTA service recovered, it would attempt to resend the message again which lead to a crash loop.
As a result of the incident, the Exchange team introduced message recipient limits and distribution list restrictions to Exchange, which is something we all use today!
More on the story here: https://techcommunity.microsoft.com/t5/exchange-team-blog/me-too/ba-p/610643
cross-posted from: https://techy.news/post/2224
There is a great video from Dave's Garage about this: https://youtu.be/pBmuY6qFMPQ
I think the thing to take away from this is the poor state of management/maintenance tools that Exchange had. Thankfully over the years this has improved, but in those early years it was pretty bad.
I was using both Exchange and Lotus Notes/Domino in that period. If the same thing had happened in Notes, we'd just shut down the mail router task, open the mail.box database and remove the offending message. Easy. But that can only be done because Notes re-used its database system for everything, including the mail queue.
Exchange has a history of reinventing the wheel even within its own architecture. Let's just say I'mm very grateful that we're now on Microsoft 365 and these things are Microsoft's problem, not mine... 😉