On Monday, T-Mobile experienced a network outage that knocked out texting and calling for many customers for 12 hours. Lots of folks are understandably wondering what caused the outage and what T-Mo is doing to prevent it from happening again, and now T-Mobile has answered those questions.
Neville Ray, T-Mobile's president of technology, has explained what caused the carrier's major outage earlier this week. Basically, there was a fiber failure and then the redundancy that was supposed to be the backup for that failure failed, too. Then there was an overload that caused capacity issues to the network that supports VoLTE calls.
Ray says that there was a leased fiber circuit failure with a third-party provider in the Southeast. This happens on every network, so T-Mobile has redundancy to prevent these failures from affecting customers. But then that redundancy also failed and caused an overload that was then compounded by other factors.
These issues resulted in an IP traffic storm that spread from the Southeast to cause "significant capacity issues" across the IMS (IP Multimedia Subsystem) core network that handles VoLTE calls.
During this outage, T-Mobile customers were having issues with VoLTE calling and texting. Non-VoLTE calling continued to work for many subscribers, but as TmoNews notes, more than 90% of voice calls on T-Mobile are VoLTE and so a VoLTE calling outage is going to have a major effect on T-Mo customers.
Ray confirms that T-Mobile has worked with its partners to add permanent additional safeguards to prevent this issue from happening again.