The FCC has released a scathing report detailing the cause and impact of a nationwide CenturyLink network outage that occurred late last year, along with recommendations to help prevent similar outages from occurring in the future.
It could be a prelude to a serious fine, especially considering that in 2015, CenturyLink was zapped with a $16 million fine from the FCC for an outage failure in seven states in April of 2014 that affected 11 million people and disabled 911 calls.
Their most recent serious outage resulted in extensive disruptions to phone and broadband service, including 911 calling. As many as 22 million customers across 39 states were affected, including approximately 17 million customers across 29 states who lacked reliable access to 911. At least 886 calls to 911 were not delivered.
“This massive ‘sunny day’ outage was completely unacceptable and impacted millions of customers across the country. Americans expect and deserve reliable phone and broadband service—especially the ability to call 911,” said FCC Chairman Ajit Pai. “It’s important for communications providers to take heed of the lessons learned from this incident. I also thank the Public Safety and Homeland Security Bureau for its thorough investigation and recommending next steps to promote network reliability.”
Beginning on December 27, 2018, CenturyLink experienced an outage on its fiber network that lasted for almost 37 hours. As described in the report, the outage was caused by an equipment failure that was exacerbated by a network configuration error.
The outage affected communications service providers, businesses customers, and consumers who relied upon CenturyLink’s transport services, which route communications traffic from various providers to locations across the country.
The report outlines lessons learned from the incident and identifies specific network reliability best practices that, if implemented, could have prevented the outage, or at least mitigated its effects, including:
- Turning off or disabling system features that are not in use;
- Including in network monitoring memory and processor utilization alarms that are regularly audited to ensure functionality and evaluated to improve early detection and calibration; and
- Having standard operating procedures for network repair that address cases where normal networking monitoring procedures are inoperable or otherwise unavailable.