Elite Limited - IXN POP Connectivity Outage – Incident details

IXN POP connectivity and DWDM path resilience have remained stable following restoration of the affected DWDM link at 08:04 BST.

This incident is now resolved.

The confirmed full service outage lasted from 02:39:35 BST to 02:51:02 BST, with a customer-impacting duration of 11 minutes and 27 seconds.

The reduced-resilience period lasted from 01:46:44 BST to 08:04:51 BST, with a total duration of 6 hours, 18 minutes and 7 seconds.

We will be escalating this incident with the POP account manager and requesting a full Reason for Outage report from the data centre regarding the unauthorised rack access and disconnection/disturbance of patch leads within our rack.

The outstanding IXN DWDM resilience issue appears to have been restored.

At 08:04 BST, the previously affected DWDM interfaces returned to an UP state following work by data centre technicians. This indicates the downed DWDM link has been repatched/restored.

IXN POP connectivity and path resilience are currently operational again. We are continuing to monitor for stability before marking this incident as resolved.

We are continuing to monitor the IXN POP following the earlier restoration of connectivity.

Services remain operational; however, the POP is still operating in a reduced-resilience state while we await DC Techs restoring the outstanding affected DWDM path.

This has been chased with DC Techs on multiple occasions. We will also be raising this incident, along with previous issues observed at this POP, with the account manager for further review and escalation.

Connectivity to the IXN POP was restored at 02:51 BST.

The confirmed full outage window was from 02:39:35 BST to 02:51:02 BST, with a total impact duration of 11 minutes and 27 seconds.

Services are operational again; however, the POP remains in a reduced-resilience state while DC Techs continue work to restore the outstanding DWDM path.

We are continuing to monitor the site until full resilience is restored.

The issue has been identified as a loss of both DWDM connectivity paths into the IXN POP.

At 02:39 BST, the remaining DWDM path also went down, causing a loss of connectivity to the IXN POP.

Initial optical diagnostics indicate receive signal loss on the affected DWDM interfaces. We are aware of unauthorised activity by a data centre engineer in our rack involving cable movement, which is considered the most likely cause.

A case has been raised with DC Techs to investigate and restore the affected connectivity and outstanding resilience as soon as possible.