Platform Emergency maintenance
Incident Report for CM.com
Postmortem

On the night from Friday, April 30 to Saturday, May 1 (CEST), we’ve encountered an issue with 2 new network switches.

This led to an interruption of the traffic in one of our data centers, which impacted several services, and a delay in part of the SMS traffic. This is likely a result of an unusually high peak in our network traffic, caused by a load test we performed. In the course of the night, the issue was solved, after which we needed Saturday morning to restore all impacted services in a controlled way.

Due to this unique turn of events and multiple technicians being involved, it will take some additional weeks to come to the definitive root cause. We perform this analysis together with the network switch supplier, in the meantime, we have taken the proper precautions to ensure a similar interference will not occur.

Posted May 05, 2021 - 16:37 CEST

Resolved
Fixes have been implemented to resolve the issues. Services operate as expected again.
Posted May 01, 2021 - 14:27 CEST
Update
We are continuing to work on a fix for this issue.
Posted May 01, 2021 - 11:49 CEST
Update
We are continuing to work on a fix for this issue.
Posted May 01, 2021 - 11:10 CEST
Update
We are continuing to work on a fix for this issue.
Posted May 01, 2021 - 10:54 CEST
Identified
The issue has been identified and a fix is being implemented.
Posted May 01, 2021 - 10:07 CEST
Update
We are continuing to investigate this issue.
Posted May 01, 2021 - 10:02 CEST
Update
We are continuing to investigate this issue.
Posted May 01, 2021 - 08:51 CEST
Update
We are continuing to investigate this issue.
Posted May 01, 2021 - 07:53 CEST
Update
We are continuing to investigate this issue.
Posted May 01, 2021 - 06:15 CEST
Update
We are continuing to investigate this issue.
Posted May 01, 2021 - 05:16 CEST
Update
We are continuing to investigate this issue.
Posted May 01, 2021 - 01:46 CEST
Update
Voice platforms are back up and performing as expected. However, we continue investigating degraded performance on our SMS Gateways.
Posted May 01, 2021 - 00:11 CEST
Update
We are continuing to investigate this issue.
Posted Apr 30, 2021 - 23:51 CEST
Update
We are continuing to investigate this issue.
Posted Apr 30, 2021 - 23:03 CEST
Investigating
Due to required emergency maintenance on our platform, you may experience some irregularities. Once the maintenance is done we will place a notification on the status page.
Posted Apr 30, 2021 - 22:33 CEST
This incident affected: Messaging (Business Messaging API), Mobile Marketing Cloud (Customer Data Platform, Campaigns, Pages, Email Campaigns, Voice Campaigns, Address Book), Platform (Platform Apps), Mobile Service Cloud (Customer Contact, Scripted Chatbot), and Voice (Voice API's).