Service Degradation in Roadmunk
Affected components
Roadmapping
Updates

Write-up published

Read it here

Resolved

  • What happened - The issue appeared to be a memory leak with the sync servers on our European deployment.

  • Why it happened - We do not deploy over the weekend which caused the server to not restart as it usually does. During this time period, the memory usage continued to rise on the machine until it ran out of available memory and thus created the incident.

  • What was done to fix it - We were able to identify the host having the issue and issued the reboot to fix the the host not being available.

  • How will this prevented in the future -

  • Begun an investigation of the memory usage of sync servers

  • Increased the memory and CPU on proxy servers

  • Added another proxy server for our European deployment

  • Updated our platform plans to include better alerting on this condition so that we can enable self-healing in future releases

Thu, Sep 23, 2021, 06:35 PM

Resolved

This incident has been resolved.

Mon, Sep 20, 2021, 01:26 PM(3 days earlier)

Investigating

We are continuing to investigate this issue.

Mon, Sep 20, 2021, 01:01 PM(24 minutes earlier)

Investigating

A number of our EU customers are experiencing issues accessing Roadmunk. Our team is aware and currently looking into this issue.

Mon, Sep 20, 2021, 12:07 PM(54 minutes earlier)
Powered by