Investigating Errors in Roadmunk Application
Incident Report for Roadmunk

Included below is a postmortem to the connectivity issue we experienced yesterday. If there are any outstanding questions after reading, please feel free to reach out to our support team -

What happened - Customers that were not already logged in to Roadmunk were unable to log in and access the application.

Why it happened - An invalid configuration change was made to Roadmunk’s authentication and released to production which caused the application to not run properly. Please note that at no time were any accounts vulnerable to unauthorized access.

What was done to fix it - When the extent of the problem became available Roadmunk’s engineering team investigated and found the root cause. The team executed a manual rollback that deployed an earlier version of authentication to alleviate the issue and allow users the ability to log in. Then, the permanent solution was solidified and deployed to the current version. The issue was resolved within 1 hour. 

How will this be prevented in the future - Moving forward, Roadmunk deployment automation has been enhanced to block deployment on this type of misconfiguration.  In addition, code and configuration review process improvements were made to catch such changes ahead of time.

Posted Jan 21, 2022 - 20:08 UTC

Good news! We were able to make the necessary changes to restore expected functionality within the Roadmunk application. Everything is operating as normal. Thank you so much for your patience.

Roadmunk Support
Posted Jan 20, 2022 - 19:52 UTC
Hi there,

Things aren't working quite as expected. We're experiencing connectivity issues on the platform and our team is investigating. If you are currently signed in, we ask that you refrain from logging out of the application.

Thanks for your patience, we will certainly keep you posted! For any questions, please reach out to
Roadmunk Support
Posted Jan 20, 2022 - 17:37 UTC
This incident affected: Roadmapping, Authentication, Integrations, Feedback & Idea Management, and Administration.