Degraded performance of Prospect CRM

Incident Report for Prospect CRM's

Postmortem

At approximately 11:40pm on the 7th March 2024, an issue with a memory leak triggered auto-heal rules on the Microsoft Azure service. This resulted in poor performance on the OData instances and degraded performance in the Prospect Apps.

The issues were identified by the Support team and immediately raised with the Infrastructure and DevOps Engineers. Once the root cause had been identified they reduced the load on these servers to mitigate the issues.

The issues caused by the auto-heal rules will now be fully investigated to make sure they do not reoccur before server load is increased again.

Posted Mar 07, 2024 - 14:36 GMT

Resolved

This incident has been resolved.

Posted Mar 07, 2024 - 12:32 GMT

Monitoring

A fix has been implemented and we are monitoring the results.

Posted Mar 07, 2024 - 12:25 GMT

Investigating

Some customers have reported slower than expected performance of Prospect CRM and associated services. We are currently investigating and will provide further updates as information becomes available.

Posted Mar 07, 2024 - 12:15 GMT

This incident affected: Individual Prospect Apps (Prospect CRM App, Prospect CMS Editor App, Hosted Customer Websites, Portals and eCommerce sites) and APIs, Login, and Core Platform.