Indicator computation and Ticket creation is not working
Resolved
May 22 at 12:30pm CEST
All stalled indicator computations have been performed, and the incident is now over.
Affected services
API — Lonza
Application
Updated
May 22 at 12:15pm CEST
Redis capacity has been increased. Ticket creation is back to normal and indicator computations that were stalling and now being processed
Affected services
API — Lonza
Application
Updated
May 22 at 11:55am CEST
We have identified the root cause of the incident. It is due to a lack of capacity in Redis, the technology we use for caching and to perform long-running operations, such as exports and computation of dynamic indicators. While the infrastructure was under capacity, computation of dynamic indicators stalled.
We have increased the capacity of Redis and the new capacity is being deployed.
Affected services
API — Lonza
Application
Created
May 22 at 11:50am CEST
Indicators are stuck in a recomputing state and tickets are failing to get created
Affected services
API — Lonza
Application