We've identified and resolved multiple contributing issues behind this incident — starting with a stuck daily backup process, and further compounded by a looping coupon codes synchronization job, both of which cascaded into very high CPU usage on our primary databases. The core sources of the problem have been identified and fixed, and services are now stable. We'll continue working to optimize these processes to prevent similar downtime going forward. Thank you for your patience throughout this incident.

We've localized the root cause of the recurring database load to our coupon codes synchronization process. Our team is working on a fix, and in the meantime, we've added more CPU capacity to the database node to mitigate further outages. We'll continue posting updates as we progress.

While monitoring after the database rebuild, we've observed the issue recurring — our primary database node is again experiencing 100% CPU usage. Services may be impacted as a result. We're actively investigating the root cause and will continue working with Google Cloud support. We'll post updates as we learn more.

We've identified the root cause of the incident: one of our primary databases became stuck in a loop during a routine daily backup task. This caused the elevated load and service degradation reported earlier. We've restarted the affected database and are monitoring recovery. We'll continue to post updates as service stabilizes.

We're currently investigating an ongoing incident affecting service availability. Our endpoints are experiencing unusually high traffic volumes, and we have not yet identified a specific root cause. Our team is actively investigating and working to restore full service. We'll post updates as we learn more.

Web Client Zones recovered.

Web Client Zones went down.

Zonal POS went down.

API went down.