All systems operational

Resolved
Delays with Automations and Custom Actions

Started
05 December 2022 at 2:54:02 PM
Status
Resolved after about 16 hours

Impact

Operational
Affected
JobWatch
Client Site
Mobile Application Services
  • Resolved

    The rollback was completed successfully. This issue is now resolved.

  • Monitoring

    A rollback to our primary node is required. This corrective action will take place this evening at approximately 18:30.

    A minor outage / increased error rate may be experienced for a brief period, whilst this action takes place.

  • Monitoring

    The underlying issue has been resolved.

    Our engineers are continuing to monitor the platform.

  • Identified

    We are continuing to work on a resolution for this incident.

    Updated list of affected services;

    • Automations and Custom Action processing
    • Jobs and Stock updates being shown in the UI
    • Outbound messages such as Job Details from Server to Devices

    Our engineers are on the case and another update will be provided shortly

  • Identified

    Our teams are currently actioning a failover between our Primary Nodes.

    A small outage might be experienced as connections to the database are switched between the SQL nodes.

  • Investigating

    We are currently investigating an incident whereby the following actions are current delayed;

    • Automations and Custom Action processing
    • Jobs and Stock updates being shown in the UI

    Our engineers are on the case and another update will be provided shortly