Encoded - Notice history

Our upstream providers have completed the rollbacks and service availability has been restored.

We will continue monitoring and provide further updates in due course.

Our upstream providers have confirmed with us that they are bringing forward their rollback of their updates to 17:00 - 18:00 BST on June 12th.

As previously mentioned, during this rollback it is expected that there will be period of downtime across all services whilst this update is applied and this is expected to last less than 5 minutes.

We will monitor services and will provide updates once the rollback has been completed.

Our upstream providers have identified the root cause of the intermittent connectivity issues. A rollback of their recent changes is scheduled to take place between 06:00 and 08:00 BST on June 13th.

We have been advised that this maintenance will involve a brief period of service unavailability across all services, expected to last less than 5 minutes during the rollback window.

Once the rollback is complete, we will continue monitoring all systems and will provide a further update.

This incident has been resolved as of 10:19am by Cashflows and post incident monitoring is in place. Please see below for full correspondence:

The issue was identified and we have performed recovery actions to restore service.
Transactions were being processed normally from 10:19am.

Technology Teams will continue to monitor.

We would like to inform you that the issue affecting transaction responses and processing has been resolved. Since taking remedial action, no further problems have been observed. The Encoded team has confirmed and marked this incident as resolved.

Incident Summary:

Time of Incident: 11:49 AM - 12:35 PM

Transaction responses were not being received or stored in our internal transaction database from any payment gateways.

Impact:

Customers and end users may have experienced issues with logins, service availability, and transaction processing during this time. Service restarts and a database failover were performed to restore normal operations.

At present, we do not believe any transactions were processed without being accounted for in customer reports. We are continuing to work closely with our payment gateway partners to verify this and ensure full reconciliation. Once verification has been completed, any affected customers will be contacted directly.

We have completed a database failover and are seeing transaction responses being stored as expected again.

We will continue to monitor services and provide updates as neccesary.

The Encoded technical team have identified the root cause and are working on restoring services.

The Encoded team have confirmed over night that emails are now sending as normal. We apologise for any inconvenience caused.

A fix has now been implemented and is being actively monitored by the Encoded team. We will provide a further update once the fix has been confirmed.

The Encoded team have identified the route cause for the issue and are working on a resolution. More information will be available once the issue has been resolved.

Between 12:20 and 13:10, an incident affected the Hosted Payment Pages, preventing customers from accessing Hosted Payment Pages based services. During this time, users encountered the error message:

"There has been an unexpected error. Please try again later."

All services have been been restored as of 13:10 and a root cause analysis is currently being performed. An incident report will produced for this incident. To request a copy of the report, please contact the Encoded Service Desk to be added to the distribution list.