Encoded - Notice history

At 17:32 a hardware failure on the primary database server caused some connections to the database to lock. This compounded into an issue resulting in connection exhaustion, which resulted in all database activity halting.

Encoded engineers were alerted at 17:42. Following initial investigations an issue with the primary database server was quickly identified, and a failover to a secondary database was initiated at 17:51. The failover operation was completed at 17:57.

Following the failover, a number of application servers required some additional work to close existing locked connections and initiate new connections to the newly promoted database. This work was completed at 18:09.

We will be performed further investigations into the root cause of the initial primary database issue and will update with an RCA once this has been completed.

If you have any queries regarding this incident, please contact us via the Encoded Service Desk

After monitoring services for some time, we have not seen any further occurances of telephone issues and can confirm this has been resolved.

If you have any queries regarding this incident, please contact us via the Encoded Service Desk

Our upstream provider has been in touch and have informed us they have resolved this incident. Encoded engineers have conducted testing to confirm this and we have been able to process calls again without any issues.

We will continue to monitor services and mark this as resolved once further checks have been completed.

Our upstream supplier has confirmed they are experiencing a service incident and are currently working to restore services.

This incident is only affecting telephone-based services, such as IVR automated payment services and Virtual Terminal services where telephone services are used, such as assisted payments and telephone callback services. All of Encoded's other services are operational.

Our investigtions found that calls are not connecting intermittently, however once a call has successfully connected, we have not observed this being dropped without due cause and services can still be used to complete payments.

We will update this page once we have received updated from our upstream supplier.

We are seeing that calls are intermittently connecting, for calls which do not connect customers may experience engaged tones, or silent calls which disconnect.

We are continuing our investigation on this incident with our upstream provider and will provide an update once available.