Incident Date: 2021-05-06
Affected Products:
Report URL: https://status.xendit.co/incidents/7qkrt15w7wwy
At 20:45 WIB we detected an increase of 5xx server errors from Xendit APIs and Dashboard. In the process of investigation, we found that the errors were caused by connectivity issues on some web services. We immediately focused on system recovery, Xendit APIs and Dashboard were gradually resumed to normal operations from 21:19 WIB and fully recovered at 21:41 WIB.
Our post-incident investigation revealed that there was a system maintenance to renew our infrastructure service mesh certificate at 20:30 WIB. Prior to this maintenance, we have successfully tested the certificate renewal in the testing environment. We identified the root cause was due to a process error during this maintenance. It caused the web services which were still using the old certificate to be unable to communicate with the web services which are using the new certificate.
Measures we will be taking to prevent similar issues from happening again in the future:
We know that you are counting on Xendit’s reliability for the operation of your business. We are truly sorry for the negative impact of this incident which affected you and your customers alike. We are committed to improving our reliability and to be able to serve our customers better in the future.
Please contact us at help@xendit.co if you have any questions or require any assistance regarding this issue.
We will strive to continue improving our services everyday and do our best to prevent a repeat of this incident. Thank you for your trust in using Xendit to power your business.