All Deployments: Phone Call Delivery Issues
Incident Report for Duo
Postmortem

Summary

On Tuesday, January 26, 2021 from 15:00 EST to 15:25 EST, all Duo deployments experienced intermittent phone callback authentication failures. No other authentication methods were affected. Duo switched to a secondary telephony provider to place phone calls at 15:25 EST. Our primary provider was returned to normal service at 16:25 EST.

Details

At 15:00 EST on January 26, the Duo Engineering Team was alerted to an increased failure rate for phone callback authentications and began investigating. After determining that failures were limited to a specific telephony provider that Duo uses to place calls, the decision was made to fail over to a secondary provider.

At 15:25 EST, the team routed impacted calls to a secondary telephony provider and confirmed that new phone callback authentications were completing successfully.

At 16:25 EST, after internal validation, the primary provider was returned to normal service and phone callback authentications remained successful.

The Duo Engineering team continues to work with existing and new telephony partners to ensure redundancy and uptime in the future.

Posted Feb 02, 2021 - 10:38 EST

Resolved
The issues with phone call delivery on all of our deployments are resolved and all services are now fully functional.
Posted Jan 26, 2021 - 18:10 EST
Investigating
We have received reports from multiple customers about issues authenticating using phone callback with Duo. We are currently investigating and will post an update once we have gathered more information.
Posted Jan 26, 2021 - 16:09 EST
This incident affected: DUO29 (Phone Call Delivery), DUO17 (Phone Call Delivery), DUO6 (Phone Call Delivery), DUO37 (Phone Call Delivery), DUO26 (Phone Call Delivery), DUO4 (Phone Call Delivery), DUO36 (Phone Call Delivery), DUO9 (Phone Call Delivery), DUO12 (Phone Call Delivery), DUO10 (Phone Call Delivery), DUO65 (Phone Call Delivery), DUO62 (Phone Call Delivery), DUO64 (Phone Call Delivery), DUO27 (Phone Call Delivery), DUO46 (Phone Call Delivery), DUO49 (Phone Call Delivery), DUO43 (Phone Call Delivery), DUO44 (Phone Call Delivery), DUO41 (Phone Call Delivery), DUO23 (Phone Call Delivery), DUO18 (Phone Call Delivery), DUO28 (Phone Call Delivery), DUO60 (Phone Call Delivery), DUO59 (Phone Call Delivery), DUO31 (Phone Call Delivery), DUO1 (Phone Call Delivery), DUO19 (Phone Call Delivery), DUO54 (Phone Call Delivery), DUO15 (Phone Call Delivery), DUO52 (Phone Call Delivery), DUO7 (Phone Call Delivery), DUO3 (Phone Call Delivery), DUO58 (Phone Call Delivery), DUO57 (Phone Call Delivery), DUO47 (Phone Call Delivery), DUO8 (Phone Call Delivery), DUO38 (Phone Call Delivery), DUO25 (Phone Call Delivery), DUO22 (Phone Call Delivery), DUO40 (Phone Call Delivery), DUO63 (Phone Call Delivery), DUO16 (Phone Call Delivery), DUO24 (Phone Call Delivery), DUO13 (Phone Call Delivery), DUO21 (Phone Call Delivery), DUO48 (Phone Call Delivery), DUO66 (Phone Call Delivery), DUO50 (Phone Call Delivery), DUO51 (Phone Call Delivery), DUO56 (Phone Call Delivery), DUO53 (Phone Call Delivery), DUO34 (Phone Call Delivery), DUO30 (Phone Call Delivery), DUO55 (Phone Call Delivery), DUO20 (Phone Call Delivery), DUO2 (Phone Call Delivery), DUO39 (Phone Call Delivery), DUO45 (Phone Call Delivery), DUO42 (Phone Call Delivery), DUO11 (Phone Call Delivery), DUO32 (Phone Call Delivery), DUO5 (Phone Call Delivery), DUO61 (Phone Call Delivery), DUO14 (Phone Call Delivery), DUO33 (Phone Call Delivery), and DUO35 (Phone Call Delivery).