Summary
On January 30, 2025, at 10:08am EST Duo's Engineering Team rolled out a planned infrastructure update as part of a capacity planning initiative.
As a result of the changes made in that update one of Duo’s Single Sign-On (SSO) servers failed to properly reconnect to one of the necessary components that serves a key role in our authentication path. This led to authentication failures for some of our customers.
Upon being alerted of the incident by impacted customers the Duo team restarted the impacted server and authentications returned to normal. In total, 50 customers were impacted.
Deployments Impacted
DUO1
DUO2
DUO4
DUO6
DUO10
DUO13
DUO15
DUO16
DUO18
DUO19
DUO23
DUO28
DUO31
DUO33
DUO36
DUO37
DUO41
DUO60
DUO79
DUO80
Timeline of Events EST
2025-01-30 10:08 Duo rolled out a planned infrastructure update.
2023-01-30 12:50 Duo engineering was alerted of multiple customers experiencing SSO authentication failures.
2025-01-30 13:01 Duo engineering ran a restart on the impacted server
2025-01-30 13:11 The service was observed to be fully restored and all SSO authentications were
working again.
Further Details
Customer Impact
50 Duo SSO customers had at least 1 user that was impacted by this outage.
What is Duo doing to prevent this in the future?
Duo Engineering has several planned action items to improve our observability on this specific kind of issue. Our alerting failed to identify us in the timely manner of the issue.
Duo Engineering is also exploring application updates that will prevent reconnection issues like the one experienced during this outage from happening again.
Customer Suggested Steps
If you are a Duo SSO customer, please follow the guidance in our documentation to set up multiple Duo Authentication Proxies for a high availability SSO configuration. We confirmed that customers who used the recommended HA setup were impacted far less than those who did not. While Duo is working to make sure this kind of outage doesn’t happen again, maintaining a high availability setup has proven to help avoid a variety of common failures modes and is therefore still a highly recommended step for all customers using Duo SSO. More information on how to do this can be found in this Knowledge Base article.
Note: You can find your Duo deployment’s ID and sign up for updates via the Status Page by following the instructions in this knowledge base article.