Possible Active Directory Sync Issues
Incident Report for Duo
Postmortem

On-Premises Active Directory Sync removing all users from some groups

Incident Report - 2019/09/18

On September 18, 2019, Duo’s Engineering Team was made aware of an issue with on-premises Active Directory syncs that resulted in some groups losing all of their members during sync. In addition, attempts to remove and re-add these groups from the directory sync page would fail. Depending on customer configuration, some removed users would have been unable to authenticate using Duo.

The root cause of this error was determined to be a bug introduced in Duo’s mid-September release, which began deploying to customers at 3 p.m. EDT on September 17, 2019. This bug was introduced when fixing an issue that affected syncing users from Active Directory that had leading or trailing whitespace in their usernames.

A fix for the issue has been added to the above-mentioned mid-September release and was redeployed to all affected customers, and was finished by 11:07 a.m. EDT on September 18th. Duo engineers also ran corrected directory syncs for all potentially affected customers in advance of their scheduled daily syncs. This effort was completed by 3:37pm ET. Duo’s Engineering team has also enhanced automated test coverage in this area to prevent this issue in the future.

Posted about 1 month ago. Sep 18, 2019 - 15:59 EDT

Resolved
We have confirmed that the issue with on-premises Active Directory sync is fully resolved. An RCA will be published momentarily.
Posted about 1 month ago. Sep 18, 2019 - 15:56 EDT
Monitoring
All on-premises Active Directory sync issues have been resolved, and we have confirmed that syncs are now running successfully. Customers can now manually sync previously affected groups without issue. We will continue to monitor to ensure no further issues.
Posted about 1 month ago. Sep 18, 2019 - 14:12 EDT
Identified
Our Engineering Team has identified an issue with on-premises Active Directory sync affecting a small subset of Duo customers and is deploying a fix now.
Posted about 1 month ago. Sep 18, 2019 - 11:55 EDT
Investigating
We are currently investigating reports of potential issues with Active Directory Sync. We will update this page as soon as we have further information.
Posted about 1 month ago. Sep 18, 2019 - 09:45 EDT
This incident affected: DUO62 (Admin Panel), DUO33 (Admin Panel), DUO58 (Admin Panel), DUO50 (Admin Panel), DUO63 (Admin Panel), and DUO56 (Admin Panel).