DUO33, DUO50, DUO56, DUO58, DUO62, and DUO63: Enrollment emails sent out via Directory Sync even if email option is not enabled.

Incident Report for Duo

Postmortem

Summary

From 01:00 AM to 07:50 PM EDT on Tuesday, March 17, 2020, unexpected enrollment emails were sent after Active Directory, Azure Active Directory, and OpenLDAP syncs ran, even though the syncs were not configured to send enrollment emails to unenrolled users. This issue only affected Directory Sync on DUO33, DUO50, DUO56, DUO58, DUO62, and DUO63.

Duo’s Engineering Team has identified the root cause of this incident, and we are committed to improving our development and testing processes to prevent the possibility of similar issues moving forward.

Details

The root cause of this incident was determined to be a bug introduced in Duo’s mid-March release, which began deploying to customers on March 16. The bug was escalated to Duo’s Engineering Team, and the release was halted on March 17.

A fix was released to all affected deployments by 7:42 p.m. EDT and the regular release process was resumed.

The enrollment links contained within these emails are now deactivated. Unenrolled users who received these emails in error will no longer be able to use them to enroll in Duo. If unenrolled users click the link, they will see the message “This enrollment code has expired. Contact your administrator to get a new enrollment code.”

We will follow up with affected customers with more information tomorrow.

Posted Mar 18, 2020 - 20:03 EDT

Resolved

After monitoring the issue in which enrollment emails were being sent out via Directory Syncs without being enabled, we can confirm that the issue has been fully resolved.

A root-cause analysis (RCA) will be posted here once our engineering team has finished their thorough investigation. Check back in or subscribe to be notified of the RCA.

Posted Mar 17, 2020 - 20:47 EDT

Monitoring

The issue in which enrollment emails were being sent out via Directory Syncs whilst the email option was not enabled has now been resolved. This affected customers on DUO33, DUO50, DUO56, DUO58, DUO62, and DUO63. A fix has been deployed and all services are now fully functional.

We will be posting a root-cause analysis (RCA) here once our engineering team has finished its thorough investigation of the issue.

Please be sure to check back or subscribe to be notified when the RCA is posted.

Posted Mar 17, 2020 - 19:50 EDT

Update

Initially DUO60 was incorrectly reported as being affected, and DUO56 was not listed as being affected. We have corrected this error and apologize for the inconvenience. We continue to actively develop a fix for this and will provide an update as soon as it has been deployed.

Posted Mar 17, 2020 - 17:27 EDT

Identified

We are aware of an issue causing enrollment emails to be sent out via Directory Syncs even if the email option is not enabled. Please be aware this does not affect authentication in any way. Our Engineering Team is working on a fix for this issue. This is only affecting the following deployments: DUO33, DUO50, DUO56, DUO58, DUO62, and DUO63, and no other deployments will experience this issue.

Please check back here or subscribe to updates for any changes.

Posted Mar 17, 2020 - 16:35 EDT

This incident affected: DUO33 (Admin Panel), DUO50 (Admin Panel), DUO56 (Admin Panel), DUO58 (Admin Panel), DUO62 (Admin Panel), and DUO63 (Admin Panel).