Hardware token login issues
Postmortem
On October 6, 2015, between approximately 5:43AM UTC and 9:04AM UTC, Duo Security users hosted on deployments DUO1 and DUO3 attempting to authenticate using hardware token passcodes were unable to log in. All other methods of authentication including U2F tokens were fully operational during this period.
After confirmation from Support that the issue was affecting multiple customers, our Operations team investigated and identified the root cause of the issue as faulty code introduced as part of a product upgrade.
We sincerely apologize and recognize the impact this service interruption may have had on your operations. High-availability access to our systems is a top priority at Duo Security and we will continue to make improvements to our product and support processes. We are adding this scenario and related use cases to our automated testing, monitoring and escalation processes to identify and prevent potential future issues.
Posted Oct 06, 2015 - 14:51 EDT
Resolved
Our engineering team has received confirmation that hardware token logins appear to be functioning as normal now across all deployments.
Posted Oct 06, 2015 - 05:36 EDT
Monitoring
Our engineering team was made aware of an issue that prevented users hosted on DUO1 and DUO3 from logging in using hardware tokens to some applications. A fix has been put in place and we are currently monitoring the status of the issue.
Posted Oct 06, 2015 - 05:22 EDT
This incident affected: DUO1 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO2 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO3 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO4 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO5 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO6 (Admin Panel, Core Authentication Service, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO7 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO8 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO47 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO10 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO11 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO12 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO13 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO14 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO15 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO16 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO17 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO18 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO19 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO20 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO21 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO22 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO23 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO24 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO25 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO26 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO27 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO28 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), DUO29 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery), and DUO30 (Core Authentication Service, Admin Panel, Push Delivery, Phone Call Delivery, SMS Message Delivery).