On October 7, 2015, between approximately 1:10PM UTC and 1:47PM UTC, Duo Security users were unable to authenticate via phone callback and SMS passcodes due to an outage at our primary telephony provider. Users were able to authenticate using other methods such as Duo Push, Duo Mobile generated passcodes, and hardware tokens.
At 1:47 UTC Duo re-routed calls through a different provider, restoring service. Some users later received phone calls that read them the following error message: "An application error has occurred. Goodbye."
Duo automatically detects and re-routes requests if a telephony provider fails to accept the call. During this event, calls were not automatically re-routed because the provider accepted the calls without indicating there was an error. We have deployed monitoring to detect this specific type of dropped call and are improving general dropped call detection to quickly identify similar issues moving forward.
On October 8, 2015, Duo's telephony provider found the root cause (a configuration issue on their side) and began working to correct it. We continue to work closely with them to ensure successful remediation.
We sincerely apologize and recognize the impact this service interruption may have had on your operations. High-availability access to our systems is a top priority at Duo Security and we will continue to make improvements to our product and support processes.