iOS Push Delivery Latency
Postmortem
From 8:59 UTC to 10:12 UTC on March 14, 2019, a subset of US based DUO deployments experienced issues sending Duo Push notifications to customers using Apple iOS devices. During this window, roughly 2% of these notifications were not properly delivered to devices.
Impacted users were able to manually “pull to refresh” within the Duo Mobile application to check for these missed notifications and continue through the authentication process. See the following knowledge base article for more information: https://help.duo.com/s/article/2051?language=en_US
The root cause of this issue has been identified as a partial outage of the Apple Push Notification service (APNs). Duo’s engineering team is committed to reducing the overall impact of similar events going forward.
Posted Mar 15, 2019 - 12:58 EDT
Resolved
The issue regarding push latency to iOS devices on our deployments is fully resolved and all services are now fully functional.
We will be posting a root-cause analysis (RCA) here once our engineering team has finished its thorough investigation of the issue.
Please make sure to check back or subscribe to be notified when the RCA is posted.
Posted Mar 14, 2019 - 06:49 EDT
Monitoring
We can now confirm that the push notifications sent to iOS devices are performing at an optimal level.
We will continue to monitor the issue and will post any updates when the incident is considered fully resolved.
Please check back here or subscribe here for further updates.
Posted Mar 14, 2019 - 06:28 EDT
Investigating
We are currently investigating an issue causing a latency of push notifications sent to iOS devices on some of our deployments and are working to correct the issue as soon as possible.
Users will still be able to approve requests by manually pulling to refresh and tapping the "Approve" button in the Duo Mobile application.
Please check back here or subscribe to updates for any changes.
Posted Mar 14, 2019 - 06:05 EDT
This incident affected: DUO1 (Push Delivery), DUO2 (Push Delivery), DUO4 (Push Delivery), DUO5 (Push Delivery), DUO6 (Push Delivery), DUO7 (Push Delivery), DUO8 (Push Delivery), DUO10 (Push Delivery), DUO11 (Push Delivery), DUO12 (Push Delivery), DUO13 (Push Delivery), DUO14 (Push Delivery), DUO15 (Push Delivery), DUO16 (Push Delivery), DUO17 (Push Delivery), DUO18 (Push Delivery), DUO19 (Push Delivery), DUO20 (Push Delivery), DUO21 (Push Delivery), DUO23 (Push Delivery), DUO24 (Push Delivery), DUO25 (Push Delivery), DUO28 (Push Delivery), DUO31 (Push Delivery), DUO32 (Push Delivery), DUO33 (Push Delivery), DUO36 (Push Delivery), DUO37 (Push Delivery), DUO44 (Push Delivery), DUO60 (Push Delivery), and DUO35 (Push Delivery).