Dual AWS region outage takes down notification dispatch
Notification dispatch was completely down for 18 minutes after two of PagerDuty's three datacenters failed simultaneously; the events-ingestion API stayed up throughout.
Redundancy plans must verify physical and network independence between supposedly separate failure domains, not just assume it from provider region boundaries.
A peering point shared by two AWS regions PagerDuty treated as independent datacenters degraded, taking both regions offline at once.
PagerDuty had designed for a single datacenter failure across three sites, but two of the three secretly shared a network peering point; when it degraded, both went down together, the dispatch cluster couldn't reach quorum under the resulting latency, and a coordinator misconfiguration slowed recovery further.
Do not assume cloud regions fail independently. Test that one region can fail without breaking the other.
