systemsfailed.devfailure archive
← All incidents
Incident report
Meta/2021-10-04/~6h

The day Facebook deleted itself from the internet

Jan 2021Dec 2021

Facebook, Instagram, WhatsApp gone globally for ~6 hours. Internal tools, badge access and conference systems were down too, slowing recovery.

Lesson

A safety mechanism reacting to a bigger failure can amplify it. Watch for circular dependencies where the tools to fix an outage are taken out by it.

A routine backbone-maintenance command, meant to assess capacity, accidentally severed all backbone connections between data centers.

Meta's DNS servers withdrew their BGP routes when they lost backbone reachability — a designed safety reflex. With DNS unreachable, every property vanished, and the very tools needed to fix it were on the network that was down.

Discuss BGP/DNS as a hard dependency and the danger of recovery tooling sharing fate with production.

Share this failurePost on XDownload card