Expired internal SSL cert breaks Actions runner connectivity
Self-hosted and larger GitHub Actions runners worldwide could not connect for nearly 5 hours, delaying or failing workflow jobs, while the resulting reconnection surge briefly degraded API Requests, Issues, and Pages with added latency and elevated error rates.
Certificate expiration is a fully predictable failure mode, so renewal must be automated with independent expiry monitoring, and reconnect paths need circuit breakers so a recovering fleet of clients doesn't stampede shared APIs on its way back up.
An SSL certificate used by a subset of internal services responsible for runner connectivity expired because of a lapse in certificate lifecycle management.
The expired certificate blocked self-hosted and larger runners from establishing connections to GitHub, causing queued and failing workflow jobs; as runners repeatedly retried connecting, the surge in reconnection traffic added several seconds of latency and elevated 5xx errors on GitHub's APIs, which in turn produced knock-on degradation in Issues and Pages before engineers rotated the certificate and the backlog drained.
Renew certificates automatically and alert before they expire. Slow reconnection attempts so one expiry does not overload shared APIs.
