Cloudflare Global Outage Caused by Misconfigured WAF Rule | The DevOps Postmortem

On July 2, 2019, Cloudflare experienced a global outage lasting approximately 27-30 minutes, leading to widespread 502 "Bad Gateway" errors for visitors to Cloudflare-proxied domains. The incident was triggered by the deployment of a single misconfigured rule within the Cloudflare Web Application Firewall (WAF) during a routine update. This rule contained a poorly written regular expression that caused excessive backtracking, resulting in CPU utilization spiking to nearly 100% across Cloudflare's global network servers handling HTTP/HTTPS traffic. 📚 Sources: • Cloudflare outage caused by bad software deploy (updated) — https://blog.cloudflare.com/cloudflar... • Details of the Cloudflare outage on July 2, 2019 — https://blog.cloudflare.com/details-o... • Cloudflare global outage on July 2, 2019 — https://postmortems.app/cloudflare-gl... • Cloudflare RCA: Major outage was a lot more than “a regular expression went bad” — https://www.packtpub.com/blog/cloudfl... • Major Websites Across Internet Are Down Due to a Cloudflare Outage — https://www.businessinsider.com/cloud... • The Regex That Took Down Cloudflare — And 10% of the Internet —   / the-regex-that-took-down-cloudflare-and-10...   • A Cautionary Tale of When Regex Can Go Wrong: Lessons from Cloudflare's Outage —   / a-cautionary-tale-of-when-regex-can-go-wro...   • Cloudflare Outage: A Case for Better CDN Reliability — https://www.catchpoint.com/blog/cloud... • Details of the Cloudflare outage on July 2, 2019 - Simon Willison's Weblog — https://simonwillison.net/2019/Jul/12... • Cloudflare's 27 Minutes Outage Explained — https://www.infoq.com/news/2020/08/cl... • Cloudflare global outage on July 2, 2019 | The DevOps Postmortem — https://thedevopspostmortem.com/cloud... 🔥 The DevOps Postmortem — real incidents, real lessons. Subscribe so you don't repeat them. #DevOps #SRE #Postmortem #CloudComputing #Kubernetes #IncidentResponse