From C10K to io_uring: The Evolution of High-Performance Server Concurrency

Udemy Course: https://staffengineer.rougeneuron.in This deep dive traces the evolution of server concurrency, starting with the C10K Problem (handling 10,000 idle, long-lived connections). The initial, intuitive thread-per-connection model failed due to resource exhaustion (massive thread stack memory, up to 2MB each ) and CPU thrashing from constant context switching , making the server vulnerable to Slow Loris-type denial-of-service attacks. The solution was a dual revolution: Event-Driven Architecture in the application (e.g., Node.js, Nginx) using non-blocking I/O and new kernel APIs (e.g., epoll on Linux ). epoll improved efficiency from a linear scan of all N connections to only reporting the K active ones. The discussion then compares modern abstractions: Node.js (great for I/O, poor for CPU work ), Go (productive, multi-core via M:N Go routines , but has latency caveats due to its Garbage Collector ), and Rust (predictable, low-latency performance without GC ), culminating in the latest frontier, io_uring , which reduces the system call overhead itself. 0:00 The Concurrency Problem & The C10K Panic 0:50 C10K: Concurrency vs. Throughput 1:40 Failure of Thread-per-Connection Model 2:07 Wall 1: Memory Exhaustion (Expensive Thread Stacks) 2:59 Wall 2: Context Switching Overhead 3:53 Wall 3: The Slow Loris Attack Vector 4:30 The Revolution: Event-Driven Architecture 4:56 Non-Blocking I/O Explained 5:25 The Event Loop & Kernel Notification 6:03 The Kernel Bottleneck: O(N) Scanning (select/poll) 6:50 The Kernel Fix: O(K) Efficiency (epoll, kqueue, IOCP) 7:59 Modern Abstractions & Workload Trade-Offs 8:36 Architecture 1: Node.js (Single-Threaded I/O Master) 9:27 Node.js Achilles Heel: Blocking the Single Thread 9:35 Architecture 2: Go (M:N Goroutines) 10:27 Go's Cost: Garbage Collector Pauses (The Discord Example) 11:23 Architecture 3: Rust (Async/Await & No GC) 12:35 Beyond C10K: C10M and Kernel Bypass (DPDK) 13:40 The Next Bottleneck: System Call Overhead 14:04 The Future: Linux io_uring (Shared Memory & Batching) 15:06 Conclusion: Choosing Runtime vs. Kernel Trust