Phase 5 — Concurrency and Networking (M8-M9)

This is the hard gate. Every previous phase built a foundation you could get away with mostly-understanding. This one you cannot. Concurrency bugs don’t reproduce; they compound. A network server that “works on my laptop” fails in three ways in production. If you graduate from this phase, you’ve cleared the single biggest barrier between hobbyist and professional systems C programmer.

Two months. You’ll learn threads and the pthreads API, the C11 memory model and atomics, why lock-free is often the wrong choice, how BSD sockets actually work end-to-end, and how modern high-performance servers (nginx, redis, envoy) are architected around event loops. By the end you’ll have written an epoll-based HTTP server that handles 100k rps on your laptop — and you’ll understand every line of it.

Why this phase matters for you as an applied ML engineer

Every inference server you’ll ever touch is a concurrent networked C or C++ program. TensorRT-LLM, vLLM (the fast paths are C++), Triton Inference Server, sglang — they all share the same architecture: a thread pool, a request queue, an epoll or io_uring event loop, batching logic, and careful lifetime management for GPU handles. The torch.compile graph executor is the same story at a smaller scale. If you can build the toy version in this phase, you can read and modify the production ones.

The ML engineer’s specific angles you’ll pick up here:

  • How request batching actually works (a lock-free or lightly-locked queue, a batch collector thread, a triggering condition variable).

  • Why “one thread per request” doesn’t scale and what production servers do instead.

  • How to reason about tail latency (p99, p99.9) — mostly a concurrency problem, not a compute problem.

  • What KV-cache sharing across requests requires from the runtime (yes, this is concurrent data structures).

M8-M9 target

By the end of M9 you should:

  • Write a correct thread pool from scratch — task queue, workers, condition-variable-driven, graceful shutdown — in one sitting.

  • Use <stdatomic.h> with the right memory order and be able to defend the choice.

  • Sketch on a whiteboard: a TCP server accepting connections into an epoll loop, with a work queue feeding worker threads.

  • Debug a real concurrency bug using ThreadSanitizer.

  • Benchmark a network server with wrk and read the tail latency numbers correctly.

  • Read a chunk of nginx or redis source and follow the control flow.

The hard gate — why this stretch is different

Every previous chapter had a definite answer. Concurrent programs have statistical answers: this works 999 times out of 1000 in testing, then hangs at 3 AM on a Sunday because a specific instruction interleaving that occurs once every 2^30 runs finally hit. The habit you must build here is: prove your code is correct by construction and by tooling (TSan, Helgrind, model checkers), not by “I ran it and it worked.”

Corollary: fewer, simpler primitives beat cleverness. A well-designed mutex-based system that a colleague can review beats a lock-free masterpiece nobody wants to touch. We cover lock-free in file 03 not because you should use it often but because you should recognize what you’re avoiding.

Curriculum

#

File

What you get

1

01_pthreads_fundamentals.md

pthread create/join, mutex, cond var, rwlock, the wait-loop pattern

2

02_memory_model_and_atomics.md

C11 <stdatomic.h>, memory orders, cache coherence intuition

3

03_lock_free_and_lock_hierarchy.md

SPSC ring buffer, ABA, why lock-free is usually the wrong first choice

4

04_thread_pools_and_work_queues.md

A proper thread pool, graceful shutdown, work-stealing preview

5

05_sockets_from_scratch.md

BSD sockets, getaddrinfo, TCP options that actually matter

6

06_the_c10k_problem_and_beyond.md

Select → poll → epoll/kqueue/io_uring, the reactor pattern

7

07_epoll_and_io_uring.md

Edge vs level triggered, read-until-EAGAIN, io_uring SQE/CQE with 2026 caveats

8

08_a_production_server_walkthrough.md

Guided read of a real event-loop server (redis or libuv)

9

09_debugging_concurrent_c.md

ThreadSanitizer, helgrind, deadlock repro, load testing with wrk

—

projects.md

Epoll HTTP server (100k rps target) + SPSC lock-free queue

Time budget (M8-M9, 10-15h/week)

  • M8 weeks 1-2: Files 01-02. Get pthreads and atomics into muscle memory. Small toy programs only.

  • M8 weeks 3-4: Files 03-04. Ship the thread pool as a stand-alone library.

  • M9 weeks 1-2: Files 05-07. Start the HTTP server. First target: correct with 10 concurrent clients.

  • M9 weeks 3-4: File 08 (read someone else’s server). File 09 (fix your own bugs). Push the HTTP server to 100k rps. Ship the SPSC queue.

Exit criteria — you don’t graduate Phase 5 until

  • You can write, from memory, a producer-consumer pair using pthread mutex + condition variable, with correct wait-loop-while-condition semantics.

  • You can name three memory orders in C11 atomics and give an example where each is the right choice.

  • Your thread pool passes ThreadSanitizer clean under a load test.

  • Your epoll HTTP echo server hits at least 50k rps with 1000 concurrent connections on your laptop (100k is the stretch target).

  • You have read at least one file of production event-loop server code (redis’s ae.c, or libuv’s src/unix/core.c, or nginx’s event/ngx_epoll_module.c) and can explain the main loop.

  • You have reproduced a deadlock, captured its state under helgrind or gdb, and fixed it.

What most people get wrong about this phase

They start with an event loop before they’ve built a threaded server. Or they start with io_uring before epoll. Or they read half a chapter of C++ concurrency and think it transfers to C. The order that works: pthreads first → atomics second → sockets third → event loops fourth → io_uring as a stretch. Each layer needs the previous one solid. Skip and the bugs will teach you the same lesson at 100× the cost.

Anchor references for the whole phase

  • Paul E. McKenney, Is Parallel Programming Hard, And, If So, What Can You Do About It? — free at kernel.org (cdn.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html). Current version v2026.06.21a. This is the canonical free C-friendly concurrency book. HN commenters call it “the best book about atomics and concurrency.” Linux-kernel-flavored, which is what you want.

  • David Butenhof, Programming with POSIX Threads (1997) — old, still the pthreads reference. The API hasn’t changed.

  • W. Richard Stevens, UNIX Network Programming, Volume 1, 3rd ed. — the sockets bible. Long, thorough, timeless.

  • Beej’s Guide to Network Programming (beej.us/guide/bgnet) — v3.3.2, April 2025, actively maintained. The friendly complement to Stevens.

  • Brendan Gregg, Systems Performance, 2nd ed. — carried over from Phase 4; you’ll use it heavily for benchmarking.

  • Note on Anthony Williams, C++ Concurrency in Action: excellent book, but C++-specific (std::thread, std::async, RAII locks). Not a direct C reference. Read for concepts if you’ll be writing C++; use McKenney if you’re staying in C.

Return to roadmap root · Previous phase: ../05_systems_programming/README.md · Next: 01_pthreads_fundamentals.md