Projects — Phase 5¶

Two shippable deliverables anchor this phase. The first is HARD GATE #1 of the roadmap and is non-negotiable — an epoll-based HTTP/1.1 echo server that sustains 100k rps on a 4-core laptop and passes a battery of correctness checks. The second is a lock-free SPSC queue with TSan-clean tests and a real benchmark against a mutex baseline. Together they take a full two months (M8-M9). Everything else in this file — the stretch projects and week-by-week plan — supports these two goals.

Ship the two deliverables and you have credibility with any systems-C hiring manager alive. Ship neither and you’ve studied concurrency but not built anything with it, which is the difference between reading about deadlifts and lifting.

Deliverable 2 — SPSC lock-free queue with benchmark¶

A single-producer, single-consumer lock-free ring buffer using <stdatomic.h>, benchmarked against a pthread_mutex_t-guarded bounded queue. Target: 5-10× throughput on 2 threads on your laptop. Below 5× means either the mutex baseline is too slow (you weren’t fair) or the atomic queue has too many memory barriers (over-engineered).

Acceptance criteria¶

Uses only <stdatomic.h>. No inline assembly, no compiler intrinsics beyond what _Atomic and atomic_* provide.
Memory-order rationale documented for every atomic op. Every atomic_load_explicit / atomic_store_explicit in the code has a comment: // acquire — reads must not be hoisted above this or similar. If you can’t defend the memory order on every op, downgrade to memory_order_seq_cst and note that you did.
TSan clean under a 1M-op benchmark. Two threads, producer pushes 1M elements, consumer pops 1M, checks a monotonic sequence number is preserved. Zero TSan reports.
Benchmark methodology written up in BENCHMARK.md. CPU model, cache size, methodology (pinned threads to specific cores or not, warmup iterations, item size). Report throughput as items/sec, both queues.
Ships as spsc.h + spsc.c with a header-only version optional. The mutex baseline as mpmc_mutex.h + mpmc_mutex.c.
Chase-Lev deque is not required. SPSC (one producer, one consumer) is the exercise; MPSC or MPMC lock-free is a stretch (see below) but not the graduation target.

Reference numbers to sanity-check yourself¶

Rough ballpark on a modern laptop (M-series Mac or Ryzen 7 laptop, 2 threads):

Mutex-based bounded queue: ~5-15 M ops/sec (limited by cache-line ping-pong on the mutex).
SPSC atomic ring: ~50-150 M ops/sec (limited by cache-line traffic on head/tail; if head and tail are in the same cache line you’ll be closer to 30 M).
Ratio: 5-10×, consistent with the folklore in the Rigtorp SPSC / rigtorp.se writeup.

If your numbers are wildly off in either direction, something is wrong; investigate before shipping.

Stretch projects — 3-4 options if you want more¶

Only if the two deliverables are truly done. Ranked by ML-engineer relevance.

(a) io_uring variant of the HTTP server¶

Same architecture, swap epoll for io_uring via liburing. Compare throughput and p99 latency to the epoll baseline. Document what you’d have to change to deploy this behind a Docker container (spoiler: seccomp profile). This is the exact comparison every ML inference server team has done in-house. Time budget: 2-3 weekends.

(b) HTTP/2 support¶

Add HTTP/2 (multiplexed streams, HPACK header compression, flow control). Benchmark with h2load -c 100 -m 32 -n 1M http://localhost:8080/. Note: HTTP/2 is a real protocol — HPACK alone is a week if you write it from scratch. Consider integrating nghttp2 (link against it) rather than reimplementing. This is what envoy does. Time budget: 3-4 weekends.

(c) TLS via BearSSL¶

Wrap the accept path with TLS handshake via BearSSL (bearssl.org), a small (~30 kLOC) constant-time TLS library. Avoid OpenSSL for this exercise; the API is too big to learn quickly, and BearSSL forces you to understand the state machine. Benchmark with h2load against https://localhost:8443/. Time budget: 2-3 weekends.

(d) A small pub/sub server¶

Redis-alike: TCP + a tiny text protocol (SUBSCRIBE topic, PUBLISH topic msg), fanout writes to all subscribers of a topic. Reuse the epoll HTTP server as the network chassis; add a shared subscription table (RCU or copy-on-write for the read-heavy topic list). Benchmark: 1 publisher, N subscribers, measure message-fanout throughput. Time budget: 3-4 weekends.

Sequencing guide — 8 weeks, week by week¶

You have M8 (weeks 1-4) and M9 (weeks 5-8), 10-15 hours per week. Plan:

Week	Files	Practice	Ship
1 (M8)	01, 02	pthread producer-consumer; atomic counter with memory-order variants	pthreads-toy repo
2 (M8)	02, 03	SPSC ring buffer draft #1 (correctness only, no benchmark)	SPSC draft, TSan-verified
3 (M8)	04	Thread pool from scratch; parallel wc; profile queue contention	Thread-pool library
4 (M8)	05	80-line echo server; UDP echo; break the server in 5 ways, fix each	Echo-server v0
5 (M9)	06, 07	Convert echo to non-blocking; add epoll; ET + drain loop	Epoll-echo v1
6 (M9)	07	Add HTTP/1.1 parsing; add keepalive; benchmark to ~50k rps	HTTP-echo v1
7 (M9)	08	Redis/valkey walkthrough; multi-thread the HTTP server; push to 100k rps	HTTP-echo v2
8 (M9)	09	TSan/ASan pass; 24h soak; SPSC benchmark; README + blog	HARD GATE demo + SPSC ship

How to know you’re on track vs slipping¶

End of week 2: you have a TSan-clean SPSC ring buffer. If not, you’re behind — spend a weekend catching up.
End of week 4: you have a blocking echo server that survives IPv4 + IPv6, Ctrl-C, and peer-kill without leaking fds. If not, you have a socket-basics gap; re-read file 05.
End of week 6: you have an epoll-based HTTP echo hitting at least 20k rps. If not, either your parser is slow, your event loop is level-triggered without drain, or you’re blocking on something in the hot path.
End of week 7: you’re at 50k+ rps. If not, profile with perf top and find the bottleneck — most likely a syscall in the hot loop or a lock on the accept path.
End of week 8: all 12 acceptance boxes checked. If not, list which ones remain and slip week 8 into an intercalary week — better to graduate late than to skip the checks.

Slippage tolerance: two weeks over 8 weeks is acceptable. Four weeks over 8 weeks means you should re-examine whether Phase 4 (systems programming) was solid; sometimes the debt is upstream.

What most people get wrong about this¶

They chase the 100k rps number before they’ve cleared the correctness box. A server that hits 120k rps but fails the 24-hour soak test with an fd leak is not a passable submission — it’s a demo. The order that works: correctness first (TSan clean, ASan clean, no fd leaks), scale second (10k concurrent), throughput third (50k → 100k rps). Every commit must keep correctness intact; you never trade correctness for a throughput number. The 12-checkbox list is deliberately correctness-heavy for this reason.

Second mistake: not shipping the blog post. The blog post is the artifact that turns “I wrote a server” into “I understand servers.” Writing forces you to justify every choice — why ET over LT, why EPOLLONESHOT, why 4 workers not 8, why HTTP/1.1 keepalive done this way. If you can’t write it up, you don’t fully understand it. The blog post is the graduation ceremony.

References¶

The Rigtorp SPSC queue — rigtorp.se/ringbuffer/ — the canonical modern C++ SPSC design. The C translation is straightforward.
1024cores.net (Dmitry Vyukov) — the classic archive of lock-free primitives. Old but foundational.
libevent, libev, libuv — three C event loop libraries at different points on the abstraction curve. Read one of their epoll backends for a second reference beyond redis’s ae_epoll.c.
nginx source — nginx.org/en/download.html — worth a second walkthrough after you’ve done redis. src/event/ngx_epoll_module.c is the file.
Beej’s Guide to Network Programming — always a warm re-read when a specific socket detail confuses you.
Gil Tene, “How NOT to measure latency” — Strange Loop 2013 talk. Watch before running any tail-latency benchmark.

Return to README.md · Next: ../07_applied_c_ml_inference/README.md