Rung 5 (M7) — Concurrency Duo: Thread Pool + SPSC Ring Buffer

This is the first rare artifact on the ladder. Most C++ portfolios do not contain any lock-free code. If yours does — and it passes ThreadSanitizer, and has honest benchmarks against Boost.Lockfree, and includes latency histograms — you are now visibly in a different candidate tier from the average applicant.

Rung 5 is a single repo with two subprojects. Both live under one roof because they solve the same problem (“communicate between threads efficiently”) at two different scales: work distribution (thread pool) vs. streaming data handoff (SPSC ring buffer). Bundling them lets one repo tell a coherent concurrency story.

What It Is

A public GitHub repo named cpp-concurrency-kit with two subdirs:

thread_pool/

  • A modern C++20 thread pool: thread_pool class with submit(F&& f, Args&&...) -> std::future<...>.

  • Uses std::jthread and std::stop_token (C++20) for cooperative shutdown.

  • Configurable worker count (default: std::thread::hardware_concurrency()).

  • Optional priority variant: priority_thread_pool with a bounded number of priority levels.

  • Optional work-stealing variant: work_stealing_thread_pool (this is the stretch — do only if base ships).

  • Unit tests + stress tests + TSan CI job.

spsc_ring/

  • A single-producer single-consumer lock-free ring buffer: spsc_ring<T, Capacity>.

  • Capacity must be a power of two (documented and enforced at compile time via static_assert).

  • Uses std::atomic<size_t> with explicit memory ordering (acquire/release, not seq_cst). Justify the choices in a comment above each atomic op — this is where reviewers will look hardest.

  • Cache-line padding to prevent false sharing between head and tail (use alignas(std::hardware_destructive_interference_size) with a fallback constant if unavailable).

  • try_push(T&&) -> bool and try_pop(T&) -> bool. No blocking variants.

  • Unit tests + stress tests + TSan CI job.

Repo-level

  • Top-level README with the 5-section base plus a Latency Histograms section rendering three charts:

    1. Thread pool: task submission-to-completion latency at concurrency 1, 4, 16 workers.

    2. SPSC ring: producer-to-consumer latency at message sizes 8, 64, 512, 4096 bytes.

    3. SPSC ring: throughput vs. Boost.Lockfree’s spsc_queue at the same message sizes.

  • Benchmarks live in benchmarks/ and use google/benchmark.

  • CI runs: build, test, ASan, UBSan, and TSan on every push. TSan is the point.

Why It Matters (Employer Signal)

One line: “Writes and validates lock-free code — rare outside HFT and systems teams.”

Lock-free correctness under ThreadSanitizer is a hard skill to fake. The signal from this rung is not just “knows concurrency” — it is “has actually finished a concurrent artifact and validated it.” That is the difference between reading Anthony Williams’s book and having shipped the code from it. For applied-C++/ML roles at ByteDance, F5, GM Cruise, and every inference-engineering team, this rung answers the question they will ask: “can this person write the runtime, not just the model?”

Acceptance Checklist

  • Public GitHub repo named cpp-concurrency-kit, MIT license.

  • Two subdirs: thread_pool/ and spsc_ring/, each independently buildable.

  • Top-level CMakeLists.txt builds both.

  • All tests pass under ThreadSanitizer. This is non-negotiable. If TSan reports a race, the rung is not done.

  • All tests also pass under ASan and UBSan.

  • Thread pool: submit + get result works for void, int, and move-only return types.

  • Thread pool: destructor drains outstanding tasks before returning (documented behavior).

  • Thread pool: stress test submits 1M no-op tasks across 8 workers; completes with no leaks under ASan.

  • SPSC ring: stress test with a producer and consumer thread each doing 10M ops; no data loss, no ordering violations, TSan-clean.

  • SPSC ring: every atomic operation has a comment explaining its memory ordering.

  • Benchmark chart PNGs committed under docs/ and rendered in README.

  • Benchmark comparison against boost::lockfree::spsc_queue documented honestly (win, tie, or lose — report the truth).

  • CI matrix: {GCC 13, Clang 17} × {Release, ASan, UBSan, TSan}. All green.

  • Repo shared on r/cpp with [Show r/cpp] tag.

README Structure Additions

On top of the 5-section base:

  1. Design notes — One paragraph per subproject. For the thread pool: why std::jthread, how shutdown works. For the SPSC ring: why power-of-two capacity, what memory ordering you chose and why, how you avoid false sharing.

  2. Latency Histograms — The three charts described above. This is the part that gets clicks.

  3. Correctness Testing — A section explicitly listing: TSan runs clean, ASan runs clean, UBSan runs clean, stress test iterations completed. This is the credibility section.

  4. Not Suitable For — Say plainly: this SPSC ring is not MPMC. Do not use in production without understanding this. Honesty here is respected.

Where to Publish and Share

  • GitHub: public, MIT, topics: cpp, cpp20, concurrency, lock-free, thread-pool, spsc-queue, benchmarks.

  • Reddit: r/cpp with [Show r/cpp]. Example title: “[Show r/cpp] cpp-concurrency-kit: TSan-clean thread pool and SPSC ring buffer with Boost.Lockfree comparison.”

  • r/cpp_questions: cross-post for code review. Lock-free code deserves multiple eyes.

  • Hacker News (Show HN): worth attempting. Title: “Show HN: SPSC ring buffer in C++20 with honest Boost.Lockfree benchmarks.” The word “honest” and the benchmark table are the hook.

  • Twitter/X: post the latency histogram chart with the repo link. This is a visual artifact and shares well.

Common Ways This Rung Fails

  • TSan reports a race and you ignore it. Do not ship until it is clean. A TSan race in a lock-free artifact means the artifact is wrong, not “probably fine.” Even if the tests pass, the code is broken.

  • You use std::memory_order_seq_cst everywhere. Correct but slow, and reviewers on r/cpp will call it out. Use acquire/release where you can justify it. If you can’t justify, seq_cst is fine — but say so in the comment.

  • You forget cache-line padding on the head/tail atomics. Then producer and consumer contend on the same cache line and your benchmark is 5-10× slower than it should be.

  • You benchmark under -O0 or with TSan enabled. Benchmarks must be -O3 -DNDEBUG and without sanitizers. Separate CI job for TSan; separate for perf. Document the flags.

  • You claim to beat Boost.Lockfree without checking. Almost certainly false at message sizes > 64 bytes. Post the numbers honestly. Reviewers respect “we lose by 15% at 4KB messages but win at 8 bytes” way more than “we beat Boost.”

  • You build a work-stealing pool as your base scope and don’t finish. Stick to the simple bounded-queue design for the base. Work-stealing is the stretch.

What Most People Get Wrong

They get the code working and never validate it. Lock-free code that “seems to work” is where careers go to die. The value of this rung is not the code — it is the evidence of validation: TSan output, stress-test iteration counts, latency histograms. If your repo doesn’t render those, a reviewer has no way to distinguish it from the hundreds of “my thread pool” repos on GitHub. The validation IS the artifact.

Second common mistake: shipping only the thread pool because the SPSC ring is harder. Then the rung’s rare-skill signal collapses — thread pools are common. The SPSC ring is what makes this rung a differentiator. If you have to cut, cut the work-stealing stretch, not the ring buffer.

Extension Challenges (Rank-Ordered)

  1. Work-stealing thread pool — highest signal. If you have two weeks of slack, do this. Reference: Chase-Lev deque.

  2. MPMC ring buffer — significantly harder than SPSC. Requires per-slot sequence numbers (Vyukov MPMC design). Do only if concurrency is genuinely your thing.

  3. Coroutine-based co_await-able thread pool — C++20 coroutines, natural fit. This is a wow-factor stretch.

  4. perf-generated flamegraph committed to docs/ and rendered in README. Zero code, high signal.

  5. Formal reasoning notes on the memory-ordering choices, cross-referenced against Herb Sutter’s atomic weapons talk. This is nerd-cred.