Rung 8 (M13) — Capstones: Alpha (MiniServe) + Beta (PyFast Library)

Rung 8 is the top of the ladder. Not because the code is the hardest — Rung 5 and Rung 7 are arguably technically harder — but because these two artifacts are the ones you will demo in studies, paste into cover letters, and lead with on LinkedIn. Everything before this rung was preparation for these two.

Capstone Alpha and Capstone Beta split by axis: Alpha is a service (extends Rung 7 into a real product), Beta is a library (extends Rung 6 into a real package). Together they give you both halves of applied-C++/ML positioning: someone who builds systems and someone who builds tools others use.

Do both. If you can only do one, do Alpha — the service artifact maps more directly to the highest-paying inference roles in 2026.

Capstone Alpha — MiniServe (“Triton-lite”)

What It Is

A public GitHub repo named miniserve-cpp — a production-grade multi-model inference server. Think “Triton for a single team.” It builds on Rung 7 but crosses a threshold: it is a product, not a demo.

Core capabilities to be added on top of Rung 7’s mini-inference-cpp base:

  • Multi-backend: ONNX Runtime + one of {TorchScript via libtorch, TensorFlow Lite, or a custom Eigen-based backend for very simple models}.

  • Model repository pattern: point the server at a directory; it discovers, loads, and hot-reloads models. Follows Triton’s model-repo conventions loosely.

  • Advanced dynamic batching: separate per-model batch configurations, priority queues, and preemption for high-priority requests.

  • Model ensembling: pipeline two models (e.g. tokenizer → embedding model), described in a YAML pipeline config.

  • Response caching using the LRU cache from Rung 2 (yes, that Rung 2 comes back — this is the ladder rewarding you). Configurable per-endpoint TTL.

  • Authentication middleware: API key + JWT bearer. Simple but real.

  • Rate limiting per API key.

  • Structured JSON logging with request IDs for tracing.

  • All Rung 7 observability retained and expanded: SLI dashboards, alerting hooks (Discord/Slack webhook on p99 breach), runbook markdown files in docs/runbooks/.

  • Full test suite: unit, integration, load, chaos (kill workers, watch recovery).

  • Kubernetes helm chart in deploy/helm/. Deployed to a real cluster (kind or a small managed cluster is fine).

Acceptance Checklist (Alpha)

  • All Rung 7 acceptance criteria still met.

  • Serves at least 3 models across at least 2 backends.

  • Model hot-reload works: drop a new ONNX file into the model repo dir, server picks it up without restart.

  • Batching + priority queue + ensembling all functional and documented.

  • LRU response cache integrated (with per-endpoint TTL).

  • Auth + rate limit tested with unit + integration tests.

  • Runbook markdown files for the top 5 predictable incidents (OOM, model load failure, backend crash, disk full, cert expiry).

  • Helm chart deploys cleanly to kind and to at least one small managed cluster.

  • Load test at concurrency 1 / 8 / 32 / 128 / 512 documented, p50 / p95 / p99 for each.

  • Chaos test: kill a worker during load test, verify recovery in the metrics.

  • Comparison table in README against Triton Inference Server: what MiniServe does, what it deliberately doesn’t do.

  • Blog post series (3 posts on personal blog): architecture, batching, observability.

  • Show HN post attempted.

  • Recorded 5-minute Loom or YouTube walkthrough demo. Link in README.

Capstone Beta — PyFast Library

What It Is

A public GitHub repo, name chosen during M11 based on which of three options you pick. Beta is a shipped C++ library with nanobind Python bindings, published to real PyPI, with a Colab notebook demo. Options:

  1. pyfast-ann-cpp — an approximate nearest-neighbor library. HNSW or IVF-Flat. Benchmarked against hnswlib and faiss-cpu. Recommended default.

  2. pyfast-agg-cpp — an Arrow-based aggregation engine. Groupby + sum/mean/count over Arrow tables. Benchmarked against DuckDB and pandas.

  3. pyfast-op — a custom torch::Library op (e.g. a fused activation or a specialized attention variant). Benchmarked against PyTorch’s implementation on the same input.

Pick one during M11 based on where the study signal is strongest for the roles you are seeing. If you are targeting vector-DB / retrieval / RAG-adjacent roles, pick #1. If you are targeting data-eng / analytics roles, pick #2. If you are targeting PyTorch-internal / kernel-engineering roles, pick #3.

Acceptance Checklist (Beta)

  • Public GitHub repo, MIT license.

  • C++20 implementation, header-mostly or header+minimal cpp.

  • nanobind bindings (not pybind11 — nanobind is the 2026 choice here for the currency signal, and pyfast is small enough to be pure nanobind idiom).

  • Real PyPI package, pip install pyfast-... works from anywhere.

  • Wheels built via cibuildwheel for Linux x86_64, macOS (Intel + Apple Silicon), Windows x86_64.

  • Zero-copy NumPy interop for input arrays.

  • Benchmark table in README against the incumbent library (hnswlib / DuckDB / PyTorch).

  • Colab notebook demo, publicly viewable, badge in README.

  • Sphinx docs on Read the Docs (or gh-pages Sphinx).

  • Type stubs (.pyi) generated and shipped.

  • At least 3 GitHub stars from strangers within 30 days of publication (this is a sanity check for reach; if not, the marketing side needs work).

  • Blog post writeup on personal blog.

  • LinkedIn announcement post.

Why These Matter (Employer Signal)

One line each:

  • Alpha: “I can operate a production-shaped inference platform end-to-end.”

  • Beta: “I ship C++ that Python developers depend on via PyPI.”

Together: “Shipping-grade applied C++/ML engineer.” This is the sentence you want to be the headline of your LinkedIn on M13 + 1 day.

These are the two artifacts that answer, respectively, the two most common study questions for applied-C++/ML roles: (1) “tell me about a system you operated in production” (Alpha), and (2) “tell me about a library others depend on” (Beta). Every other rung on the ladder is preparation for being able to answer these credibly.

Where to Publish and Share (Both)

  • GitHub: both public, MIT, well-tagged with topics.

  • Real PyPI (Beta only): register the name early in M12; publish on M13.

  • Docker Hub / ghcr.io (Alpha only): image published, versioned.

  • Colab (Beta): notebook publicly viewable, badge in README.

  • Personal blog: writeup for each capstone. Alpha gets a 3-post series (architecture, batching, ops). Beta gets a single deep-dive post.

  • Hacker News: Show HN attempts for both. Space them at least 2 weeks apart. Alpha’s Show HN framing: “Show HN: MiniServe — a small Triton-inspired inference server I built to learn.” Beta’s framing: “Show HN: pyfast-ann-cpp — fast approximate nearest neighbor for Python, written in C++.”

  • LinkedIn: one long-form post per capstone, at least a week apart. Frame the arc: “13 months ago I started returning to C++. Here is what I shipped.” Recruiters read exactly this kind of post.

  • Reddit: r/cpp [Show r/cpp] for both. r/MachineLearning [P] for Alpha. r/Python for Beta.

  • Twitter/X: architecture diagrams + benchmark charts. These are the visual assets that get shared.

  • Recruiter outreach: with both capstones live and deployed, begin direct outreach to hiring managers at the inference teams you have been tracking. This is the point of the ladder.

Common Ways This Rung Fails

  • Trying to build both Alpha and Beta from a clean slate. No. Alpha extends Rung 7. Beta extends Rung 6. If you rebuild them, you run out of time.

  • Beta scope explosion. All three of Beta’s options (ANN, aggregation, torch op) can eat a full year if unconstrained. You have one month. Pick one, keep the surface small.

  • Skipping the Colab notebook (Beta). The notebook is what makes people click. Without it, PyPI downloads are a rounding error.

  • Skipping the Loom demo (Alpha). Same principle. A 5-minute video is worth ten times the README.

  • Not registering the PyPI name early. Do this in M12. Names get taken.

  • Assuming HN will bite. It might not. That is fine. Post anyway. The link surviving on HN’s newest for 15 minutes is enough to seed a few Twitter shares.

  • Publishing without a target-role list. By M13 you should have a list of 20-30 specific companies and role IDs you are applying to. The capstones exist to be linked from cover letters to those specific roles — not to sit on GitHub hoping to be found.

What Most People Get Wrong

They ship the capstones and then… don’t apply to jobs. The capstones are lead-generation for job applications. On the day after Alpha’s Show HN post, you should be sending outreach to hiring managers with the deployed URL in the message. On the day after Beta’s PyPI publish, you should be updating your resume with the pip install pyfast-... line and applying to 5 roles that week.

The ladder is not the destination. The ladder is a job-application funnel. Rung 8 is where the funnel narrows to the point of decision. If you climb the ladder and then don’t apply, you have built the world’s most elaborate procrastination device. Do not do this.

Second common failure: over-polishing at the expense of shipping. The publication-cadence rule still applies at Rung 8. Ship Alpha rough on day M13-1. Ship Beta rough on day M13-15. Polish for two weeks after each. The countdown starts from publication, not from code-complete.

The M13 + 1 Day Plan

This is what your day looks like the day after Rung 8 ships.

  1. Update LinkedIn headline to include “C++ / Applied ML — shipped [Alpha] and [Beta].” Attach both links.

  2. Update resume. The Projects section reads: 8 rungs, top-line signals only, direct URLs.

  3. Send 5 cold-outreach messages to hiring managers at inference-team companies (ByteDance, F5, GM Cruise, Perplexity, Anthropic, and mid-size startups you have been tracking). Each message includes the deployed Alpha URL and the pip install line for Beta.

  4. Apply to 5 roles that week using the same links.

  5. Post the LinkedIn long-form summary. Tag it with a #cpp / #machinelearning combo. Ask your network for shares.

  6. Book 3 mock studies with peers or paid platforms (studying.io, exponent). The ladder built the artifact signal; mock studies build the live signal.

Extension Challenges

There are none. Rung 8 is the top. If you have slack at M13, you use it to apply to jobs, not to build more artifacts.