02 — Capstone Beta: PyFast¶

PyFast is a Python-native library with a C++ core that solves a real ML/data primitive faster than the pure-Python competitor. It ships to PyPI. It has a demo Colab notebook. Its README has a benchmark chart. It exists to prove you can be the person who owns the boundary between a fast C++ engine and a comfortable Python API — which is exactly what growing companies like Modal, MLX, Cerebras, and every ML platform team at scale actually need.

Where MiniServe positions you as “backend / infra,” PyFast positions you as “library author.” Different door. Both worth having open.

Positioning¶

The archetypal PyFast user is a data scientist or ML engineer who pip installs your package, spends 60 seconds in the Colab, and then decides your library saves them a day per week. That user is not a C++ engineer. Your job is to make the boundary invisible to them.

The archetypal PyFast reader-of-code is a hiring engineer looking at your GitHub. Their question is: can this person write clean bindings, package a wheel for three OSes, write meaningful tests, and hold a stable API? PyFast answers yes.

The bindings choice: nanobind vs pybind11 (2026)¶

This is your first decision. Get it right, save weeks.

	pybind11	nanobind
Age / maturity	2015, extremely stable	2022, actively used in JAX / MLX / PennyLane
Binary size	Larger	Substantially smaller (often 3–5x)
Compile time	Slower	2–4x faster
Dispatch overhead	Higher	Lower (competitive with hand-written CPython C)
Python Stable ABI	No	Yes (from Python 3.12) — one wheel supports many Python versions
Multiple inheritance	Yes	No (limitation)
Ecosystem docs / tutorials	Vast	Growing

2026 default: pick nanobind. It is the natural successor. Smaller wheels, faster compile, Stable ABI support. Its limitations (no multiple inheritance, no full Python ABI compat pre-3.12) rarely bite in a library at your scope. Cite the migration to nanobind by JAX and MLX in your blog post — it grounds the choice.

Fallback: pybind11. If your candidate design requires Python multiple-inheritance or if a specific dependency you need only ships with pybind11 bindings, use it. Never mix; do not import both into one wheel.

The three candidate designs (pick one in W45)¶

Pick during M11 kickoff. Do not try to do more than one. All three would be good capstones; you only need one.

Candidate A: `pyfast_ann` — Approximate Nearest Neighbor index (HNSW-lite)¶

What. An in-memory ANN index with add, search(k), save, load. Backed by a hand-rolled HNSW-style graph in C++. Python API mimics hnswlib / faiss.

Why worth building. Vector search is a red-hot 2025-2026 skill area (RAG, embeddings, retrieval). Every ML platform team wants faster or leaner ANN. hnswlib is old and lightly maintained; faiss is heavy. There is real room for a small, well-benchmarked competitor.

Complexity. Medium-high. The HNSW paper (Malkov & Yashunin, 2016) is dense but readable. You will spend a week getting the graph right.

Bench target. pip install hnswlib, run the same queries, show comparable recall at 90% of the memory or 80% of the query time. Even parity is publishable if the code is clean.

study leverage. High. Every vector-DB startup will love this.

Candidate B: `pyfast_agg` — Columnar aggregation engine on Arrow¶

What. Numpy-compatible API for group-by aggregations over Apache Arrow columnar buffers. Faster than pandas.groupby on wide tables. Uses SIMD kernels for sum, mean, min, max, count_distinct per-column.

Why worth building. “Pandas is slow” is a $10B market (Polars, DuckDB, Dask). A niche, focused columnar aggregator is a manageable slice. Arrow is the interop layer of choice; using it means you interop with the whole modern data ecosystem.

Complexity. Medium. Arrow’s C++ API is well-documented. Group-by is a well-understood algorithm (hash partition + per-partition aggregate).

Bench target. vs pandas.groupby().agg() on 10M-row tables. Aim for 5–10x speedup on numeric aggregations. vs polars you will lose — acknowledge it, positioning yourself as “educational and embeddable” not “trying to replace polars.”

study leverage. High for data platform / analytics engineering roles.

Candidate C: `pyfast_op` — Custom PyTorch op via `torch::Library`¶

What. One custom operator (e.g., a fused GELU + linear, or a specialized attention variant) registered with PyTorch’s dispatcher. CPU implementation in C++, optional CUDA fallback if you have GPU access.

Why worth building. Signals you can extend PyTorch itself — the most credentialing move possible in the applied-ML-C++ world. Very few candidates have done this.

Complexity. High. PyTorch’s build system (setuptools extensions or CMake with torch::Library) is finicky. Version drift is real. But the payoff in credibility is proportional.

Bench target. vs the naive PyTorch composition of the same op. Should win on latency and memory. If CUDA: vs torch.compile’d version.

study leverage. Very high for Nvidia, PyTorch team, and any ML infra team at scale. Also higher risk of getting stuck in build-system hell.

Recommendation for Raghul¶

Given the applied-ML-C++ positioning and the Zoho/India MNC target market, Candidate A (pyfast_ann) is the recommended pick. Reasons:

Vector search is universally understood — every ML team gets it in 60 seconds.
HNSW is a paper you can master in one week.
Benchmarking against hnswlib is easy and honest.
It ships as a small pure-C++ wheel with nanobind — lowest packaging risk.
It leaves clean room for follow-up posts (“adding IVF,” “adding on-disk index,” “CUDA port”) in year 2.

If that is unavailable to your interest (some people find graph algorithms tedious), pick B. Avoid C unless you have GPU access and a taste for build systems.

Everything below assumes A. The structure ports to B or C with minimal changes.

Feature set (for `pyfast_ann`)¶

Must-have¶

Core index. HNSW-style graph. Configurable M (out-degree), ef_construction, ef_search.
API. Index(dim, metric='l2'|'cosine'), .add(vec, id), .add_batch(vecs, ids), .search(query, k), .search_batch(queries, k), .save(path), .load(path).
Persistence. Custom binary format. Version-stamped header.
Numpy interop. All vector inputs accept numpy.ndarray directly, no Python-list overhead.
Thread safety on read. Multiple threads can .search concurrently. Writes are serialized.
Benchmarks. vs hnswlib on SIFT-1M and GloVe-100 (standard ANN benchmarks). Both recall@10 and QPS.
Wheels for Linux (manylinux2014), macOS (both x86_64 and arm64), Windows. cibuildwheel handles this.
PyPI publication. Under a unique name (pyfast-ann is likely taken — verify in M11, pick something unique).
Colab demo notebook. 5 cells: install, load SIFT sample, build index, search, benchmark against hnswlib.

Non-features¶

No distributed index. In-memory single-machine only.
No CUDA. CPU only, with SIMD (_mm256_* or svfloat_* on ARM).
No filtering / hybrid search. Pure vector.
No dynamic dimensionality. Fixed at construction.
No CLI. Library only.

Acceptance criteria (Beta)¶

pip install <pkg> from PyPI works on a fresh Python 3.10+ venv on Linux, macOS-arm64, and Windows. Verified on all three.
Colab demo runs top-to-bottom without errors, shared via [![Open In Colab]] badge in the README.
Benchmark table in README against hnswlib on SIFT-1M and GloVe-100: recall@10 and QPS at three ef_search values.
Test suite passes on GitHub Actions on all three OSes.
Documentation at least covers: install, quickstart, API reference (docstrings + Sphinx or mkdocs), one “how it works” page explaining HNSW conceptually with a diagram.
LICENSE = MIT or Apache 2.0.
Blog post published, following the outline below.

Blog post outline (M13)¶

Working title: “pyfast_ann: writing my own vector index in C++ and shipping it to PyPI.”

Structure:

The elevator. What it is, one benchmark line, one code snippet.
Why HNSW. Two paragraphs on the algorithm. One diagram (adapted from the paper, credited).
The C++ core. Data layout choices (SoA for vectors, flat-array graph). SIMD kernel for L2 distance with a snippet.
The nanobind boundary. Why nanobind. What the bindings actually look like (5 lines).
Packaging pain. cibuildwheel setup. manylinux quirks. The one thing that broke on macOS-arm64. (There will be one. Tell that story.)
The benchmarks. Fair-methodology paragraph. Chart. Discussion.
What is next. IVF, on-disk, filtering. Signals ambition without over-promising.

Packaging: the checklist that saves you three days¶

pyproject.toml with scikit-build-core or nanobind’s CMake integration.
cibuildwheel in GitHub Actions building wheels for cp310-cp313, manylinux2014, macos-arm64+x86_64, win_amd64.
Python Stable ABI enabled via nanobind: NB_MODULE(..., NB_STABLE_ABI).
hatch or flit for the metadata; classic setuptools is fine but painful.
Test-PyPI first. Always. TWINE_REPOSITORY_URL=https://test.pypi.org/legacy/ twine upload dist/*.
Only then upload to real PyPI. You cannot delete or replace a version once published — only yank.
Semantic versioning from day one. 0.1.0 → 0.2.0 → … → 1.0.0 when the API stabilizes.

Failure modes¶

The graph is buggy (bad recall). Fall back to a flat brute-force index for the first release, ship, and add HNSW in v0.2. A working flat + numpy library on PyPI is still a legitimate capstone.
cibuildwheel eats 3 days. Ship Linux-only wheels first. Announce macOS/Windows as “coming next release.” Do not let packaging block launch.
Nobody uses it. Expected. PyPI download numbers are for you, not the world. Ten downloads in the first month is a fine outcome; the capstone value is the artifact and the blog post, not adoption metrics.

What most people get wrong¶

Building a library nobody would pip install. “Yet another factorial” or “my toy sorting library.” Pick something with a real user shape.
No numpy interop. If your API is add([1.0, 2.0, 3.0]) instead of add(np.array([[1.0,2.0,3.0]])), you have already lost.
Skipping the Colab notebook. The Colab badge is the single highest-conversion element on a Python library README.
Publishing to PyPI without Test-PyPI first. You will find out about a broken wheel by having a broken v0.1.0 forever.
Trying to beat SOTA. The bar is exists, works, benchmarks honestly, ships on 3 OSes. Not “beats faiss.” Anyone who thinks a solo capstone beats faiss has not tried.