01 · Why C for ML Inference¶
Python is glue. Every fast thing you touch in ML — PyTorch’s aten::matmul, NumPy’s dot, JAX’s XLA-compiled kernels, vLLM’s PagedAttention, llama.cpp’s Q4_K matmul — is C, C++, or CUDA underneath a thin Python wrapper. The interesting job market in 2026 is not “who writes the Python” — it is “who writes the kernel.” This file is the honest argument for spending M10–M11 on C specifically, not on Mojo, not on Zig, not on Rust.
The stack of any modern LLM serving path¶
When a user hits an OpenAI or Anthropic API, the request roughly traverses:
Layer |
Language |
Why |
|---|---|---|
HTTP gateway, auth, rate-limit |
Go / Rust / Python |
I/O bound, business logic |
Batching, KV-cache scheduling |
Python + C++ |
orchestration in Py, hot path in C++ |
Attention / matmul kernels |
C++ / CUDA / C |
every cycle counts |
Quantization, dequant, GEMM tiles |
C with SIMD intrinsics |
register-level control |
Driver / firmware |
C |
kernel space, no runtime |
Read that column three times. Everywhere the word “performance” appears in a job description, the language column collapses to C, C++, or CUDA. Python appears only where latency is measured in milliseconds, not nanoseconds.
Three living pieces of evidence¶
ggml / llama.cpp (ggml-org/llama.cpp, 119k stars, 1,038+ contributors as of July 2026) is pure C for the tensor library, C++ for the model orchestration. It powers Ollama, LM Studio, GPT4All, and a growing fraction of on-device inference. It has zero Python in the hot path. When Georgi Gerganov added a new quantization format, he did it in ggml-quants.c — 4,000 lines of hand-written SIMD.
Redis / Valkey (see Phase 7 · 01_reading_production_code.md) — 120k+ lines of C serving 999.8K RPS on an 8 vCPU c8g.2xlarge in Valkey 8.1.1 benchmarks. When Snap migrated 70% of its cache off Redis Inc. onto AWS ElastiCache Valkey, they cut cache infra cost from $2.1M/yr to $840K/yr — the underlying language is C, and every microsecond of parsing the RESP protocol matters.
CPython itself is C. Every time you type import numpy, you are loading a .so file whose entry point is a C function. Writing a Python C extension is a marketable skill at Zoho, at any ML infra team, and at every startup that needs to ship a fast primitive without rewriting the world.
The “but what about X” section¶
“Mojo will replace C for ML kernels.” Mojo hit 1.0.0 beta 1 on May 7, 2026. The std lib is Apache 2.0. The compiler is not open source yet — Modular promises fall 2026. Oak Ridge’s SC25 WACCPD paper shows Mojo GPU kernels roughly competitive with CUDA/HIP for memory-bound workloads but with real gaps in atomic ops. Modular’s May 2026 blog post shows 48% less code than CUTLASS for the same matmul performance. This is real, not hype. But: (a) llama.cpp, ggml, cuBLAS, OpenBLAS, and every driver on Earth remain C; (b) the compiler being closed means you cannot ship a Mojo dependency into most production stacks yet; (c) Mojo threatens CUTLASS and Triton, not the C ecosystem. Verdict: monitor Mojo, don’t invest in it as your primary skill in 2026.
“Zig eats C’s lunch.” Zig is still pre-1.0 in July 2026. Latest stable is 0.15.2 (Oct 2025). Andrew Kelley told The Register on May 28 2026 he wants “uncompromising perfection” before 1.0. TIOBE April 2026 has Zig at rank #39 (0.31%). More importantly: the Bun team began a Rust rewrite in 2026, having originally chosen Zig. If Bun — Zig’s largest production user by revenue — is defecting, that is a signal. TigerBeetle stays loyal (Zig-first fintech DB), Ghostty terminal shipped Jan 2025, but none of these are threats to the C moat. Verdict: Zig is a bonus skill after C, not a replacement.
“Rust is the future.” For network services and CLI tools, largely yes — Cloudflare’s Pingora (Rust) replaced Nginx internally, confirmed 2026. But you already know Python; the marginal skill that pays is not “another safe language.” It is unsafe systems literacy — the ability to reason about UB, aliasing, cache, and hardware. That skill is best trained by C, and it transfers to Rust unsafe blocks, to CUDA, to kernel work. You can pick up Rust in three months after C. The reverse is not true.
“Carbon (Google’s C++ successor).” MVP expected late 2026 earliest. 1.0 after 2028. Not a factor.
The India-specific case¶
At Zoho and every Bengaluru chip startup (VASBM analog IP, RRP Electronics DSP-for-ISRO, Saankhya Labs modem silicon), C is not a legacy skill — it is the required skill. Glassdoor India in mid-2026 lists ~197 open “C programmer” roles. Zoho pays roughly ₹8–12 LPA for fresh hires and ₹15–25 LPA for senior C engineers on the platform side. ISRO’s Careers page (isro.gov.in/Careers.html) posts C-heavy embedded roles irregularly. The Indian systems-engineering market has not pivoted to Rust or Zig at any meaningful scale. You are geographically well-placed to monetize this exact skill.
The Zoho-specific case for you¶
You already work on Applied ML at Zoho India. Zoho’s platform team ships C for the actual server tier. The internal mobility path from “ML engineer using our platform” to “ML engineer contributing to our platform’s inference layer” is exactly what M10–M11 unlocks. The moment you can write a competent quantization kernel and read ggml-quants.c, you can propose an internal project — deploy a llama.cpp-based inference sidecar for a Zia use case, own the C code, own the on-call.
What most people get wrong about this¶
They think “learning C for ML” means learning matrix math in C. It doesn’t. It means learning memory hierarchy in C. The math is high-school linear algebra. The hard part is: how many bytes fit in your L1? How many cycles does a DRAM miss cost? Can you fit a Q4_0 block in a register? The rest of Phase 6 is the answer to those questions. If you leave with only one insight, let it be this: modern LLM inference is bandwidth-bound, not FLOPs-bound, and that single sentence explains why Q4 quantization is ~2× faster than Q8 for the same model — half the bytes to move.
Return to README.md · Next: 02_simd_and_vectorization.md