Rung 6 — SIMD GEMM Benchmark 🟧 HARD GATE #2¶

Headline. A public benchmark study that walks from naive matrix multiplication, through cache-blocked matmul, to AVX2 or NEON SIMD matmul — all in pure C with intrinsics — and lands within 3x of OpenBLAS on documented matrix sizes, with a roofline analysis and a long-form writeup.

Month target. M11 (May 2027). HARD GATE: publicly shipped by the last day of M11.

What You Build¶

Repo name: simd-gemm-journey. This is the résumé line rung. Everything about the presentation should reflect that.

Three matmul implementations, each in its own file, each independently benchmarkable:

gemm_naive.c — the textbook triple loop. C = A * B, row-major, single-precision float. Establishes the baseline.
gemm_blocked.c — cache-blocked (tiled) matmul. Block sizes tuned per L1/L2 cache. Loop reordering (ikj order for row-major).
gemm_simd.c — the flagship. SIMD via intrinsics:
- On x86: AVX2 with _mm256_fmadd_ps, 8-wide FP32.
- On Apple Silicon: NEON with vfmaq_f32, 4-wide FP32.
- Pick ONE target platform; do both only if you have time. Do not compromise depth for breadth.
- Includes register-blocking (compute a 6×16 or 8×8 output tile fully in registers before writing back).
- Optional: packing / re-layout of A and B into contiguous panels (this is what BLIS does).

Benchmark infrastructure:

bench/ — harness that runs each implementation on matrix sizes {64, 128, 256, 512, 1024, 2048}, reports GFLOPS, and generates a comparison plot.
OpenBLAS as the reference. Same benchmark harness runs cblas_sgemm for each size.
Roofline plot. X-axis operational intensity (FLOPS/byte), Y-axis GFLOPS. Plot each implementation as a point; plot the machine’s peak compute and memory-bandwidth ceilings as lines. This single plot is the image people share on Twitter/LinkedIn.
Cache-miss analysis via perf stat -e cache-misses,cache-references,L1-dcache-load-misses on Linux, or Instruments “Counters” template on macOS. Reported in a table, one row per implementation.
Reproducibility: either a GitHub Actions workflow that runs the benchmark on a documented runner and commits the numbers, OR a Colab-equivalent notebook. Reviewers will try to reproduce.

Mandatory writeup:

README.md with the money-shot plot (GFLOPS vs. matrix size, all four lines: naive, blocked, SIMD, OpenBLAS) inline.
WRITEUP.md — 200-400-3000 word long-form post — the story of the optimization. Which change bought which speedup? Where did you plateau? What did the roofline reveal? Include code snippets, the roofline plot, and honest “here’s where I got stuck for 3 days” moments.
docs/machine.md — exact CPU model, cache sizes (from lscpu or sysctl), compiler version, flags used. A reader must be able to check whether their machine is comparable.

Target size: ~500-800 LOC of C (matmul itself is small; the harness and analysis is the bulk of the work).

Why This Rung, Why Now — and Why It Is a HARD GATE¶

Rung 6 is the rung that stops you from being “another C programmer” and starts you being “a C programmer who understands the arithmetic of the machine.” Every ML-infra hiring manager — Nvidia inference teams, llama.cpp / ggml maintainers, Cloudflare Workers AI, Modal, Replicate, Groq, tinygrad-adjacent, ML compiler groups — will read the writeup and update their prior on you significantly.

It is a HARD GATE because the M13 pitch sentence’s clause “SIMD-accelerated ML inference kernels” is fabricated if this rung does not exist. You cannot claim it in a résumé, cannot defend it in an study, cannot use it in a LinkedIn headline. Rung 6 is that clause’s proof-of-work. Miss M11 and the year’s ML-infra positioning collapses.

Acceptance Criteria (all mandatory — this is a hard gate)¶

SIMD implementation lands within 3x of OpenBLAS on at least three of the six benchmark sizes (typically 256, 512, 1024)
Naive → blocked speedup ≥ 4x on 1024×1024 (proves cache blocking is real)
Blocked → SIMD speedup ≥ 3x on 1024×1024 (proves SIMD is real)
Roofline plot committed with machine ceilings drawn from measured peak GFLOPS and memory bandwidth (measure them; don’t quote datasheet numbers)
Cache-miss table in the writeup with perf / Instruments data
Writeup ≥ 2000 words, published on personal blog and cross-posted
Benchmark script reproducible on a fresh machine (documented in README)
LinkedIn post published targeting ML-infra recruiters with the roofline plot as the image
Submitted a talk proposal to a Bangalore meetup (PyDataBLR, BangaloreGoLang, ChennaiPyLadies-adjacent, or a Zoho-internal tech session)

Where to Publish¶

GitHub: pinned on profile. Topics: c, simd, avx2 or neon, gemm, high-performance-computing, blas.
Personal blog + dev.to + Hashnode: the long-form writeup. This is the piece you will link on your résumé for 5 years.
Hacker News — Show HN: submit the blog post. Titles like “Show HN: I got within 2.5x of OpenBLAS with hand-written AVX2”. HN loves this genre.
Reddit — r/programming: cross-post the blog.
Reddit — r/C_Programming: technical-detail thread about the register-blocking discovery.
Reddit — r/MachineLearning: if the writeup ties matmul to inference throughput, post there with an ML angle.
LinkedIn: long-form post with the roofline plot. Tag #MLInfra, #SystemsEngineering. This is the post you write for recruiters.
Bangalore meetup circuit: submit a 20-minute talk proposal. Even if not accepted, the submission is a signal.

Signal to Recruiter / Employer¶

“This person thinks like an inference-engine engineer. They can read a roofline plot and know whether they’re compute-bound or memory-bound. They can write intrinsics without hand-holding. They will not need six months to become useful on ggml, a CUDA kernel adjacent codebase, an ML compiler, or a hot-path in a serving system.”

This is the résumé line that opens Nvidia inference-team, Cloudflare Workers AI, llama.cpp / ggml maintainers, Modal, Replicate, Groq, Tenstorrent, tinygrad-adjacent doors. Also opens Indian ML-infra plays: Sarvam AI, Krutrim, CoRover, and Zoho’s own AI infra team if you want internal movement.

Common Failure Modes¶

The “cheating” -O3 baseline. Your naive matmul is fast because gcc -O3 -march=native auto-vectorized it. Your SIMD version only shows a 1.4x speedup because the baseline was already vectorized. Detection: compile the naive baseline with -O2 -fno-tree-vectorize and document this. Show the assembly of the naive inner loop — it must be scalar.
Benchmark-only correctness. You benchmark speed but never memcmp the output against a reference. Your SIMD version is fast because it computes garbage. Detection: every benchmark run also asserts ||C_yours - C_ref||_inf < 1e-4 (float tolerance). This assert is part of the harness.
Roofline drawn from datasheet, not measured. You quote 128 GFLOPS from Intel’s PDF instead of running a peak-FLOPS microbenchmark. Detection: the roofline commit includes the peak-FLOPS and peak-bandwidth micro-benchmarks with their source.
The writeup is a code dump. 2000 words of pasted C with two sentences of prose. Detection: a reviewer skimming for prose finds long paragraphs about why, not what.
Only one matrix size benchmarked. You show 1024×1024 and call it done. Detection: the plot must show all six sizes. Small sizes reveal overhead; large sizes reveal memory-boundedness. Both matter.

Estimated Hours¶

Naive matmul + benchmark harness: 8h
Cache-blocked implementation + tuning: 20h
SIMD implementation (register blocking + intrinsics): 35h (this is where the pain lives)
OpenBLAS integration for reference: 4h
Roofline measurement + plot: 10h
perf / Instruments cache-miss analysis: 8h
Writeup + editing: 20h
Reproducibility (CI or Colab): 6h
Meetup submission + LinkedIn post + polish: 5h

Total: ~115 hours across M10-M11. ~14h/week for 8 weeks. Comparable weekly load to Rung 5. This is the second peak of the year.

Prior-Art / Inspirations to Study First¶

Kazushige Goto’s “Anatomy of a High-Performance Matrix Multiplication” paper — the canonical text on GEMM optimization. Dense but transformative.
flame/blis — the BLIS project source. Study frame/3/gemm/. This is what “industrial” looks like.
salykova/matmul.c and similar single-file matmul walkthroughs on GitHub — study for structure only; do not copy code.
Simon Boehm’s “How to Optimize a CUDA Matmul Kernel” blog post — CUDA, not CPU, but the methodology (progressive optimization + speedup table + roofline) is exactly what your writeup should imitate.
ggml ggml.c matmul kernels — the inference-time GEMM you’d contribute to if you ever open a PR against llama.cpp. Skim before you start; it is the target you are training for.
Agner Fog’s optimization manuals — particularly Volume 2 (“Optimizing subroutines in assembly language”) and Volume 4 (“Instruction tables”). Reference material, not front-to-back reading.

Return to README.md · Previous: 05_rung_5_epoll_http_server.md · Next: 07_rung_7_oss_merged_pr.md