09 — The Paper Canon

You need to read ~45 papers over 13 months. Not skim. Not “look at the abstract.” Read. With a pen, on paper if you can. This directory tells you which papers, in what order, and how to extract what matters.

How to read a systems paper (the only method that works)

Most papers in this canon are systems papers. They are not proofs; they are engineering artifacts. Read them differently than ML theory papers.

First pass — 20 minutes:

  1. Read title, abstract, intro, conclusion. Skip everything else.

  2. Answer in your notes: what problem? what one trick? what numbers changed? what did it cost?

  3. If you can’t answer those four questions, you didn’t understand the abstract — reread it.

Second pass — 60–90 minutes (the real read):

  1. Read section by section. Redraw every figure by hand.

  2. For every claimed number: where did it come from? What baseline? What hardware? Would the number hold on your hardware?

  3. When you hit an equation, close the paper and try to derive the next line. Then check.

  4. Log every unfamiliar term. Look up nothing until you finish the pass.

Third pass — only if the paper matters (~2 hours):

  1. Now look up the unknowns.

  2. Read the related work section carefully — it’s a free syllabus.

  3. Open the code. If the paper released code, the paper is a lie until you’ve matched a figure from the repo.

  4. Write a 200-word summary in your own words. If you can’t compress it to 200 words, you didn’t grok it.

The five extraction questions (paste this into every note)

Every paper note in this roadmap uses the same template:

# <Paper title> (<year>, arxiv:XXXX.XXXXX)

## Problem
One paragraph. What was broken before this paper?

## Key trick
ONE sentence. If it takes more, you don't understand it yet.

## The numbers
- Baseline: X on hardware Y
- After: Z on hardware Y
- The cost: (what did they trade away? memory? quality? code complexity?)

## What I extract
Bullet list of concepts I now own. Not "I read about" — I OWN.

## Open questions
Things I don't get yet. Come back after the next 3 papers.

## Cross-refs
Papers that quote this / papers this quotes.

Do not skip the “cost” line. Every serious systems paper trades something. Papers that claim pure wins are either (a) fixing a stupid earlier decision or (b) hiding the cost. Naming the cost is the mark of understanding.

Classification legend

  • [EASY] — 1 evening. Read once, get 80% of the value. Foundations papers mostly.

  • [MEDIUM] — 1 weekend. Two passes, a re-read of one section, some code inspection.

  • [HARD] — 1 week of evenings. Third-pass mandatory. Implement something small to prove you got it.

Prereq tags tell you which papers must land before this one. Do not ignore them; the canon is a dependency graph, not a list.

Where the papers live in the 13-month plan

  • Months 1–2 (Phase 0/1): Foundations (01_foundations_papers.md) — transformer + scaling + attention variants.

  • Months 3–7 (Phase 2/3): Kernels (02_kernels_papers.md) — roofline, online softmax, FA1/2/3, FlashDecoding, FlashInfer, Triton.

  • Months 8–12 (Phase 4): Engines (03_engines_papers.md) — Orca, PagedAttention, Sarathi, RadixAttention, speculative decoding lineage, structured decoding.

  • Months 11–14 (Phase 5, parallel with Phase 4 late stage): Quantization (04_quantization_papers.md).

  • Months 13–18 (Phase 6): Distributed (05_distributed_papers.md) — Megatron, ZeRO, MoE, disaggregation, DeepSeek-V3/R1, Llama-3 report.

The month-by-month reading schedule is in 06_paper_reading_schedule.md. It aligns paper reads with the phase you’re working on; don’t read the FlashAttention paper before you’ve written a naive attention kernel — you won’t feel the pain that motivates it.

The spaced-repetition rule

You WILL forget these papers. That’s normal — the fix is spaced repetition, not more reads.

Use one of:

  • Anki — for the numbers (H100 HBM bandwidth, A100 ridge point, FA1’s O(N²d²/M) formula, PagedAttention block size, etc.). ~50 cards for the whole canon.

  • A plain markdown “flash-cards.md” in your notes repo — one heading per fact, one line answer. Grep is your review app.

  • Monthly re-read of ONE paper’s abstract + your notes. Set a repeating calendar entry: first Sunday of every month, 30 minutes, pick the oldest paper you haven’t touched in 60 days.

The people who remember these papers 3 years later are the ones who wrote something using each idea. The best spaced repetition is shipping a project that uses the paper’s technique. That’s what the portfolio ladder is for.

What NOT to do

  • Do not read the canon in a burst. People who “read 40 papers this month” retain 5%. The canon is 13 months long by design.

  • Do not read every paper in a hot area. FA has 3 papers, not 15. Speculative decoding has 5 milestones, not the 200 arxiv drops. Trust the curation.

  • Do not treat papers as authoritative on today’s code. vLLM’s paper is from 2023; vLLM V1 (2025) has redesigned half the pieces. Papers explain why, source code explains what runs. Read both.

  • Do not skip the older papers. GPipe (2018) is still the mental model for pipeline stalling; you cannot understand zero-bubble scheduling without it.

The meta-canon (books to keep on the desk while reading papers)

These are not read cover-to-cover; they are references you flip open when a paper leaves you stranded.

  • CS:APP (Bryant & O’Hallaron) — for memory hierarchy and cache reasoning.

  • PMPP 4th ed. (Hwu, Kirk, El Hajj) — for CUDA execution model reference.

  • HuggingFace Ultra-Scale Playbook — for parallelism cost tables. Free online.

  • “How to Scale Your Model” by Google DeepMind — for TPU/JAX framing but universal roofline-of-systems reasoning. Free online.

  • Chip Huyen — AI Engineering (2024) — for the product/serving layer vocabulary.

Now open 01_foundations_papers.md and go.