17 — The DeepSeek-V3 & R1 Technical Reports: A Guided Reading

Primary sources:

  • DeepSeek-V3 Technical Report — arxiv:2402.01306 (DeepSeek-AI, Dec 2024)

  • DeepSeek-R1 — arxiv:2403.07691 (DeepSeek-AI, Jan 22 2025)

  • DeepSeek-V2 (MLA foundation) — arxiv:2402.01306

  • DeepSeekMoE — arxiv:2402.01306

  • DeepSeekMath (GRPO) — arxiv:2402.01306

Why these two reports are THE most instructive public documents

Every phase of this roadmap converges here. DeepSeek-V3 is a single frontier-scale system where architecture, numerics, parallelism, and inference economics are co-designed — and the authors were unusually candid about all four. R1 then shows how to bolt a reasoning-oriented RL loop on top with minimal infra changes.

If you read only three papers in this entire roadmap, read V3, R1, and the Llama-3 infrastructure sections. Everything you learned in phases 1–5 (attention math, KV cache economics, quantization, kernel fusion, engine internals) exists to let you appreciate these two documents.

The V3 report has 7 sections, ~50 pages, ~30 figures. The R1 report is much shorter (~15 pages) but denser per-page. Budget ~15 hours of active reading for both, split over two weeks.

Part 1 — DeepSeek-V3 report: section-by-section reading plan

The headline numbers you should memorize

Metric

Value

Why it matters

Total parameters

671 B

Frontier scale

Activated per token

37 B

Compute-frugal per forward

Training tokens

14.8 T

Chinchilla-plus

Training GPU hours

2.788 M H800-hours

~$5.6M at $2/hr — order of magnitude cheaper than headlines

Context length

128 K

Long-context native

Precision

FP8 (mixed with bf16)

First frontier model trained mostly in FP8

Loss spikes

Zero irrecoverable

Numerical discipline paid off

Print those numbers. When someone claims “training frontier models costs $100M+,” you have a counterexample and know how they did it.

Section 1 — Introduction & architecture overview (30 min)

What to extract:

  • V3 is a fine-grained MoE: 256 routed experts + 1 shared expert per layer, top-8 routed.

  • Uses MLA (Multi-head Latent Attention) — inherited from V2 — for KV cache compression.

  • Uses MTP (Multi-Token Prediction) — trains extra heads to predict token t+2, t+3, etc.

  • FP8 training with fine-grained scaling.

Draw the block diagram from memory before moving on. If you can’t, re-read.

Section 2 — Architecture: MLA (2 hours)

MLA is the single most inference-economical idea in the entire report. Read this section twice.

Standard MHA KV cache per token: $$ \text{KV bytes} = 2 \times n_{\text{layers}} \times n_{\text{kv_heads}} \times d_{\text{head}} \times \text{dtype_bytes} $$

For a hypothetical 671B dense with MHA: prohibitive. GQA helps ~4×. MLA goes further:

  • Project K, V through a low-rank down-projection into a compressed latent vector $c^{KV}$ of dim $d_c$ (typically ≪ n_heads · d_head).

  • Only cache $c^{KV}$ (and a small decoupled RoPE key $k^R$).

  • Reconstruct per-head K, V on the fly via up-projections, which can be absorbed into the query and output matrices — so the on-the-fly cost at attention time is minimal.

Concrete V3 numbers:

  • $d_c = 512$ (KV compression dim), $d^R = 64$ (decoupled RoPE dim)

  • Per-token cache: (512 + 64) × n_layers × 2 bytes (bf16)

  • For 61 layers: ~70 KB per token

  • vs GQA with 8 KV heads × 128 dim × 61 layers × 2 = 125 KB per token

  • vs MHA-style at 128 heads: ~2 MB per token

At 128k context, batch 32: MLA fits in ~280 GB KV vs multi-terabyte for naive MHA. This is why long context works economically.

Exercise: derive on paper why the up-projection can be “absorbed” into $W_Q$ and $W_O$ (hint: the up-projections are constant per-head matrices, so $Q W_Q W_{K,up}^\top c^{KV} = (Q W_Q W_{K,up}^\top) c^{KV}$).

Read the MLA equations (V3 paper §2.1.1, or V2 paper for the full derivation) until you can rewrite them without looking.

Section 2 (continued) — DeepSeekMoE architecture (1.5 hours)

Two innovations vs vanilla MoE (Switch/GShard):

  1. Fine-grained experts — splitting experts into more, smaller units (256 experts per layer, each much narrower than a Switch-Transformer expert) increases combinatorial specialization. More flexibility per parameter budget.

  2. Shared expert(s) — 1 always-on expert per layer handles the common patterns; routed experts specialize.

Then the killer detail: auxiliary-loss-free load balancing. Read §2.1.2 very carefully.

Traditional aux-loss balancing (Switch): add α · CV(fraction routed to each expert)² to the training loss. Problem: it fights the actual task loss, hurting quality at scale.

DeepSeek’s solution:

  • Each expert has a bias $b_i$ added only to routing scores (not to the actual output).

  • After each step, if expert $i$ was over-utilized in the batch, b_i -= γ; if under-utilized, b_i += γ.

  • Bias never enters the loss. Balance emerges from a control loop, not a gradient signal.

Result: near-perfect balance without task-loss degradation. This is the technique to reproduce in any MoE work you do.

Also read: node-limited routing (each token routes to experts on at most M nodes), which caps all-to-all traffic.

Section 2.2 — Multi-Token Prediction (MTP) (45 min)

Instead of predicting only token t+1, predict t+1, t+2, …, t+D with sequentially-conditioned heads (each head sees the previous prediction’s hidden state).

Two payoffs:

  1. Densifies the training signal — better data efficiency.

  2. The extra heads become a natural drafter for speculative decoding at inference. V3 reports 85–90% acceptance rate for the MTP head at position t+2 — near-free 1.8× decode speedup.

This is the single tightest connection you’ll see between training design and inference economics. Note it, memorize it, quote it in studies.

Section 3 — Infrastructure (3 hours — the crown jewel)

This section is the training-systems masterclass. Slow down here.

§3.1 Compute cluster: 2048 H800 GPUs. Not H100. Read the H800 spec sheet: NVLink bandwidth cut in half vs H100 (~200 GB/s vs 450 GB/s), same HBM. The entire training strategy is shaped by this constraint. Whenever V3 does something that looks over-engineered, ask: “would this have been necessary on H100?” — often no.

§3.2 Training framework — HAI-LLM:

  • Parallelism: 16-way PP × 64-way EP × ZeRO-1 DP. Zero tensor parallelism. This is worth pondering.

  • DualPipe: their custom pipeline schedule that overlaps forward and backward passes across microbatches to hide the H800 NVLink bottleneck. Read Figure 4 until you understand the F/B bubble reduction.

  • All-to-all optimization: custom kernels for the MoE dispatch/combine step, warp-specialized to overlap with computation.

  • Recomputation strategy: selective — recompute RMSNorm and MLA up-projections, not attention.

  • Bit-level memory optimizations: stored optimizer states in bf16 with fp32 master copies only for specific slices; exponential moving avg on CPU.

§3.3 FP8 training — the technical highlight (read TWICE):

This is the first frontier-scale demonstration that FP8 training works end-to-end.

Key techniques:

  • Fine-grained scaling: tile-wise for activations (1×Nc), block-wise for weights (Nc×Nc, with Nc=128). Not per-tensor.

  • Increased accumulation precision: partial sums in the MMA are periodically moved from the tensor-core FP32 accumulator (which on Hopper has limited precision) to a full-precision FP32 register accumulator, every N accumulations.

  • E4M3 for both forward and backward (previously papers used E5M2 for gradients). E4M3 has more mantissa precision, less range — the fine-grained scaling compensates for the reduced range.

  • Master weights stay in FP32; optimizer states in BF16.

  • Kept in higher precision: embeddings, output head, MoE gating, normalization, attention operators.

Numerical outcome: relative loss error < 0.25% vs bf16 baseline over 1 T tokens. No loss spikes.

Draw the FP8 training numerical flow from memory: FP8 tensors → tile-scaled MMA → periodic FP32 promotion → FP32 accumulate → FP8 output. This is the diagram Hopper/Blackwell training will use for the next 3 years.

§3.4 Inference & deployment (2 hours):

The V3 inference deployment plan (§3.4) is the reference architecture for MoE serving:

  • Prefill stage: TP=4 within node + EP=32 across 4 nodes + SP=8 (sequence parallel). Each prefill unit = 4 nodes × 8 GPUs.

  • Decode stage: TP=4 + EP=320 across 40 nodes. Each expert lives on exactly one GPU (with a few duplicated hot experts).

  • Disaggregated prefill/decode — different physical clusters, exactly the pattern from 09_disaggregated_serving.md.

  • Redundant experts for load balancing — hot experts are duplicated across GPUs, so more incoming tokens can be dispatched.

  • Micro-batch overlap of communication and compute.

Notice how far this deployment is from what a solo tinkerer can reproduce. A single-node vLLM serving V3 is essentially impossible without severe quantization; even 8×H100 barely fits. This is why wide-EP serving as a category exists.

Section 4 — Pre-training (1 hour)

  • Data: 14.8 T tokens. Rebalanced toward math/code vs V2. Multi-lingual.

  • LR schedule: step-wise with warmup, one big cosine phase, then a decay phase; final “annealing” phase at high-quality data (a Chinchilla-style refresh).

  • Training stability: the report emphasizes that with proper FP8 discipline + aux-loss-free balancing, not a single loss rollback was required over the entire run. This is the strongest possible endorsement of the numerical stack.

Section 5 — Post-training (30 min)

  • SFT on 1.5M curated instances.

  • RL via GRPO (introduced in the DeepSeekMath paper, arxiv:2402.03300).

  • Reward model built on judgment tasks.

  • Distillation from R1 (reasoning boost).

The V3 report leaves reasoning details thin because they were saved for R1.

Section 6 — Evaluation (skim)

Standard benchmarks. V3 beats Llama-3-405B on most, matches GPT-4o. Skim; the numbers age fast, the methodology doesn’t.

Section 7 — Conclusion / Limitations (10 min)

The candid limitations paragraph is worth reading. Themes: EP overhead, still-large inference infra, single-provider ecosystem.

Part 2 — DeepSeek-R1 report: guided reading

R1 is a short but paradigm-shifting paper. Two models, two ideas:

R1-Zero — pure RL from base

  • Start from DeepSeek-V3-Base (no SFT).

  • Apply GRPO with only rule-based rewards (correctness on math/code — verifiable).

  • Emergent behaviors: increasing response length, self-reflection (“wait, let me reconsider…”), verification loops.

  • Result: strong math/reasoning performance.

  • Limitation: language mixing, poor readability.

The insight to take: reasoning behaviors can emerge from scaling RL alone on verifiable rewards, without any SFT trace demonstration. This was the shock in Jan 2025.

R1 — cold-start + multi-stage

Recipe:

  1. Small cold-start SFT on curated CoT (a few thousand examples) — fixes readability.

  2. RL on reasoning tasks (like R1-Zero).

  3. Rejection sampling: generate many CoTs, keep the correct ones, filter for readability.

  4. SFT on the filtered set + non-reasoning data.

  5. Final RL for helpfulness/harmlessness across all task types.

Distillation section: they take R1-generated CoTs, SFT-only train small models (Qwen-7B, Qwen-32B, Llama-8B, Llama-70B) on them. The distilled models are strong — Qwen-32B distilled beats o1-mini on some math benchmarks. This is what the open-source ecosystem then built on.

Failed experiments (§ appendix — read this)

The paper explicitly documents:

  • Process reward models (PRMs) — hard to train, susceptible to reward hacking.

  • MCTS — search space too large in token space, value function unstable.

Both are interesting negatives — the field spent a lot of 2024 on PRMs and MCTS; DeepSeek’s negative result reoriented much of 2025.

Part 3 — What this changed in the field (2025 impact)

The R1 report caused three tectonic shifts:

  1. RL from base with rule rewards is a legitimate strategy. Every lab that had assumed you need PRMs, MCTS, or process supervision had to update. Kimi K1.5, Qwen-QwQ, and many others followed within months.

  2. Rollout infra became a first-class systems problem. GRPO with G=8 rollouts × 4096 tokens per prompt × thousands of prompts per step blew up cluster utilization. This is why 16_rollout_infra.md exists as its own topic. verl, OpenRLHF, AReaL exploded in adoption.

  3. Open-weight reasoning distillation became a democratization axis. Small teams that couldn’t afford R1-scale RL runs SFT-distilled from R1 outputs and got 90%+ of the quality. The “reasoning tax” ceased to be exclusive.

The Zoho-relevant subplot: R1-Distill-Qwen-14B and Qwen-32B are the current sweet-spot on-prem reasoning models. Both fit a single 2×A100/L40S deployment. If your CRM/agentic product wants “thinking mode” without shipping data to an API, this is the class of model you deploy. You should know it cold.

Part 4 — Extract these lessons into your notebook

Write these as one-liners in your learning notebook:

  1. MLA compresses KV cache 10–30× with near-zero quality loss — the up-projection can be absorbed.

  2. Aux-loss-free MoE balancing works via a routing bias updated by a control loop, not gradient.

  3. MTP training heads → speculative decoding drafters at inference. Free 1.8× decode.

  4. FP8 tile/block-scaling + periodic FP32 promotion = frontier training in 4× fewer memory bytes.

  5. Zero TP + 16 PP + 64 EP — bandwidth-constrained hardware pushes you toward EP-heavy strategies.

  6. Disaggregated prefill/decode with EP=320 for decode is the endgame of MoE serving.

  7. Pure RL from base with verifiable rewards can elicit CoT reasoning (R1-Zero).

  8. Distillation from strong reasoners collapses the compute barrier — CoTs are the transferable currency.

If you can whiteboard 6 of these 8 cold, you have absorbed the reports properly.

Part 5 — Exercises

  1. KV cache math. Compute the KV cache size for V3 at 128k context, batch 32, and compare against a hypothetical 671B GQA-8 (same layers) at the same context/batch. Present the ratio. Then compute what fraction of a single H800’s 80 GB HBM each version consumes for KV.

  2. DualPipe drawing. From the V3 paper Figure 4, redraw the DualPipe schedule for 4 stages × 8 microbatches on paper. Identify every bubble; count them; compute the theoretical vs achieved efficiency.

  3. FP8 numerical experiment. Take a small GEMM (1024×1024×1024). Run it in bf16 vs fp8-e4m3 with (a) per-tensor scale (b) per-tile scale. Measure relative error vs fp32 reference. Confirm that fine-grained scaling is doing work.

  4. MTP verifier. Read the V3 MTP §2.2 carefully, then implement a toy MTP head on top of a GPT-124M. Measure the acceptance rate you’d get on validation data if the MTP predictions were used as speculative decoding drafts.

  5. R1 distillation replica. Pick one R1-Distill checkpoint (Qwen-7B). Run 100 GSM8K problems through both R1-Distill-Qwen-7B and vanilla Qwen-7B-Instruct. Report the accuracy gap and reasoning-length distribution. Publish the notebook.

  6. Serving cost model. Given V3’s §3.4 deployment plan (prefill: 4 nodes × 8 H800, decode: 40 nodes × 8 H800), and assuming $2/H800-hour, compute the marginal cost per 1M output tokens if the fleet hits 200 tok/s per user × 100 concurrent users decode-side. Compare to GPT-4o list price.

Part 6 — Companion papers to read alongside

  • DeepSeek-V2 arxiv:2402.01306 — the MLA and DeepSeekMoE origin story. If §2 of V3 confuses you, back up here.

  • DeepSeekMoE arxiv:2402.03300 — fine-grained + shared expert derivation.

  • DeepSeekMath arxiv:2402.01306 — GRPO’s home paper. Read the algorithm section.

  • Kimi K2 arxiv:2403.07691 — the other 2025 open MoE frontier report; compares/contrasts with V3 on many design axes.

  • Llama-3 report — the Meta counterpart, especially the 4D parallelism and infrastructure sections. Reading V3 and Llama-3 side-by-side is the highest-density training-systems education available for free anywhere.

Reading schedule

Two weeks, ~8 hours/week:

Day

Content

Hours

1

V3 §1 + §2.1.1 (MLA)

2

2

Re-read MLA, derive absorption on paper

1.5

3

V3 §2.1.2 DeepSeekMoE + aux-loss-free

1.5

4

V3 §2.2 MTP

1

5

V3 §3.1–3.2 (cluster, HAI-LLM, DualPipe)

2

6

V3 §3.3 FP8 training (twice)

2

7

V3 §3.4 inference deployment

2

8

V3 §4 pre-training + §5 post-training

1.5

9

R1 §1–2 (R1-Zero + emergent RL)

1.5

10

R1 §3 (cold-start pipeline)

1

11

R1 §4 (distillation) + failed experiments

1

12

Complete exercises 1, 3, 5

3

Log observations in your notebook after each session. This is training-systems literacy at the highest level available in public.

Big picture — why this ties the whole roadmap together

Everything from Phase 1 through Phase 6 exists so you can read these reports as an author, not as a spectator:

  • Phase 1 (transformer arithmetic) → you can derive MLA absorption yourself.

  • Phase 2 (GPU arch, tensor cores) → you understand why FP8 tile-scaling is necessary on Hopper.

  • Phase 3 (attention kernels) → you can imagine writing the MLA kernel.

  • Phase 4 (engine internals) → you can imagine the V3 inference deployment as a scheduler problem.

  • Phase 5 (quantization) → you understand what “no loss spikes with FP8” costs in engineering.

  • Phase 6 (parallelism, disaggregation) → you can predict what happens if you change PP=16 to PP=8, or move EP=64 → EP=128.

If, after reading V3+R1, you finish this file thinking “I could have contributed to this project,” you’re ready for Phase 7 and, more importantly, for the studies that follow.

References

Next: 18_projects.md — the phase-closing capstone projects that convert this reading into artifacts.