18 — Phase 6 Projects: The Distributed-Scale Portfolio¶

Phase 6 concludes with four capstone projects. Each is a public artifact — a repo, a blog post, and a benchmark table you can defend line by line. Do all four and you will have compressed 18 months of on-the-job learning into a shippable portfolio.

The projects are graded not by output “quality” in a fuzzy sense but by whether you can, in a 45-minute study, walk someone through every design decision, produce the numbers on demand, and predict what would happen if a parameter changed. That is the exit test.

Project 6A — Pretrain a 124M model to a target loss with logged curves¶

Goal: own the end-to-end pretraining loop at nanoGPT scale.

Deliverables¶

GitHub repo with training script (single-file, ≤500 lines), data-prep script, config file, W&B project link.
A 124M GPT-2-style model trained on FineWeb-Edu sample-10BT (10B tokens) reaching ≤ 3.28 val loss on the FineWeb 10B held-out split. This is the modded-nanogpt speedrun target — a well-defined, community-recognized bar.
A short blog post (~1500 words) with:
- Predicted-vs-measured tokens/sec, memory footprint, and final loss (three predict-first exercises baked into the writeup)
- Full LR schedule + grad-accum config with justification
- Loss curve screenshot from W&B
- Cost accounting (hours × $/hour = total spend; expect $30-100 on 8×H100 rented)

Milestones¶

Week	Task
1	FineWeb-Edu ingestion + tokenization sanity check (round-trip an English page through the tokenizer, byte-level parity)
2	Single-GPU 124M forward/backward parity vs HF GPT-2 architecture (same weights → same loss on toy input)
3	Multi-GPU DDP or FSDP2 launch; overfit a small batch first; then a 100M-token dry run; then full 10B
4	Full run, W&B logging, eval on lm-eval-harness (HellaSwag, ARC-e, PIQA), publish

Success criteria (the exit test)¶

You can produce the loss curve on demand.
You can predict, for the same architecture at 350M or 1B, both (a) the new tokens/sec and (b) the new final loss, before running it.
You can explain every hyperparameter choice by referring to a paper or a Chinchilla-style scaling argument.

Common failure modes to avoid¶

Skipping the sanity-check phase and debugging a 10B-token training run at hour 8 (you will lose 3 days).
Using non-power-of-2 dimensions and paying a 20% throughput tax.
Forgetting torch.compile and running at 60% of achievable speed.
Not versioning the data (a re-shuffle silently invalidates comparisons).

Project 6B — Serve a 70B-class model with TP=2 and TP=4; produce the scaling report¶

Goal: internalize the economics of multi-GPU serving.

Deliverables¶

Deployment scripts for vLLM and SGLang serving Qwen2.5-72B-Instruct (or Llama-3.1-70B-Instruct) at TP=2 and TP=4.
Two artifacts:
- The scaling table (see below format) — the primary deliverable.
- A ~1000-word write-up interpreting it: where does TP scaling break, why, and what would you do about it.
Reproducible benchmark harness in the repo — one command from clone to numbers.

The scaling table (this is the artifact)¶

Run vllm bench serve (or SGLang equivalent) with a realistic trace — ShareGPT sampling, Poisson arrivals, prompt lengths 100-2000, output lengths 100-500.

Config	Throughput (tok/s)	p50 TTFT (ms)	p99 TTFT (ms)	p50 ITL (ms)	p99 ITL (ms)	Goodput @ SLO	GPU-hours per 1M output tokens
1× H100, model too big	—	—	—	—	—	—	N/A (won’t fit)
2× H100 TP=2, batch=1	…	…	…	…	…	…	…
2× H100 TP=2, batch=16	…	…	…	…	…	…	…
4× H100 TP=4, batch=1	…	…	…	…	…	…	…
4× H100 TP=4, batch=16	…	…	…	…	…	…	…
4× H100 TP=4, batch=64	…	…	…	…	…	…	…

SLO for goodput: define once (e.g., TTFT ≤ 2 s AND ITL ≤ 100 ms). Same SLO across rows.

What you must be able to explain¶

Why TP=2 → TP=4 does not give 2× throughput at batch 1 (all-reduce cost + reduced arithmetic intensity per GPU).
Which SLO breaks first as batch increases: TTFT or ITL — and why (queueing vs. batch-fill delay).
The $/1M-tokens number and how it compares to a rented API.

Stretch¶

Add a TP=8 single node row for extreme small-batch latency; note the diminishing return.

Project 6C — DPO fine-tune a 7-8B model on a preference dataset¶

Goal: own the modern alignment loop with LoRA-scale compute.

Deliverables¶

A LoRA-DPO’d Qwen2.5-7B-Instruct (or Llama-3.1-8B-Instruct) on UltraFeedback or HH-RLHF or a custom preference set you build from your own harness.
Adapter weights on HF Hub (a few MB; publish under your account).
Blog post with:
- Regression battery: pre-DPO vs post-DPO on MMLU, TruthfulQA, MT-Bench, and 3 tasks you care about (ideally CRM/tool-calling adjacent).
- Length statistics before/after (DPO commonly inflates response length — measure it).
- Preference-margin distribution (“chosen − rejected” logprob margin over training) as a training-diagnostic figure.
- Chosen β and the ablation showing why (β ∈ {0.05, 0.1, 0.3}).

Minimum evidence bar¶

Confirmed improvement on the target metric.
Confirmed no regression greater than 2 points on MMLU (typical DPO risk).
Length inflation kept below ~1.3× (otherwise the model is gaming length-biased judges).

Stretch¶

Repeat with IPO or ORPO (single-stage: SFT + DPO fused) and compare. This ablation is a strong signal — most engineers skip it.

Project 6D — Run a GRPO experiment (the rollout-infra project)¶

This is the hot-niche project. Do it even if it takes twice as long as the others; it’s the one that gets you calls back.

Goal¶

Do one full GRPO training run on a verifiable task (math or code), using vLLM as the rollout engine inside verl or TRL. Then publish an honest rollout-throughput analysis.

Deliverables¶

Repo forking verl or TRL (whichever you understand better after §16).
GRPO trained model: Qwen2.5-7B-Instruct → Qwen2.5-7B-GRPO on GSM8K (or MATH-500, or a HumanEval-style code set) — improve pass@1 by ≥5 absolute points.
The rollout-infra blog post — this is the artifact.

The blog post (this is your career artifact)¶

Structure (~2500 words):

The setup — the training loop as three graphs (rollout, reward, update). Diagram from §16 in your own words.
The numbers — for your specific run:
- Wall-clock breakdown per step (rollout ms / reward ms / update ms).
- Rollout throughput as tokens/sec (with vLLM) vs the same batch through the training-framework’s forward pass (i.e., what you’d have gotten without vLLM). Expect 3-10× win.
- GPU utilization time-series across the step (peaks and dead zones).
The weight-sync mechanism — describe what verl/TRL is doing (NCCL P2P, disk shuffle, or LoRA-only), and what fraction of step time it costs.
The staleness experiment — run one ablation where the rollout engine’s weights lag by k=1, 2, 4, 8 steps. Report training stability (loss, KL to ref) and final metric. This is the on-policy-ness experiment, and it’s the single most compelling thing you can measure in this space.
The reward-hacking observation — you will observe at least one form of it. Document it (e.g., “the model learned to output very long CoTs even when wrong; here’s the length distribution before and after”). Propose a defense.
The Zoho angle — one paragraph tying it to agentic harness / tool-calling: “the same rollout infra that trains GRPO could, at inference time, be the fleet that serves an agentic loop with prefix caching and disaggregation.”

Why this project matters more than the other three¶

Because in 2025-2026, “RLHF/rollout infrastructure engineer” is one of the highest-demand niches on the market, and it fuses:

Deep inference-engine understanding (vLLM/SGLang backend).
Distributed training understanding (FSDP2/Megatron).
Systems glue (Ray/NCCL/DTensor resharding).
ML understanding (GRPO/DPO stability).

Your Zoho background pattern-matches almost perfectly (long-running services + agentic harnesses = rollout controllers). This is the project that translates “applied ML engineer at Zoho” into “distributed RL systems engineer” without lying about your resume.

Sequencing recommendation¶

Not strictly serial — projects overlap, but treat this as guidance:

Weeks 1-5   → Project 6A (pretrain 124M)
Weeks 4-8   → Project 6B (70B TP scaling report) — cheap once rented cluster running
Weeks 7-11  → Project 6C (DPO fine-tune)
Weeks 10-16 → Project 6D (GRPO + rollout post) — the hot niche

Total: ~4 months elapsed at 12-15 hrs/week. That is the Phase 6 timeline the roadmap allocates (months 13-18).

Cost estimate (rented compute)¶

Rough all-in for all four projects:

Item	Hours	$/hr	Cost
6A pretrain (8× H100, dev + full run)	20	$24	$480
6B serving benchmarks (4× H100)	30	$12	$360
6C DPO fine-tune (2× A100)	40	$4	$160
6D GRPO run (4-8× H100)	60	$18	$1080
Total			~$2100

Do 6C first if budget is tight — it’s the cheapest and least risky.

What each project buys you at study¶

Project	study signal
6A	“I know pretraining is not magic” — filters out claim-only candidates instantly
6B	“I understand distributed serving economics” — the operator/staff signal
6C	“I’ve run alignment loops” — table-stakes for post-training roles
6D	“I can build the rollout infra you need” — the niche differentiator, and the one that combines with your Zoho background into a coherent staff-level narrative

Exit criteria for Phase 6 as a whole¶

Only tick these off honestly.

I’ve pretrained a ≥124M model with logged curves reaching a defensible loss target on FineWeb-Edu.
I’ve served a 70B-class model with TP=2 and TP=4 and produced the scaling table.
I’ve DPO’d a 7-8B model with a full regression battery, no MMLU regression.
I’ve run one GRPO experiment with vLLM-in-training rollouts and published the rollout-infra post.
I can whiteboard 5D parallelism communication patterns and communication cost per token.
I can explain when disaggregated prefill/decode pays and when it doesn’t, with numbers.
I’ve read DeepSeek-V3 + R1 and can teach the top 8 lessons from §17.
I’ve read either the HF Ultra-Scale Playbook or the Jax Scaling Book cover to cover.

Six of eight is a pass. All eight is a distinction.

The narrative once Phase 6 is done¶

After these projects your one-sentence positioning is:

“I build agentic services on top of LLMs and the inference/rollout infrastructure that serves them — I understand every layer from the HTTP request to the tensor core, including the multi-node fabric between them.”

That sentence is the reason to have done this phase. Almost nobody in the market can say it credibly. You will be able to.

Phase 7 (production/enterprise) will convert this into deployable reference architectures. But the arithmetic of ambition is set here.

References¶

modded-nanogpt — https://github.com/KellerJordan/modded-nanogpt
FineWeb / FineWeb-Edu — https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu
vLLM benchmarks — vllm bench serve --help
SGLang benchmarks — https://github.com/sgl-project/sglang/blob/main/docs/references/benchmark_and_profiling.md
verl — https://github.com/volcengine/verl
TRL GRPO — https://huggingface.co/docs/trl/main/en/grpo_trainer
UltraFeedback — https://huggingface.co/datasets/openbmb/UltraFeedback
lm-evaluation-harness — https://github.com/EleutherAI/lm-evaluation-harness

End of Phase 6 file set. All 19 files (README + 01-18) now in place in /13_MONTH_ROADMAP/07_distributed/.