North Star Artifacts

The 8 Portfolio Rungs — Mapped, Specified, Graded

These are the 8 things you will have shipped publicly by Month 13. Not summaries of what you learned. Not certificates. Artifacts: code repositories, model weights, live endpoints, published writing. Things a senior engineer or hiring manager can click on and evaluate in 10 minutes.

Each rung is harder than the last. Each one builds on the previous. Missing a rung does not mean you skip to the next — it means you go back and finish. The two hard gate rungs (⛔) are binary: they either exist with all required artifacts, or Month 13 has not been reached.

Rung 1 — Math From Scratch

Month: M2 (Sprint 4) What it is: A Jupyter notebook on GitHub implementing SVD from scratch in NumPy and gradient descent on a non-trivial loss landscape, with visualizations. Signal it sends: You are not a math tourist. You implemented the thing, not read about it. Where published: GitHub (public repo, clean README) Specificity gradient: Notebook must include: matrix decomposition with hand-verified result, gradient descent on a 2D bowl function with step-size sensitivity plot, and one paragraph explaining what eigenvalues are in plain language. Acceptance criteria:

  • SVD matches NumPy’s np.linalg.svd output for a 4×4 matrix

  • Gradient descent converges on f(x,y) = x² + 5y² with ≥3 learning rate experiments

  • README explains the notebook in 3 sentences a non-ML engineer can understand

  • GitHub repo is public, starred or unstarred — doesn’t matter. It exists.

What weakens this rung: Using sklearn’s PCA or scipy’s SVD and calling it “from scratch.” Evaluators who know ML will check your imports.

Rung 2 — Classical ML Battle

Month: M4 (Sprint 8) What it is: A GitHub repo benchmarking 3 classical ML models (e.g., Logistic Regression, Random Forest, XGBoost) on a public tabular dataset, with full evaluation: ROC curves, precision-recall curves, calibration plots, feature importance. Signal it sends: You can build a complete, reproducible ML pipeline — not just fit a model. Where published: GitHub (public repo) + brief writeup posted to personal blog or LinkedIn Specificity gradient: Dataset must be real (Kaggle, UCI, or similar) with ≥10K rows. Pipeline must be reproducible end-to-end: python run.py from a clean environment produces identical results. Acceptance criteria:

  • 3 models, all evaluated with ROC-AUC, PR-AUC, and calibration curve

  • Holdout test set never touched during model selection (held out at the start)

  • requirements.txt pins all versions

  • README has a results table with numbers, not narrative

  • Pipeline is reproducible by a stranger with no context

What weakens this rung: Only accuracy reported. No calibration. Test set used for tuning. These are the exact red flags an interviewer looks for.

Rung 3 — Deep Learning From Scratch

Month: M6 (Sprint 12) What it is: Three artifacts in one repo: (1) a complete backpropagation engine in pure NumPy, (2) a CNN trained on CIFAR-10 from scratch in PyTorch (>85% accuracy), (3) a character-level language model with training curves. Signal it sends: You understand what is actually happening inside a neural network. Not “transformers use attention” — the actual gradient math. Where published: GitHub (public repo) + HuggingFace Spaces demo for the CharLM (optional but strong) Specificity gradient: Backprop engine must handle arbitrary computation graphs. CNN must document the architecture decision (why these layers?). CharLM must show loss curve, not just a sample output. Acceptance criteria:

  • Backprop gradients match PyTorch autograd on the same computation (test script included)

  • CNN: CIFAR-10 test accuracy ≥85%, training curve plotted, confusion matrix included

  • CharLM: loss descends over training, generated sample is coherent (not random characters)

  • Each artifact has its own README section explaining the design choices

What weakens this rung: Using a tutorial notebook without modification. If the architecture is exactly Karpathy’s nanoGPT with your name on it, that’s not a rung. Apply it to your own dataset or problem.

Rung 4 — Transformer Lab

Month: M8 (Sprint 16) What it is: Three artifacts: (1) a character-level GPT trained on a domain-specific corpus (not Shakespeare), (2) a fine-tuned BERT or similar on a classification task, (3) DDPM trained on MNIST with sample grid and loss curve. Signal it sends: You understand the modern architecture landscape — not just one model, but the family and its variants. Where published: GitHub (public repo) + HuggingFace Hub for the fine-tuned model Specificity gradient: The GPT corpus must be your choice — pick something that interests you (code, scientific abstracts, lyrics, etc.). The BERT fine-tune must include a proper evaluation against a baseline. Acceptance criteria:

  • nanoGPT: perplexity reported on held-out set, sample outputs included

  • BERT fine-tune: F1 score on test set, comparison to a logistic regression baseline

  • DDPM: 8×8 sample grid of generated digits, loss curve over training

  • All models on HuggingFace Hub with model cards

What weakens this rung: Running pre-trained models without fine-tuning. Fine-tuning without a baseline comparison. No model cards.

⛔ Rung 5 — LLM Engineering (HARD GATE — Month 10)

Month: M10 (Sprint 20) What it is: Four artifacts that must exist simultaneously: (1) fine-tuned model weights on HuggingFace Hub with model card, (2) RAGAS evaluation scores in a public document, (3) live accessible demo URL, (4) RAG pipeline code on GitHub. Signal it sends: You are not an API wrapper. You have trained, evaluated, deployed, and documented an LLM system end-to-end. Where published: HuggingFace Hub (model) + GitHub (code) + HuggingFace Spaces or Replicate (demo) + public Google Doc or GitHub markdown (eval report) Specificity gradient: The fine-tune task must be specific and chosen for a reason (not generic instruction following). The RAG pipeline must use a real corpus (not Wikipedia toy examples). RAGAS scores must be real numbers from a real run. Acceptance criteria:

  • Model is publicly accessible on HuggingFace Hub with a model card

  • RAGAS: faithfulness, answer relevancy, context precision all reported as numbers

  • Demo URL loads and returns a response within 30 seconds (tested from a different network)

  • RAG pipeline README has a “quick start” that runs in under 5 commands

  • All 4 artifacts are live simultaneously, not staged sequentially

Disqualifiers (these make this rung invalid):

  • API-wrapping GPT-4 and calling it a fine-tuned model

  • RAGAS scores from a test fixture, not a real corpus

  • Demo that only works locally

  • “Evaluation in progress” placeholder

Gate failure protocol: See 99_pre_mortem/09_summary_and_reset_protocol.md. Do not proceed to Phase 5 until all 4 artifacts exist.

⛔ Rung 6 — Production ML System (HARD GATE — Month 12)

Month: M12 (Sprint 24) What it is: A live, monitored ML system with five verifiable artifacts: (1) live inference endpoint, (2) DVC + MLflow experiment lineage, (3) GitHub Actions CI/CD pipeline with green run history, (4) Evidently monitoring report with real drift data, (5) runbook document. Signal it sends: You have shipped ML to production. Not “I understand the concepts.” I have done it and it is running. Where published: GitHub (code + Actions history) + hosted endpoint (any cloud) + Evidently report as GitHub markdown or public doc + runbook in repo Specificity gradient: The system must be live at interview time — not “I had it running last month.” The Evidently report must have been generated from real inference data, not synthetic drift injection. The CI/CD must show at minimum 5 successful pipeline runs. Acceptance criteria:

  • Inference endpoint returns a prediction from a public URL (not localhost)

  • GitHub Actions: ≥5 successful runs visible in repository history

  • DVC pipeline: dvc repro runs without error from a clean clone

  • MLflow: experiment comparison between ≥3 model versions visible in UI

  • Evidently: monitoring report shows at least one real drift metric on production data

  • Runbook: step-by-step restart procedure, estimated time to recovery, owner defined

Disqualifiers:

  • System that only runs locally

  • CI/CD that only lints (no model validation)

  • Evidently run on synthetic data with no real inference history

  • Runbook that says “see README”

Gate failure protocol: Same as Rung 5. Fix this before Month 13 activities.

Rung 7 — Paper Reproduction

Month: M12 (Sprint 24, alongside Rung 6) What it is: A GitHub repository reproducing a key result from one ICLR/NeurIPS/ICML 2023-2025 paper, with a documented deviation analysis explaining where your results differ from the paper’s and why. Signal it sends: You can read the research literature and translate it into working code. This is the research-practitioner bridge. Where published: GitHub (public repo with deviation analysis in README) Specificity gradient: Choose a paper with a concrete, measurable result (not a theoretical claim). The deviation analysis is the most important artifact — it shows you understand the gap between paper ideal and implementation reality. Acceptance criteria:

  • Paper citation in README (arXiv ID, authors, venue)

  • Core result reproduced to within ±5% of reported numbers (or deviation explained)

  • Deviation analysis section: what differed, computational reason, whether it matters

  • Runtime estimate included (so others know what hardware is required)

What weakens this rung: Reproducing a paper that already has 50 GitHub reproductions with no new analysis. Pick something underrepresented. The frontier papers list is in 07_phase_6_research_and_mastery/02_frontier_papers_2024_2025.md.

Rung 8 — Technical Writing

Month: M13 (Sprint 25-26) What it is: 3 technical blog posts published publicly + one conference talk submitted or one paper review submitted to a workshop/venue. Signal it sends: You can teach what you know. You are contributing to the field, not just consuming it. Where published: Substack, Medium, personal site, or dev.to (your choice — pick one and be consistent). Talk submitted to a meetup, local ML conference, or workshop at NeurIPS/ICML/ICLR. Specificity gradient: Posts must be technical with code, not narrative summaries. Each post must make one argument the reader can act on. The talk/review submission has a specific venue — not “I plan to submit.” Acceptance criteria:

  • 3 posts published at the same URL/publication (consistent presence)

  • Each post has at least 500 words and at least one code block that runs

  • Talk submitted (rejection is acceptable — submission is the artifact)

  • At least one post has been shared in one community (Reddit, Discord, Twitter/X) and received ≥1 response

What weakens this rung: “I’ll post later.” Drafts in Notion. Three posts in Week 52. Consistency of publication matters. Start writing in Month 11.

The Portfolio at M13 — What a Reviewer Sees

When someone reviews your portfolio at Month 13, they can verify, in order, in under 20 minutes:

  1. You understand the math (Rung 1)

  2. You can build a real pipeline, not just fit.predict (Rung 2)

  3. You know what’s inside the black box (Rung 3)

  4. You understand the modern architecture landscape (Rung 4)

  5. You have deployed an LLM system end-to-end (Rung 5 ⛔)

  6. You have shipped a monitored production system (Rung 6 ⛔)

  7. You can read and reproduce research (Rung 7)

  8. You can teach (Rung 8)

No amount of verbal fluency substitutes for any of these. They are verifiable in 10 minutes. That is why they are the target.

Return to: README.md · Next: 04_weekly_rhythm.md