04 — Developing a Technical Opinion

The most common failure mode of a senior ML engineer is terminal neutrality: “it depends” on every contested question, no stake in any claim, no prediction that can be falsified. This is not wisdom. It is the intellectual equivalent of hiding behind probabilistic hedging to avoid being wrong. A technical opinion is not a preference — it is a structured belief backed by evidence, with explicit conditions under which you’d change your mind.

This file is about building the discipline to form, defend, update, and communicate technical positions. That discipline is what separates a practitioner who is worth talking to from one who merely knows the same things everyone else knows.

Why a Technical POV Matters More Than Knowing Everything

In 2025, knowing what transformers are is table stakes. Every engineer with six months in the field knows what attention is. What differentiates you:

  • You have an informed position on contested questions. “Are SSMs replacing transformers?” is not a question with a consensus answer — it requires you to have read Mamba, evaluated the failure modes on in-context retrieval benchmarks, understood what tasks transformers remain dominant on, and formed a time-horizon claim.

  • Your position is falsifiable. “SSMs will be the dominant architecture for audio and genomics tasks at the 1B parameter scale by end of 2025, but will not displace transformers for general-purpose LLMs in production by that date.” That is a claim. “It depends on the use case” is not.

  • You update publicly and explicitly. When new evidence arrives (e.g., Jamba, Mamba-2, Falcon Mamba benchmarks), you revise the claim and record what changed your mind.

This is how technical credibility compounds. Opinions that age well (or fail gracefully with explicit reasoning) make you worth following.

How to Form a Technical Opinion: The Structured Method

The Contested Question Framework

Pick a contested technical question. Examples:

  • “Are SSMs a credible replacement for transformers in LLM pretraining at scale?”

  • “Will quantization to 4-bit be the default serving strategy for 70B+ models by 2026?”

  • “Is synthetic data for instruction tuning better or worse than human-curated RLHF data?”

Step 1 — Map the Claims Landscape (30 minutes)

Write down every position you’ve heard stated on this question. Note who holds it and what evidence they cite. Do not evaluate yet. You are building a map, not a verdict.

Step 2 — Identify the Core Mechanism (1-2 hours)

What is the underlying technical mechanism that determines the answer? For SSM vs transformer: the question reduces to “what is the information bottleneck in the SSM recurrence formula, and is it fundamental or engineering-solvable?” Reading Mamba + Mamba-2 + the RWKV and RetNet papers gives you the mechanism, not just the benchmarks.

Step 3 — Find the Strongest Counterevidence (1 hour)

Actively seek the evidence that most damages your current leaning. If you think SSMs will not replace transformers, find the best benchmark that shows SSMs outperforming transformers. If you think they will, find the ablation that shows SSM failure on in-context retrieval tasks. The quality of your opinion is determined by the quality of the counterevidence you’ve processed.

Step 4 — Define Your Conditions for Being Wrong

Write: “I would update this position if [X]. Specifically, if [benchmark / paper / experiment result], I would revise to [alternative position].”

This is not weakness. This is Bayesian honesty. It is also what makes you worth arguing with.

Step 5 — State the Claim with Time and Scope

“SSMs (Mamba-class architectures) will not achieve parity with transformers on general-purpose language benchmarks (MMLU, HumanEval, GPQA) at the 7B+ parameter scale by end of 2025, but will outperform at equivalent parameter count on structured sequence tasks (audio, genomics, long-document summarization where in-context retrieval is not required).”

That is an opinion. Write it down. Date it.

The Practice of Prediction + Measurement

What most people get wrong: They form opinions but never record them, so they never learn from their failures. The best technical thinkers keep prediction logs — not to be right, but to calibrate.

Keep a prediction log. Format:

## Prediction Log

| Date | Claim | Domain | Confidence (%) | Resolution Date | Outcome | Lesson |
|------|-------|--------|----------------|-----------------|---------|--------|
| 2025-01-15 | GRPO will replace PPO as default RL algo for LLM training in open-source repos by Q3 2025 | RL alignment | 72% | 2025-10-01 | PENDING | — |
| 2025-02-01 | 4-bit quantization (GGUF/GPTQ) will be default serving format for 7B-13B models in production | Inference | 85% | 2025-12-01 | PENDING | — |

Review quarterly. Grade yourself. Adjust calibration. After 12 months, you will have a data-driven understanding of where your instincts are sharp and where they are noise.

Calibration research (Tetlock, Superforecasters) shows that the top ~2% of predictors are distinguished not by domain knowledge alone, but by the habit of recording and reviewing predictions systematically.

The Feynman Test Applied to ML

Richard Feynman’s test: “If you can’t explain it simply, you don’t understand it.” Applied to ML:

The Feynman Test for ML understanding: Can you explain the core mechanism of a paper to a smart software engineer who has never read it — using only a whiteboard and no jargon — in 10 minutes, such that they could implement a simplified version?

If yes: you understand it.
If no: you’ve read the words, not the idea.

Apply this test to yourself regularly:

  • Can you explain grouped-query attention to a non-ML engineer in 5 minutes?

  • Can you explain why GRPO removes the need for a critic network without using the words “GRPO” or “PPO”?

  • Can you explain why Flash Attention is faster without using the word “memory”?

Keep a running list. “Things I think I understand” → stress-test with Feynman. The gap between the two columns tells you where your learning is shallow.

How to Write a Technical Blog Post That Is Worth Reading

95% of ML blog posts are not worth reading. They are either:

  1. Paper summaries with no original insight (“Here is what the paper says”)

  2. Tutorials that exist 400 copies already on Medium

  3. Opinion pieces with no supporting evidence (“I believe SSMs are the future”)

A blog post is worth reading if it provides one of these things that the reader cannot get elsewhere:

  1. Original reproduction evidence: “I reproduced Flash Attention 2 from scratch. Here is where the math paper is unclear, here is the kernel code, here is the benchmark I ran.”

  2. Quantified comparison: “I compared PEFT methods on 7B Llama 3 for instruction tuning. LoRA rank 64 vs QLoRA vs full fine-tune. Here are the loss curves, MMLU delta, and GPU-hours.”

  3. A falsifiable prediction with reasoning: “Here is why I think MoE will be the dominant architecture for 100B+ parameter models by 2026, and here is what would change my mind.”

  4. A failure post-mortem: “I tried to replicate this paper result and couldn’t. Here is what I found.”

The structure that works:

Hook (1 paragraph): What's the contested question or surprising result?
Context (2-3 paragraphs): Why does this matter? What's the prior state of knowledge?
The finding (your contribution): Experiment, reproduction, or structured argument
Evidence: Numbers, charts, code snippets — not claims
What this changes: Implications. What would you do differently now?
What remains unknown: Intellectual honesty

Distribution matters. A blog post with zero readers is a diary entry. Post on:

  • Substack (for long-form technical subscribers)

  • HuggingFace Spaces / model card (for technical communities)

  • Twitter/X with a thread summary (for discovery)

  • r/MachineLearning (if it has original content — they will know if it doesn’t)

Target: 500+ genuine reads in first 2 weeks, at least 3 substantive comments from people who clearly read it. That is a reasonable bar for “worth writing.”

How to Run a Paper Discussion Group

A paper reading group is the highest-ROI learning activity available to a working engineer. The Latent Space Paper Club maintained an 18-month unbroken streak at ~80 papers. That is roughly the top 2% of ML reading groups by longevity.

The format that sustains (evidence: Latent Space, Eugene Yan model):

  • Frequency: Weekly or biweekly. Monthly is too infrequent to build momentum.

  • Size: 4-8 people. Below 4, cancellations kill it. Above 8, discussion fragments.

  • Pre-read requirement: 1-hour minimum async read before the session. Non-negotiable. Skipping pre-read reduces discussion value by approximately 80%.

  • Facilitator role: The facilitator’s job is NOT to present the paper. It is to prepare 5-7 discussion questions and drive the group toward the interesting disagreements. Bad facilitators present. Good facilitators provoke.

  • Discussion questions submitted 48 hours in advance: Ask all attendees to submit one question they want answered. Pre-work forces engagement before the session.

Questions worth asking in every session:

  1. What is the core claim, in one sentence?

  2. What experiment would disprove this claim?

  3. What is the most suspicious ablation or missing ablation?

  4. What does this paper not compare against, and why might that be?

  5. If you were to implement this, what would you do differently?

The format that fails: One person presents slides, group listens passively for 45 minutes, 5 minutes of polite questions, everyone leaves. This is a lecture, not a discussion.

What most people get wrong: Trying to read every paper in the group’s selection instead of picking one paper per session and reading it deeply enough to disagree with the authors.

The Technical Opinion Stack

By the end of Month 13, you should have at least 5 formed, documented technical opinions:

  1. One on architecture (e.g., transformer vs SSM vs hybrid)

  2. One on training methodology (e.g., RL alignment methods)

  3. One on inference/deployment (e.g., quantization strategies)

  4. One on data (e.g., synthetic vs human-curated for instruction tuning)

  5. One on the field’s direction (e.g., scaling law trajectory, reasoning model emergence)

These are not permanent. They are living documents. What makes them valuable is not being right — it is being structured about being wrong.

Return to README.md · Previous: 03_open_source_contribution.md · Next: 05_staying_current_system.md