Phase 5: Production & MLOps (Months 11–12)¶

You have built models. You have fine-tuned LLMs. You have implemented architectures from scratch. None of that matters if the model cannot survive contact with production traffic, real users, and time. Phase 5 is where applied ML engineering diverges from research — and where most ML practitioners discover they have been building science projects, not systems.

The brutal truth about the research-to-production gap: a model that scores 95% in a notebook and a model that works reliably in production are two completely different engineering artifacts. The gap is not ML knowledge — it is software engineering, data engineering, and operational discipline. The notebook model runs once on clean data. The production model runs ten thousand times a day on data you did not curate, from users you did not anticipate, in a feature distribution that drifts every week without anyone noticing until something breaks badly.

MLOps Maturity Levels (Google’s Model)¶

Google’s MLOps maturity framework is the clearest map of where teams actually sit versus where they think they sit.

Level	Name	What It Means	Deployment Method	Monitoring
0	Manual Process	Notebooks. One-off scripts. Someone runs the training job by hand.	Manual copy-paste of model files.	None — “it seemed fine last week.”
1	ML Pipeline Automation	Automated training pipeline. Feature engineering is code. Experiments are tracked.	Still manual: a human reviews and pushes the model.	Basic metrics dashboards.
2	CI/CD for ML	Automated training + automated model validation + automated deployment triggered by data or code changes.	Automated: pipeline tests the new model, compares it to the champion model, deploys if it passes.	Production monitoring for data drift, model degradation, and alerting.

Target by end of Phase 5: Level 2 on at least one project.

Most companies are at Level 0 in disguise — they have CI/CD for their application code but Level 0 for their ML models. Getting to Level 1 in one project during this phase is realistic and demonstrable. Level 2 is the capstone target and directly maps to a senior ML engineer role.

The MLOps Stack (End-to-End Data Flow)¶

Every arrow in this diagram is a place where things fail silently in production. The feedback loop back to data versioning is the one most teams never close — they retrain on the same stale data and wonder why the model does not improve.

What This Phase Covers¶

Topic	File	Core Tool(s)
Experiment tracking & data versioning	`01_experiment_tracking_and_versioning.md`	MLflow, W&B, DVC
Model serving and inference optimization	`02_model_serving_and_inference.md`	FastAPI, TorchServe, vLLM, TGI
Containerization and orchestration	`03_containerization_and_orchestration.md`	Docker, Kubernetes, KServe
Monitoring and data quality	`04_monitoring_and_data_quality.md`	Evidently, NannyML, Arize
CI/CD for ML	`05_ci_cd_for_ml.md`	GitHub Actions, Great Expectations
Phase projects (capstone + 2 supporting)	`06_phase_projects.md`	Full stack

Exit Criteria for Month 12¶

These are binary gates. You either have evidence or you do not. “I understand how it works” is not an exit criterion.

Has a running MLflow or W&B experiment tracking setup for at least one model project — with at least 5 experiments logged, metrics visible, and model artifact saved
Has served a model as a REST API — FastAPI minimum for classical models; vLLM for any LLM serving work
Has DVC set up on at least one project — data tracked, remote configured, dvc repro executes the pipeline cleanly
Has written at least one GitHub Actions workflow that: lints code, runs tests, evaluates the model against a performance threshold, and blocks merge if the threshold fails
Has implemented at least one monitoring check — data drift detected and reported using Evidently on at least one feature
Hard Gate #2 Project: Full MLOps Pipeline project complete (see 06_phase_projects.md) — public GitHub repo with green CI history

What Most People Get Wrong¶

“MLOps is not DevOps for ML — it has fundamentally different challenges.”

Data versioning (DVC) is harder than code versioning (Git) because data is large, binary, and its meaning changes over time even when the bytes do not. A dataset from six months ago may have the same schema but a completely different feature distribution — Git has no mechanism to track that.

Model behavior is non-deterministic. Two training runs on identical data and hyperparameters produce models with slightly different weights. This is normal, but it means “the model works” is a distribution, not a fact — you need statistical tests to know whether a new model is actually better than the champion model or just within noise.

Monitoring ML systems requires tracking statistical distributions, not just uptime. A web service goes down: Prometheus fires an alert in 30 seconds. An ML model starts returning systematically biased predictions because the input feature distribution shifted: nobody notices for three months. The people who treat ML deployment as “just Flask + Docker” are the ones whose models silently degrade in production until a business metric collapses and someone finally asks what changed.

Time Budget (10–15 hrs/week, Months 11–12)¶

Week	Focus	Hours
1	Experiment tracking setup (W&B), DVC data versioning basics	12
2	Model serving (FastAPI + vLLM), Docker + containerization	14
3	Kubernetes basics, monitoring setup (Evidently), CI/CD pipeline	13
4–5	Hard Gate Project: Full MLOps Pipeline	30
6–7	vLLM serving project + monitoring project	25
8	Review, documentation, portfolio write-up	10

Return to [../05_phase_4_generative_ai_frontier/README.md] · Next: [01_experiment_tracking_and_versioning.md]