Rung 8 — The M13 Capstone: ML-Powered Microservice on Kubernetes 🚨 HARD GATE¶

Target month: Shipped by end of M13 = end of July 2027 Calendar deadline: July 31, 2027 — THE PLAN ENDS HERE Phase alignment: Phase 08 — Distributed Systems & ML/AI Integration Signal level: The M13 pitch made concrete → this is the artifact you send to the recruiter with the message “here’s what I’ve been building.”

🚨 This is the second of the two hard gates in the ladder. If Rung 8 is not shipped by end of M13, the entire 13-month plan has not delivered its promised outcome. Every other rung is a stepping stone to this one. See 09_hard_gates.md for the full argument.

What It Is¶

A production-grade Java microservice that serves an ML model in real time, deployed on Kubernetes, integrating what you know about ML (your day job at Zoho) with what you’ve spent 13 months rebuilding (Java, JVM, Spring, distributed systems). This is the rung where your ML background stops being a footnote and becomes a differentiator.

The concrete artifact¶

A Java 21 + Spring Boot 3 microservice that:

Serves predictions from an ONNX-format model via ONNX Runtime for Java OR from a TensorFlow SavedModel via TensorFlow Java
Exposes POST /predict accepting a JSON payload, returning predictions + latency + model version
Exposes GET /model/info returning the loaded model’s metadata (version, input schema, load timestamp)
Uses virtual threads for request handling (finally cashing in Rung 5’s virtual-thread investment)
Has an in-process feature preprocessing pipeline in Java (not shelled out to Python) — this is the non-obvious hard part
Publishes structured Micrometer metrics: request rate, prediction latency by percentile, model reload count
Supports hot model reload without service restart (watch a mounted config-map / S3 path for new model version)
Ships as a Docker image, deployed to Kubernetes (Minikube is acceptable for the demo, but the manifests must be portable to a real cluster)

The Kubernetes deployment¶

Kubernetes manifests in k8s/: Deployment, Service, ConfigMap, HorizontalPodAutoscaler, PodDisruptionBudget
Helm chart OR raw manifests — raw manifests are actually preferred because they show you know K8s primitives, not just Helm
HPA scales on CPU + custom metric (predictions/sec via Prometheus adapter, if you’re going for it)
Deployed to either Minikube (local, cheap, sufficient for demo) or a real cluster on GKE / EKS / DigitalOcean Kubernetes (paid, stronger signal, budget-dependent)
Ingress via nginx-ingress or the cluster’s default ingress; TLS via cert-manager + Let’s Encrypt
Live URL you can curl from your laptop

The model¶

You are an applied ML engineer. This is your home turf, so make it interesting but not too interesting:

A pre-trained model you already understand (e.g., sentence-transformers all-MiniLM-L6-v2 for text embeddings, or a small BERT for classification, or a scikit-learn tabular model exported to ONNX)
The model does one clearly-articulated thing: “classify support tickets by category” or “score URLs for likelihood-of-abuse” or “embed queries and return the nearest cached response”
Model file is under 200 MB (bigger models complicate the K8s image / storage story unnecessarily)
Predictions must be sub-100ms p95 on a 2-CPU pod. This is your target.

Bias toward NLP or tabular models, away from vision. Vision inference in Java has more preprocessing pain (image decoding, resizing, normalization) that isn’t the signal you’re trying to send.

Where To Publish¶

Repo: github.com/RaghulR2003/ml-serving-java — public, pinned as the top slot on your GitHub profile
Live URL: e.g., ml.raghulr2003.dev/predict with a working demo request
Blog: Hashnode 3-part series (the capstone deserves a series, not a single post):
- Part 1: “Why Java for ML serving? A Zoho engineer’s argument.”
- Part 2: “Loading an ONNX model in Java 21 — the preprocessing you can’t skip.”
- Part 3: “Kubernetes, virtual threads, and 100 QPS on 2 CPUs.”
LinkedIn: Three long-form posts, one per blog part. Space them a week apart. This is your M13 announcement campaign.
Reddit: r/java and r/MachineLearning both get one post. r/MachineLearning skews toward Python, so your “why Java” angle is genuinely novel content there.
Portfolio landing: Rung 8 is the hero of the landing page. Everything else is supporting evidence.
Optional but strong: Submit a lightning talk proposal to JavaOne or a local Java user group (Bangalore JUG, Chennai JUG) for late 2027. Even submitting has resume value.

Signal It Sends¶

You bridge two worlds. Java engineers who understand ML models are rare. ML engineers who ship real Java services are rarer. This rung says you are both.
You know Kubernetes. Not just “I’ve used Docker.” Actual K8s primitives — Deployment, Service, HPA, ConfigMap. Every senior Indian tech role for 2027 lists K8s.
You measure at the right level. Sub-100ms p95 on 2 CPUs at 50 RPS is a specific, defensible claim. Any study partner will engage with the numbers, and you’ll have the evidence.
You wire everything together. Rate limiter (Rung 5), Spring Boot production patterns (Rung 7), JVM performance instinct (Rung 6), modern Java idioms (Rungs 3, 4) — all show up in Rung 8. The portfolio has a shape, not a list.
You have a story. “Applied ML engineer at Zoho who spent 13 months rebuilding Java fluency and shipped this.” That is a hireable story anywhere in the Indian tech ecosystem.

Common Failure Modes¶

Python-in-a-subprocess. Shelling out to Python for preprocessing kills the whole “Java for ML serving” argument. Do the preprocessing in Java. If ONNX doesn’t cover it, use DJL or write it by hand.
Toy model. Serving “add two numbers” is not ML. Serving iris classification is barely ML. Ship something that could plausibly be called production ML — embeddings, real classification, real ranking.
Kubernetes as decoration. A single-pod Deployment with no HPA, no resource limits, no probes — that isn’t K8s, that’s Docker with extra YAML. Do it properly or drop it and ship on plain Fly.io.
Skipping the blog series. The three-part blog series is 30% of the signal. Rung 8 without the series is a strong repo. Rung 8 with the series is a career artifact.
Overselling the model. Do not claim accuracy numbers you didn’t measure. Do not compare to state-of-the-art. This isn’t a research paper. It’s a serving artifact. Honesty about the model’s limits in MODEL_CARD.md is itself a senior-signal.
Free-tier expiry between shipping and studying. If Rung 8 ships in July 2027 and you’re studying in September 2027, your “live URL” better still resolve. Budget $20/month for the last 3 months to keep the demo alive during study season.
Missing the deadline. July 31, 2027 is the plan’s end. If Rung 8 slips into August, you lose the narrative of the 13-month arc. Ship even a rough version on time; polish afterward.

Time Estimate¶

Model selection + local inference proof-of-concept: ~6 hours
Java feature preprocessing pipeline: ~10 hours
Spring Boot service + REST endpoints: ~8 hours
ONNX Runtime / TF-Java integration + benchmarking: ~8 hours
Hot model reload plumbing: ~6 hours
Virtual thread executor configuration + tests: ~4 hours
Metrics + observability + logging: ~5 hours
Dockerfile + multi-arch image + registry push: ~4 hours
Kubernetes manifests + local minikube run: ~8 hours
HPA + probes + resource tuning: ~5 hours
Deployment to real cluster + DNS + TLS: ~8 hours
k6 load tests + tuning to hit p95 goal: ~10 hours
Testcontainers integration tests: ~6 hours
README + ARCHITECTURE.md + MODEL_CARD.md + diagrams: ~8 hours
Blog series (3 posts): ~30 hours
Video walkthrough: ~6 hours
LinkedIn campaign, Reddit, cross-posting: ~5 hours
Total: ~137 hours over 8-10 weeks

Rung 8 is the largest single rung. It is the entire M12 and M13. Nothing else meaningful happens in M12-M13 except polishing earlier rungs.

Prerequisites¶

Rungs 1-7 all shipped or credibly close (Rung 7 is the most direct prerequisite — you carry over Docker, Postgres knowledge, Spring patterns)
All files in 08_distributed_systems_applied_integration/ read and worked through
Kubernetes literacy: you understand what a Pod, Deployment, Service, ConfigMap, HPA are without needing to look them up
Docker fluency: you’ve built and shipped multi-stage images in Rung 7
ML model in mind before M12 starts — do not shop for a model in M13
Access to a Kubernetes cluster: minikube for local, or a paid $30-50/month cluster for a real deployment
Domain (raghulr2003.dev or similar) already registered for TLS-friendly deployment

Stretch Goals (Optional, Only If You Ship Before July 15, 2027)¶

A/B model serving. Serve two model versions behind a weighted split, log which version served each request. This is what real ML serving looks like at scale and is directly study-relevant.
A GRPC endpoint alongside REST. GRPC is standard for internal ML serving. Adding one shows you know the difference and when to use each.
A vector-DB integration for retrieval-augmented use. If your model is an embedder, hook it up to pgvector or Qdrant. Now you have an actual RAG-style pipeline in Java. This is unusually valuable in July 2027’s market.
A conference talk submission. Bangalore JUG, Chennai JUG, Devoxx India, or a Zoho internal tech talk. The act of submitting is a portfolio item.

Why This Rung Is A Hard Gate¶

Rung 8 is not a hard gate because it’s the hardest — it’s a hard gate because without it, the M13 pitch is empty. Rungs 1-7 are individually valuable, but they read as “I studied Java for a year.” Rung 8 is what makes the story read as “I am a Java engineer who ships ML services.” That distinction is the entire point of Directory 12.

If Rung 8 slips to August 2027, you’ve missed the narrative window. If it slips further, you’ve turned the plan into “nice preparation, no capstone,” which is what 90% of long study plans deliver. Do not let that happen.

See 09_hard_gates.md for the full argument on what to cut if you’re behind.

Return to README.md · Next: 09_hard_gates.md

Rung 8 — The M13 Capstone: ML-Powered Microservice on Kubernetes 🚨 HARD GATE¶

What It Is¶

The concrete artifact¶

The Kubernetes deployment¶

The model¶

Where To Publish¶

Acceptance Criteria¶

Functional¶

Quality¶

Kubernetes¶

Documentation¶

Story¶