Cloud-Native Java on Kubernetes — What Actually Ships

Kubernetes has won. Every large MNC runs some flavour of it (EKS, GKE, AKS, OpenShift, on-prem vanilla). This file is not a Kubernetes tutorial — there are 10,000 of those. This file is the Java-specific view: what the JVM needs to behave well in a container, what probes actually mean, how to structure a Helm chart, and where GraalVM native-image earns its keep vs where CDS is the pragmatic middle ground.

You already covered container-aware JVM flags and multi-stage Docker in Phase 07 file 07. This file layers Kubernetes-shaped concerns on top of that foundation.

The Container-Aware JVM — Recap

Java 17+ respects cgroup limits by default. The knobs you still care about in K8s:

env:
  - name: JAVA_TOOL_OPTIONS
    value: >-
      -XX:MaxRAMPercentage=75.0
      -XX:InitialRAMPercentage=50.0
      -XX:+ExitOnOutOfMemoryError
      -XX:+HeapDumpOnOutOfMemoryError
      -XX:HeapDumpPath=/tmp/heap.hprof
      -XX:+UseZGC                        # or G1 for smaller heaps (< 8GB)
      -XshowSettings:vm

Rules:

• Set resources.limits.memory in the Pod spec. MaxRAMPercentage=75 leaves ~25% for the JVM’s non-heap needs (metaspace, thread stacks, direct buffers).

• Set resources.requests.memory == resources.limits.memory for Guaranteed QoS. Anything less and your pod can be OOM-killed under node pressure.

• CPU limits are subtler: hard CPU limits cause throttling that manifests as latency spikes. Set CPU requests, avoid CPU limits on latency-sensitive Java services. This is contested advice — read the Kubernetes CPU throttling debate before defending either position in an study.

⚠️ What most people get wrong: they set resources.limits.cpu=1000m on a Spring service, watch p99 latency spike randomly, and blame the JVM. It’s CFS quota throttling. Remove the CPU limit or size it generously.

The Three Probes — Say This Precisely

Kubernetes exposes three probes; most Java devs use only one. Use all three, and know why.

Probe	What it means	What K8s does on failure	Wire it to
startupProbe	“App is still starting; don’t run other probes yet”	Delays liveness/readiness checks	/actuator/health/liveness with generous failureThreshold
livenessProbe	“App is alive; restart if not”	Kills the pod, K8s restarts it	/actuator/health/liveness (cheap check)
readinessProbe	“App is ready to serve traffic”	Removes pod from Service endpoints; NO restart	/actuator/health/readiness

startupProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8080
  failureThreshold: 30       # 30 * 5s = 2.5 min max startup
  periodSeconds: 5

livenessProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8080
  periodSeconds: 10
  timeoutSeconds: 3
  failureThreshold: 3

readinessProbe:
  httpGet:
    path: /actuator/health/readiness
    port: 8080
  periodSeconds: 5
  timeoutSeconds: 3
  failureThreshold: 2

Spring Boot 3.x built-in support — in application.yml:

management:
  endpoint:
    health:
      probes:
        enabled: true
      group:
        liveness:
          include: livenessState
        readiness:
          include: readinessState, db, redis

The liveness probe should NOT check downstream dependencies. If Redis is down and liveness fails, K8s restarts your pod; the restart won’t fix Redis. Failing readiness (temporarily remove from traffic) is the right response.

Graceful Shutdown — The 15-Second Dance

When K8s deletes a pod, this sequence runs:

1. Pod status → Terminating.

2. Pod is removed from Service endpoints (eventually, subject to kube-proxy sync delay).

3. preStop hook runs (if configured).

4. SIGTERM sent to PID 1.

5. terminationGracePeriodSeconds (default 30s) countdown starts.

6. If still running when the timer expires: SIGKILL.

Between 1 and 2, your pod may still receive traffic. This is why you need:

spec:
  terminationGracePeriodSeconds: 60
  containers:
    - name: shortly
      lifecycle:
        preStop:
          exec:
            command: ["sh", "-c", "sleep 15"]   # wait for endpoints to propagate

And in your application.yml:

server:
  shutdown: graceful
spring:
  lifecycle:
    timeout-per-shutdown-phase: 30s

That combination gives you: 15s of “still in endpoints but sleeping” + 30s to drain active connections + 15s slack before SIGKILL. Real production services need this to avoid dropped requests during rolling deploys.

Horizontal Pod Autoscaler — CPU is a Lie

HPA on CPU is the default because it’s easy. For a JVM service under GC, CPU is often not the right signal.

Signal	Good for	Notes
CPU	Simple, always available	GC and JIT confuse the reading; conservative target (60%)
Memory	Rarely useful	JVMs “use” all their heap; memory rises to MaxRAMPercentage and stays
Custom metric (RPS, queue depth, p95 latency)	The right answer for latency-sensitive services	Requires Prometheus Adapter or KEDA
KEDA scaler on Kafka lag	Event-driven consumers	The most honest signal for consumers

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: shortly
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: shortly
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Pods
      pods:
        metric:
          name: http_server_requests_per_second
        target:
          type: AverageValue
          averageValue: "200"

Warmup penalty: every new pod is slow for the first ~30 seconds (JIT + JVM ramp). Scale up ahead of demand, not reactively. behavior.scaleUp.stabilizationWindowSeconds: 0 and scaleDown.stabilizationWindowSeconds: 300 avoids flapping.

Sidecar Patterns — What’s Actually Useful

Sidecars are containers that share the pod network / lifecycle. Common Java use cases:

Sidecar	Purpose	Notes
Envoy / Istio proxy	Service mesh, mTLS, retries, load balancing	Overhead is real; pay the cost only when you need mesh features
Cloud SQL Proxy / RDS Proxy	Managed DB connection	Runs alongside your app, exposes localhost:5432
Filebeat / Fluent Bit	Log shipping	Only if stdout isn’t already collected — usually it is
OpenTelemetry Collector	Local OTLP receiver / batcher	Recommended — sidecar or DaemonSet
cloudflared / ngrok	Tunnel for dev / demo	Not production

Rule: if the platform provides it as a DaemonSet or a managed service, don’t run a sidecar for it. Your pod-to-container ratio matters for cost.

Helm — A Sane Chart Structure

Helm is Kubernetes’ package manager. A minimum-viable chart for a Spring service:

charts/shortly/
├── Chart.yaml
├── values.yaml               # default values
├── values-staging.yaml       # overrides per env
├── values-prod.yaml
└── templates/
    ├── deployment.yaml
    ├── service.yaml
    ├── configmap.yaml        # non-secret config
    ├── secret.yaml           # sealed / external-secrets reference
    ├── ingress.yaml
    ├── hpa.yaml
    ├── servicemonitor.yaml   # Prometheus Operator scrape config
    └── _helpers.tpl

Rules of thumb:

• Never store real secrets in values.yaml. Use External Secrets Operator, Sealed Secrets, or reference K8s Secrets already provisioned out-of-band.

• Keep environment differences to values-{env}.yaml. The templates should not if eq .Values.env "prod".

• Pin your chart version and your image tag. latest is not a version.

• helm template in CI to lint. helm diff upgrade before every apply.

GitOps with ArgoCD — The Deployment Discipline

ArgoCD watches a Git repo of Kubernetes manifests / Helm charts and reconciles the cluster to match. The workflow becomes:

1. CI builds the image, pushes to a registry, tags it (usually the git SHA).

2. CI opens a PR against a gitops repo bumping the image tag in values-{env}.yaml.

3. Merge → ArgoCD detects the change → applies to the cluster → syncs.

4. Rollback = revert the PR.

Why this matters for studies: “how do you deploy?” gets a real answer. “We use GitOps with ArgoCD; every deployment is a merged PR with a diff and a review. Rollbacks are reverts. No one runs kubectl apply from a laptop.” That single sentence is worth 30 minutes of “we use Jenkins” hand-waving.

GraalVM Native vs CDS — The Startup Trade-Off

Recapping from Phase 07 file 07 with a K8s-specific lens:

Option	Startup	Peak throughput	Image size	K8s fit
Plain JVM	4–10s	Baseline (JIT-optimized)	~200MB	Default — fine for long-running pods
JVM + Application CDS (Spring Boot 3.3+)	2–3s	Baseline	~200MB	Recommended default — zero code changes
GraalVM Native Image	40–150ms	10–25% slower than JIT peak	80–120MB	Serverless (Knative, Cloud Run, Fargate), tight scale-to-zero SLA

When native-image is the right choice:

• Scale-to-zero workloads where cold starts hit users.

• Very short-lived jobs (Kubernetes Jobs that run for < 30s).

• Memory-constrained edge / IoT.

When it isn’t:

• Long-running services (pods live for hours; startup cost amortizes).

• Heavy reflection / dynamic proxies / bytecode generation (native metadata pain).

• Peak throughput matters more than cold start (most REST services).

Default to CDS. Reach for native when you have a specific startup-cost problem.

Resource Sizing — A Starting Point

Exact numbers vary. As a starting point for a typical Spring Boot service:

Load profile	Requests/limits
Low traffic (< 50 RPS)	cpu: 250m / -, memory: 512Mi / 512Mi
Medium (50–500 RPS)	cpu: 500m / -, memory: 1Gi / 1Gi
High (500+ RPS)	cpu: 1000m / -, memory: 2Gi / 2Gi, replicas: 5+
ML inference in-process	Add 1–4Gi for the model, +1 CPU per concurrent inference

Measure before you tune. Grafana + kubectl top beat guessing.

Namespace, RBAC, Network Policy — The Boring Necessities

• Namespace per environment (dev, staging, prod) and often per team in shared clusters.

• RBAC: the service account your pod uses should have zero cluster-wide permissions unless it needs them. Most Spring services need nothing — they don’t call the Kubernetes API.

• NetworkPolicy: default-deny in production. Whitelist ingress from the gateway + egress to Postgres/Redis/Kafka/S3. Two years from now, when someone deploys a compromised sidecar, this is the difference between an incident and a breach.

• PodSecurity admission (baseline or restricted): no host namespaces, no privileged containers, runAsNonRoot: true, readOnlyRootFilesystem: true, drop all capabilities.

Sins Checklist

• readinessProbe checks downstream dependencies (should be liveness’s job — wait, no, neither: readiness can, liveness must not)

• No preStop sleep → dropped requests on every rolling deploy

• terminationGracePeriodSeconds shorter than the app’s shutdown time

• CPU limits on latency-sensitive services (throttling)

• latest image tag

• Secrets in values.yaml in Git

• No NetworkPolicy in production

• kubectl apply from developer laptops instead of GitOps

• Running an Envoy sidecar with no service-mesh features enabled — pure overhead

Practice

1. Deploy shortly (Phase 07) to a local kind cluster. Wire all three probes correctly. Confirm rolling deploy drops zero requests under k6 load.

2. Write a minimal Helm chart. Override values for a dev and a prod namespace.

3. Install ArgoCD in the cluster. Point it at a Git repo. Deploy by merging a PR.

4. Run the ONNX classifier (file 05) on the same cluster. Wire HPA to a Prometheus custom metric (RPS). Load-test until it scales, then release load and confirm scale-down.

---

Return to README.md · Previous: 05_serving_ml_models_from_java.md · Next: 07_case_studies_and_war_stories.md