03 — JIT & Optimization

The HotSpot JIT is a small compiler that watches your program run, decides which methods are worth optimizing, and generates native code that is usually faster than what a C compiler would produce for the same source. That last part surprises people. The JIT wins because it optimizes with profile data — it knows which branches you actually took, which types you actually saw, which methods were actually virtual-vs-monomorphic — things an AOT compiler can only guess about.

The cost is that the optimizations are speculative. When the JIT’s guesses turn out wrong, it deoptimizes: throws the compiled code away and falls back to the interpreter. Understanding what makes the JIT succeed — and what makes it deoptimize in a loop — is what separates people who can measure JVM performance from people who can fix it.

1. Tiered Compilation, Concretely

Java 8+ runs tiered compilation by default. There are five internal levels, but the two you care about are:

• Tier 3 (C1 with full profiling) — kicks in after ~2,000 invocations. Compiles quickly, moderately optimized, keeps counting.

• Tier 4 (C2, fully optimized) — kicks in after ~10,000 invocations (or when back-edge counters cross a threshold, for hot loops). Slow to compile, aggressively optimized. This is the code your steady-state throughput depends on.

See what compiled with:

java -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining App

Output lines look like:

  1234  432       4       com.example.RateLimiter::tryAcquire (89 bytes)
  1250  432       4       com.example.RateLimiter::tryAcquire (89 bytes) made not entrant

The columns: timestamp (ms), compile-task-id, tier (3 or 4), method, size. made not entrant means: this compiled version has been discarded, calls will fall back to interpreter (or a lower tier) and eventually recompile. That’s a deopt.

2. The Big Optimizations (What C2 Actually Does)

C2’s optimization list is long. These are the four you must be able to name and defend, because they explain 80 % of “why is this Java code so fast?”:

Inlining

Small, hot methods get their bodies copied into their callers. This unlocks every other optimization — constant propagation, escape analysis, dead-code elimination — because the compiler can now see across the call boundary. Default inline size limit: 35 bytes for hot methods, 325 for maximum. Print with -XX:+PrintInlining.

Killer: megamorphic call sites. If a virtual/interface call has seen only one concrete type (monomorphic), the JIT inlines it. Two types = bimorphic, still inlines both with a guard. Three or more = megamorphic = falls back to a vtable/itable lookup, no inlining. This is why hot code paths often use final classes/methods, or sealed hierarchies, or LambdaMetafactory’s single-method callsites.

Escape Analysis + Scalar Replacement

If the JIT can prove an object never escapes its enclosing scope (never leaks to another thread, never gets stored in a field, never returned), it can:

1. Scalar-replace it — explode its fields into local variables, so no allocation happens at all.

2. Stack-allocate it — rare in HotSpot but the theoretical fallback.

This is why Optional.of(x).orElse(y) is typically as fast as an if. The Optional box doesn’t escape, EA proves it, C2 elides the allocation entirely. Turn EA off with -XX:-DoEscapeAnalysis and watch your benchmarks tank.

Loop Unrolling & Vectorization

Hot counted loops get unrolled (bodies duplicated) so pipeline stalls are reduced and SIMD instructions become applicable. Java has autovectorization for basic loops on int[], long[], double[]. The Vector API (JEP 448, still incubating as of Java 21) exposes explicit SIMD when you need more control.

Lock Elision / Coarsening

C2 can remove synchronized blocks whose monitor object is proven thread-local (elision) or merge adjacent synchronized blocks on the same monitor (coarsening). This is what makes StringBuffer — which is unnecessarily synchronized — not catastrophically slow. But note: with virtual threads and JEP 491’s synchronized fixes, these old optimizations are getting revisited.

3. Deoptimization (a.k.a. “Why Did My Steady-State Go Away?”)

Deopts happen when a speculative assumption breaks:

• Class hierarchy change: a previously monomorphic virtual call sees a new subclass. Recorded as class_check in JIT logs.

• Uncommon branch taken: C2 didn’t compile a branch it thought was unreachable. When you finally take it, it deopts. unstable_if.

• Null / range check failure: a check C2 elided based on profile data turns out to fail. null_check, range_check.

A one-off deopt is nothing. A deopt storm — the same method going in and out of C2 hundreds of times per second — is a real performance bug. Symptoms: CPU high, -XX:+PrintCompilation full of made not entrant lines. Cause: usually a hot method that alternates types (e.g., a List field sometimes ArrayList, sometimes LinkedList, sometimes Collections.emptyList()). Fix: make the type site monomorphic if possible, or accept the vtable cost and stop chasing inlining.

Print the reasons with:

java -XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation App
# Produces hotspot_pid<PID>.log — gigantic XML, use JITWatch to read it.

JITWatch (open-source, github.com/AdoptOpenJDK/jitwatch) is the canonical tool for reading compile logs. Learn it in an afternoon; it will pay for itself the first time you chase a deopt storm.

4. Warmup: Why Your Benchmark Lies Without It

Everything above means: the first N seconds of your program run different code than the next N minutes. A benchmark that measures the first 100 ms is measuring the interpreter and C1, not the code your production workload will actually run.

Rules of thumb:

• Give C2 at least 5–10 seconds of warmup at production-representative load before you trust timings.

• On startup-sensitive workloads (Lambda-style, serverless), warmup is a real cost — consider -XX:+UseSerialGC, -XX:TieredStopAtLevel=1 (skip C2), or GraalVM native-image.

• JMH (file 05) handles warmup for you, correctly. If you’re not using JMH for numbers you plan to publish, you are doing benchmarking wrong.

5. GraalVM and AOT: Trade-offs

Two alternative execution models are worth naming:

GraalVM JIT. Replaces C2 with a Java-implemented JIT. Sometimes faster than C2 on high-abstraction code (streams, lambdas, functional patterns). Same warmup profile as HotSpot. Enable with -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler on a GraalVM install.

native-image (AOT). Compiles Java ahead-of-time to a static native binary using the Graal compiler + Substrate VM. No JIT at runtime, no warmup, no Class.forName (mostly), no dynamic class loading. Trade-offs:

Aspect	HotSpot JIT	native-image
Startup	500 ms–5 s	10–100 ms
Peak throughput	Reference (100 %)	~70–95 % (depends on workload)
Memory (RSS)	200 MB–2 GB+	20–200 MB
Reflection / dynamic proxies	Free	Requires reachability metadata (reachability-metadata.json)
Build time	Seconds	Minutes

Spring Boot 3 has first-class native-image support via Spring Native / Spring AOT. Quarkus was built for it. Use it when startup latency and memory footprint dominate your SLA — serverless, edge compute, containers packed tight. Do not use it if you need runtime-loaded classes (JDBC drivers, byte-buddy proxies, dynamic scripting).

6. Working With the JIT: Concrete Practices

Five things you can do without ever reading a compile log:

1. Keep hot methods small and monomorphic. Split cold error-handling out of the hot path.

2. Prefer composition over deep inheritance in hot code. Fewer megamorphic sites.

3. Avoid mixing Optional, boxed primitives, and streams in inner loops unless you’ve measured. EA usually saves you, but not always — profile before assuming.

4. Warmup at boot. For latency-critical services, run a few thousand synthetic requests through hot paths before opening the pod to real traffic. Spring Boot 3 has hints for this via @ImportRuntimeHints.

5. Suspicious of magic “speed” tricks. String.intern, manual bit-tricks, sun.misc.Unsafe — the JIT already knows tricks you don’t. Measure with JMH before rewriting.

⚠️ What Most People Get Wrong

“Java is slow because it’s interpreted.” Java is compiled to native code by C2 within seconds of startup. On steady-state throughput, HotSpot regularly matches or beats naive C++ on the same algorithm because C2 has profile data C++ doesn’t. What Java is not good at, without extra work, is cold startup and low memory. That’s the trade-off native-image is trying to close.

“Making a method final makes it faster.” In 2005, yes — it helped the JIT devirtualize. Since roughly Java 7, the JIT does class hierarchy analysis (CHA) and can devirtualize any method that has no overriding subclass currently loaded (and deopt if one appears later). final still communicates intent to readers; it no longer meaningfully affects performance. Use it for design reasons, not speed.

Recap

• Tiered compilation: interpreter → C1 (tier 3) → C2 (tier 4). ~10k invocations to reach C2.

• Inlining, escape analysis, loop unrolling, lock elision are C2’s big wins.

• Deopts happen when speculation fails. Deopt storms are real bugs — -XX:+PrintCompilation + JITWatch to diagnose.

• Warmup is not optional. Non-JMH benchmarks are lying to you.

• GraalVM JIT and native-image trade peak throughput for startup/memory. Pick per workload.

• final for design intent, not performance. The JIT already devirtualizes.

---

Return to README.md · Previous: 02_memory_and_gc.md · Next: 04_diagnostic_tools.md