06 — Compiler and Optimization¶

Your compiler is a very smart co-author. Most of the code it generates from your C++ source is not the C++ you wrote — it is a heavily-rearranged, inlined, vectorized approximation of your intent. This file teaches you how to talk to that co-author: which flags matter, when link-time and profile-guided optimization pay for themselves, when auto-vectorization works and when it doesn’t, and how to read the assembly so you can tell the difference between “the compiler did what I wanted” and “the compiler gave up.”

The optimization levels, plainly¶

Flag	What it does	When to use
`-O0`	No optimization, fast compile, faithful debugging	Development only. Never ship.
`-O1`	Basic optimizations, mostly free wins	Rare. Middle ground nobody uses.
`-O2`	Full optimization without code-size explosion	Default for release builds. What almost everyone ships.
`-O3`	`-O2` + aggressive inlining, vectorization, loop transforms	Numeric/tight-loop code. Sometimes slower than `-O2` due to code bloat + i-cache pressure. Benchmark.
`-Os`	Optimize for size	Embedded, or when the binary is huge and i-cache thrashing dominates.
`-Oz` (Clang)	More aggressive size opt	Mobile / embedded only.
`-Og`	Optimizations friendly to debugging	Great middle ground for CI’s debug tests.

The trap. Do not assume -O3 is faster than -O2. On real codebases the gap is often <2% and occasionally negative (larger code → more i-cache misses). Benchmark both on your workload. In P6.1 you will find that -O2 is often the winner despite conventional wisdom. Report actual numbers.

Always pair with:

-g — DWARF debug info. Free at runtime; makes crash dumps and profilers useful. Ship this in production.
-fno-omit-frame-pointer — keep the frame pointer register available. Costs ~1% CPU. Buys you working stack traces in perf and Instruments. Worth it 99% of the time.
-march=native — target the exact CPU you’re building on. Enables AVX2/AVX-512/NEON. Great for personal boxes; risky for release binaries that will run on unknown hardware. For P6.1 on your VM, use -march=native and record which uarch you tested.

LTO (Link-Time Optimization)¶

Normally the compiler optimizes one translation unit at a time. LTO defers optimization to link time, letting the compiler inline across .cpp files, propagate constants across libraries, and dead-code-strip aggressively.

Two flavors:

Full LTO (-flto) — one giant IR blob at link time. Slowest link; strongest optimization. 5-15% speedup on real services in independent measurements.
ThinLTO (-flto=thin, Clang / recent GCC) — parallel per-module LTO with cross-module summaries. Nearly the perf of full LTO (~3% behind on average, gap closing) at a small fraction of the link time.

In 2026, default to ThinLTO. Firefox has reported 5%+ from LTO alone. Clang’s own self-compile with -flto=thin shows measurable improvement. Full LTO is only worth it if you can absorb 5+ minute link times.

Enabling ThinLTO in CMake:

include(CheckIPOSupported)
check_ipo_supported(RESULT ipo_ok)
if(ipo_ok)
  set_property(TARGET miniserve PROPERTY INTERPROCEDURAL_OPTIMIZATION TRUE)
endif()

Or raw: CXXFLAGS="-O2 -flto=thin" LDFLAGS="-flto=thin".

Gotchas: your linker must match (lld for Clang, gold/mold for GCC). Static libraries built without -flto will not participate. Rebuild everything.

PGO (Profile-Guided Optimization)¶

The compiler’s biggest handicap is that it does not know which branch is hot. PGO fixes this by having you run the binary under representative load, collect a profile, then recompile with that profile as a hint.

The two-step build:

# Step 1: instrumented build
clang++ -O2 -fprofile-generate=./pgo-data -o miniserve.inst *.cpp
./miniserve.inst --load-test    # run under representative workload
llvm-profdata merge -o merged.profdata ./pgo-data/*.profraw

# Step 2: optimized build using the profile
clang++ -O2 -fprofile-use=merged.profdata -o miniserve *.cpp

Real-world numbers: Clang self-compile with PGO + ThinLTO + BOLT + jemalloc = ~1.75x speedup over -O2. Firefox, Chrome, and Postgres all ship with PGO. The catch: you need a representative profile. If your PGO run only hits the healthy path, the compiler mispredicts your error path and you can regress.

Worth it for P6.1? Yes, as a Phase 6 exercise. Not because you desperately need the 5-15% for a homework service, but because PGO is a resume line most C++ candidates cannot honestly write.

BOLT (llvm-bolt) is a post-link binary optimizer built by Meta. It reorders basic blocks and functions based on runtime profile. Stacks on top of PGO. Skip in Phase 6; know the name.

Inlining, `[[likely]]`, `[[unlikely]]`¶

The compiler decides what to inline based on heuristics — function size, call site count, constexpr-ness. Overrides:

inline keyword — a hint, mostly ignored by modern compilers. Do not sprinkle.
[[gnu::always_inline]] / __forceinline — actually forces inlining. Use sparingly on tiny hot helpers.
[[gnu::noinline]] — force NOT to inline. Useful for cold paths (error handling) to keep the hot path’s i-cache clean.
[[likely]] / [[unlikely]] (C++20) — hint branch predictor and code layout. Real wins on error-checking hot paths.

if (auto err = validate(req); [[unlikely]] err != OK) {
  return handle_error(err);
}

Use [[unlikely]] for the failure branch of every validation. Costs nothing to write; keeps the success path linearly laid out.

Auto-vectorization and when it gives up¶

Modern compilers vectorize inner loops when they can prove:

Iteration count is knowable or bounded.
No aliasing between input/output pointers.
No exceptions, no calls to non-vectorizable functions.
No data dependencies across iterations.

When it fails silently, your hand-written loop runs at 1/8 the throughput it should. Diagnose with:

clang++ -O2 -Rpass-missed=loop-vectorize -Rpass-analysis=loop-vectorize file.cpp

The compiler will tell you why it did not vectorize. Common answers: possible aliasing (fix with __restrict__), unknown trip count (add #pragma clang loop vectorize(assume_safety)), function call in loop (hoist or inline it).

#pragma omp simd forces vectorization when you know it is safe and the compiler is being conservative. #pragma GCC ivdep (GCC) or #pragma clang loop vectorize(enable) are the alternatives.

For SIMD you cannot get from auto-vectorization, drop to intrinsics: <immintrin.h> on x86, <arm_neon.h> on ARM. This is Phase 5 territory (revisit those exercises with your Phase 6 eyes) — do not write intrinsics unless a profile said to.

Aliasing and `restrict`¶

C++ has no restrict keyword, but every major compiler supports __restrict__ as an extension. It tells the compiler “these pointers do not alias” and unlocks vectorization + fewer memory reloads.

void axpy(float a, const float* __restrict__ x, float* __restrict__ y, int n) {
  for (int i = 0; i < n; ++i) y[i] += a * x[i];
}

Use on numeric kernels. Do not use if you cannot uphold the promise — undefined behavior awaits.

Reading assembly (godbolt.org)¶

Compiler Explorer is the single most important tool for growing your compiler intuition. Paste code, pick a compiler and flags, see the generated assembly side by side. Do this reflex-fast:

When you write a hot function, godbolt it at -O2 and read the output.
Look for: is it inlined? Is the loop vectorized (look for xmm/ymm/zmm on x86, v0-v31 on ARM)? Are there unexpected memory loads?
Compare -O2 vs -O3 output. Note the size cost.
Compare Clang vs GCC. Sometimes they generate very different code.

You are not learning to write assembly. You are learning to verify the compiler did what you asked. This one skill puts you above 90% of C++ candidates.

Sanitizers (which are compile flags, so this file)¶

Not optimization, but same category — the compiler adds instrumentation. Non-negotiable in your CI:

-fsanitize=address (ASan) — buffer overruns, use-after-free. Runtime overhead 2x.
-fsanitize=thread (TSan) — data races. Slower (5-15x), essential for anything with std::thread.
-fsanitize=undefined (UBSan) — signed overflow, misaligned reads, nullptr deref. Cheap (1.2x). Run in every CI.
-fsanitize=memory (MSan, Clang only) — uninitialized reads. Requires libc++ built with MSan; painful setup.

Run ASan + UBSan in your CI on every commit for P6.1. Ship without them enabled. The moment a real user hits a UAF, ASan-in-CI is what saved you.

The Phase 6 compilation checklist for P6.1¶

At W43 you should be able to check every box:

Release build uses -O2 -g -fno-omit-frame-pointer -flto=thin.
CI runs one job with -O2 -fsanitize=address,undefined.
At least one benchmark comparing -O2 vs -O3 documented in the README with numbers.
LTO on/off benchmark in the README.
PGO attempted at least once, result (better/worse/neutral) documented.
For your two hottest functions, godbolt links or screenshots pinned in the README.
-march=native documented — what uarch, what SIMD width the binary needs.

What most people get wrong¶

“-O3 is best.” No. -O2 is often equal or better on real services. Measure. This is the single most common myth.
Enabling LTO but linking against non-LTO static libs. No cross-module optimization happens; you paid link time for nothing.
Running PGO with a synthetic profile. The profile has to reflect real traffic patterns, including error rates. Otherwise you optimize the healthy path and pessimize the sad path.
Never reading assembly. You cannot debug why the compiler did not vectorize if you cannot read what it did emit.
Shipping -fsanitize= in production. Sanitizers are for CI, not prod. Their overhead is unacceptable for user-facing services.