05 — CI and Docker

A CI pipeline is a proof-of-work that your code actually builds on machines that aren’t yours. Docker is the mechanism that makes the CI machine reproducible — same OS, same compiler, same package versions, forever. GitHub Actions is the default — free for public repos, unlimited minutes on Linux/Mac for open source, and it’s what your Phase 4 template will ship with. Bazel is worth one page of your attention because Snowflake, Databricks, Google, and NVIDIA use it internally, but you are unlikely to reach for it in your target ML-platform role.

This file gives you: (1) a matrix workflow that covers Linux + macOS + two compilers + Debug/Release/Sanitized in reasonable job-time, (2) caching strategy for Conan and vcpkg so you don’t rebuild deps every run, (3) a real Dockerfile that reproduces the Ubuntu 24.04 side of CI locally, and (4) enough Bazel to read a BUILD file in an study without panic.

1. GitHub Actions matrix workflow

Save as .github/workflows/ci.yml. Explanations after the block.

name: ci

on:
  push:
    branches: [main]
  pull_request:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  build-test:
    name: ${{ matrix.os }} / ${{ matrix.compiler }} / ${{ matrix.preset }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-24.04, macos-14]
        compiler: [clang-18, gcc-13]
        preset: [debug, release, asan]
        exclude:
          # macOS doesn't use gcc in practice; save the minutes.
          - os: macos-14
            compiler: gcc-13

    runs-on: ${{ matrix.os }}

    steps:
      - uses: actions/checkout@v4

      - name: Set up CMake
        uses: lukka/get-cmake@latest
        with:
          cmakeVersion: '3.28.6'
          ninjaVersion: 'latest'

      - name: Install Linux toolchain
        if: matrix.os == 'ubuntu-24.04'
        run: |
          if [[ "${{ matrix.compiler }}" == "clang-18" ]]; then
            wget -qO- https://apt.llvm.org/llvm.sh | sudo bash /dev/stdin 18
            echo "CC=clang-18"     >> $GITHUB_ENV
            echo "CXX=clang++-18"  >> $GITHUB_ENV
          else
            sudo apt-get update && sudo apt-get install -y gcc-13 g++-13
            echo "CC=gcc-13"       >> $GITHUB_ENV
            echo "CXX=g++-13"      >> $GITHUB_ENV
          fi

      - name: Install macOS toolchain
        if: matrix.os == 'macos-14'
        run: |
          # macOS runners ship apple-clang; treat the 'clang-18' matrix cell as apple-clang.
          echo "CC=clang"     >> $GITHUB_ENV
          echo "CXX=clang++"  >> $GITHUB_ENV

      # ---------- Dependency manager: vcpkg ----------
      - name: Restore vcpkg cache
        uses: actions/cache@v4
        with:
          path: |
            ~/.cache/vcpkg
            vcpkg_installed
          key: vcpkg-${{ matrix.os }}-${{ matrix.compiler }}-${{ hashFiles('vcpkg.json') }}
          restore-keys: |
            vcpkg-${{ matrix.os }}-${{ matrix.compiler }}-

      - name: Set up vcpkg
        uses: lukka/run-vcpkg@v11
        with:
          vcpkgGitCommitId: 'a42af01b72c28a8e1d7b48107b33e4f286a55ef6'  # pin to a known-good SHA

      # ---------- Configure / build / test ----------
      - name: Configure
        env:
          VCPKG_ROOT: ${{ github.workspace }}/vcpkg
        run: cmake --preset ${{ matrix.preset }}

      - name: Build
        run: cmake --build --preset ${{ matrix.preset }} -j

      - name: Test
        env:
          ASAN_OPTIONS: detect_leaks=1:abort_on_error=1:print_stacktrace=1
          UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1
        run: ctest --preset ${{ matrix.preset }} --output-on-failure

      - name: Upload test log on failure
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: testlog-${{ matrix.os }}-${{ matrix.compiler }}-${{ matrix.preset }}
          path: build/${{ matrix.preset }}/Testing/Temporary/LastTest.log

Why this shape:

• fail-fast: false — you want to see every failing cell of the matrix, not the first one only. Otherwise you fix one, push, learn there’s another, fix, push. Slow feedback loop.

• concurrency block — cancels stale runs when you push a new commit to the same PR. Saves CI minutes.

• exclude rules — macOS + gcc is nominally supported but no one ships that way. Skip.

• Preset-driven — all the compiler flag details live in CMakePresets.json. The workflow just picks a preset name. This is the payoff of Phase 4’s investment in presets.

• Cache keyed on hashFiles('vcpkg.json') — same manifest = same cache. Change a dep = cache miss = deps rebuild once, then cache hit forever.

Expected job time on a warm cache: 2–4 minutes per cell. On a cold cache: 8–15 minutes. Total matrix ~5 cells × 3 min = ~15 minutes wall-clock (they run in parallel).

2. Caching strategy — do this right or CI becomes glacial

vcpkg

Cache ~/.cache/vcpkg (binary cache) plus the project’s vcpkg_installed/. Key on the OS, compiler, and hash of vcpkg.json. When any of those change, cache misses and rebuilds. When none change, ~5 seconds to restore.

Conan

- uses: actions/cache@v4
  with:
    path: ~/.conan2
    key: conan-${{ matrix.os }}-${{ matrix.compiler }}-${{ hashFiles('conanfile.txt', 'conanfile.py') }}

Conan’s cache is settings-hashed already, so a Debug and Release build coexist in one cache directory without collision.

ccache/sccache (compiler cache)

One more layer of speed — caches compiled object files by hash of preprocessed input + compiler + flags. Enable in CMake:

find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
    set(CMAKE_C_COMPILER_LAUNCHER   "${CCACHE_PROGRAM}")
    set(CMAKE_CXX_COMPILER_LAUNCHER "${CCACHE_PROGRAM}")
endif()

And cache ~/.ccache in the GHA workflow. Second and later CI runs on the same PR become dramatically faster (10–30% typical, 60%+ on incremental changes).

3. Dockerfile — reproducible local + CI environment

Save as Dockerfile at project root. Ubuntu 24.04 (LTS through April 2029), LLVM 18, CMake 3.28+, Conan 2, and vcpkg pre-bootstrapped.

# syntax=docker/dockerfile:1.7
FROM ubuntu:24.04

ENV DEBIAN_FRONTEND=noninteractive \
    LANG=C.UTF-8 \
    CMAKE_GENERATOR=Ninja

# 1. Base packages
RUN apt-get update && apt-get install -y --no-install-recommends \
        build-essential \
        curl ca-certificates git gnupg lsb-release wget \
        python3 python3-pip python3-venv pipx \
        ninja-build cmake pkg-config \
        zip unzip tar \
    && rm -rf /var/lib/apt/lists/*

# 2. LLVM 18 (clang, clang-tidy, clang-format, llvm-symbolizer)
RUN wget -qO- https://apt.llvm.org/llvm.sh | bash /dev/stdin 18 all && \
    ln -sf /usr/bin/clang-18       /usr/local/bin/clang && \
    ln -sf /usr/bin/clang++-18     /usr/local/bin/clang++ && \
    ln -sf /usr/bin/clang-tidy-18  /usr/local/bin/clang-tidy && \
    ln -sf /usr/bin/clang-format-18 /usr/local/bin/clang-format

# 3. GCC 13 for the matrix
RUN apt-get update && apt-get install -y --no-install-recommends gcc-13 g++-13 && \
    rm -rf /var/lib/apt/lists/*

# 4. Conan 2 via pipx (isolated, no site-packages pollution)
RUN pipx install --global conan==2.7.0 && \
    conan profile detect --force

# 5. vcpkg (bootstrap, then git-clean)
RUN git clone --depth=1 https://github.com/microsoft/vcpkg.git /opt/vcpkg && \
    /opt/vcpkg/bootstrap-vcpkg.sh -disableMetrics
ENV VCPKG_ROOT=/opt/vcpkg
ENV PATH="/opt/vcpkg:${PATH}"

# 6. Non-root user for interactive dev
ARG USER=dev
ARG UID=1000
RUN useradd -m -u ${UID} -s /bin/bash ${USER}
USER ${USER}
WORKDIR /workspace

CMD ["/bin/bash"]

Build + run:

docker build -t mynn-dev .
docker run --rm -it -v "$PWD:/workspace" mynn-dev
# inside:
cmake --preset release && cmake --build --preset release && ctest --preset release

This image is ~2–3 GB. That’s fine for CI and dev. If you want a slim runtime image (deployment), do a multi-stage build — stage 1 is this dev image, stage 2 is debian:12-slim with only your final binary and its runtime shared libraries.

Multi-stage runtime image sketch

FROM mynn-dev AS build
COPY --chown=dev:dev . /workspace
WORKDIR /workspace
RUN cmake --preset release && \
    cmake --build --preset release && \
    cmake --install build/release --prefix /opt/mynn

FROM debian:12-slim AS runtime
RUN apt-get update && apt-get install -y --no-install-recommends libstdc++6 && \
    rm -rf /var/lib/apt/lists/*
COPY --from=build /opt/mynn /opt/mynn
ENV PATH="/opt/mynn/bin:${PATH}"
ENTRYPOINT ["/opt/mynn/bin/mynn_server"]

Result: ~80 MB runtime image, no build tools, no source, small attack surface. This is the pattern you’ll use when P5.3 (ONNX inference server) ships.

4. Bazel — the one-page primer

Bazel (Google, 2015, open-source Blaze) is a hermetic, hash-based, dependency-graph-first build system. It is the default at Google, Databricks, Snowflake, Adobe internals, and a growing minority of other large-monorepo shops. You are unlikely to introduce it. You may be dropped into a codebase that already uses it. Know how to read it.

Concept model

• WORKSPACE (older) or MODULE.bazel (newer, bzlmod, default from Bazel 7): top-level file declaring external dependencies.

• BUILD / BUILD.bazel in each source directory: declares targets in that directory.

• Rules: cc_library, cc_binary, cc_test, py_binary, pybind_extension, etc. Rules are typed functions with named args.

• Labels: //mylib:mylib = target mylib in package //mylib. @fmt//:fmt = target fmt in external repo fmt.

• Hermeticity: Bazel controls the toolchain. No apt install implicit deps. Builds are reproducible bit-for-bit given the same inputs.

A BUILD.bazel you can read

cc_library(
    name = "mynn",
    srcs = ["src/layer.cpp", "src/train.cpp"],
    hdrs = glob(["include/mynn/*.hpp"]),
    includes = ["include"],
    deps = [
        "@eigen//:eigen",
        "@fmt//:fmt",
    ],
    copts = ["-std=c++20", "-Wall", "-Wextra"],
    visibility = ["//visibility:public"],
)

cc_binary(
    name = "train_mnist",
    srcs = ["apps/train_mnist.cpp"],
    deps = [":mynn"],
)

cc_test(
    name = "mynn_tests",
    srcs = glob(["tests/test_*.cpp"]),
    deps = [
        ":mynn",
        "@googletest//:gtest_main",
    ],
)

Build it

bazel build //:mynn
bazel build //:train_mnist
bazel test //:mynn_tests

When to reach for Bazel

• Multi-language monorepo (C++ + Java + Python + Go all linked at build-time). CMake breaks down; Bazel excels.

• Remote build execution / remote caching — first-class in Bazel, painful in CMake.

• Millions of files, thousands of engineers — the Google-scale case.

When not to

• Solo or small-team C++ project. CMake + Conan/vcpkg is faster to bootstrap, has broader library support, and won’t require you to write custom rules for anything off the beaten path.

• studies: know Bazel exists, know the target model (which is similar to modern CMake targets, actually), know it’s hash-based and hermetic. You will not be asked to write a BUILD file.

Ecosystem you should know exists

• EngFlow, BuildBuddy, Aspect Build — commercial vendors making Bazel accessible outside Google. Remote caches, dashboards, migration tooling.

• rules_cc, rules_python, rules_go — official language rules.

• buck2 (Meta, Rust) — a Bazel-alike from Meta. Similar model. Growing.

5. What most people get wrong

• They add runs-on: ubuntu-latest and call it done. No macOS, no compiler variance. Every Mac-specific bug reaches main.

• They don’t cache dependencies. 20-minute cold builds every push. Wire the caches.

• fail-fast: true (default) hides half the failures. Explicitly set false.

• They ship Docker images with the whole build toolchain in the runtime layer. 3 GB images that are 95% clang and cmake. Use multi-stage builds.

• They pin ubuntu-latest (moving target). ubuntu-24.04 is pinned; upgrade deliberately when 26.04 lands.

• They install packages fresh in every CI run (no cache, no Docker layer). Use actions/cache on apt too, or move to a Docker container action.

• They vendor vcpkg/ or .conan2/ in Git. Both are large and machine-specific. .gitignore them.

• They forget the concurrency block. Every commit runs the whole matrix even if superseded seconds later. Wastes CI minutes.

6. Practice exercises

1. Take your P4.1 template and add the workflow above. Get the full matrix green.

2. Add ccache. Measure the second CI run vs the first. Report the ratio.

3. Write the multi-stage Dockerfile. Get the runtime image under 150 MB.

4. Read one real BUILD.bazel file from open source (e.g., tensorflow/tensorflow/core/BUILD or envoyproxy/envoy/source/common/common/BUILD). Explain what one cc_library target is doing.

---

Nav: ← 04 Sanitizers and static analysis · Next: projects →