The Linux Specifics: procfs, cgroups, namespaces, io_uring¶

POSIX gives you portability across every Unix; Linux gives you the interfaces the entire modern cloud is actually built on. Kubernetes, Docker, systemd, eBPF, cgroups, io_uring — all Linux-specific, all things you must understand at some level if you’re going to be paid to write systems C in 2026. This file is a tour, not a deep dive. Each of these could be a book; you get the mental model and the pointers to go deeper when you need to.

The applied-ML relevance: every model server you’ll deploy runs in a container, which is a Linux namespace + cgroup setup. Every GPU/CPU limit you hit is a cgroup limit. Every “why is my container OOMKilled?” is a memory.max limit in /sys/fs/cgroup/. This isn’t infrastructure trivia — it’s operational literacy.

`procfs` — the kernel as a filesystem¶

/proc is a virtual filesystem where the kernel exposes process and system state as text files. Read it, understand it, use it in scripts and monitoring code.

Essential paths:

Path	What’s there
`/proc/<pid>/status`	Human-readable per-process state (VmRSS, State, Uid, Threads, …)
`/proc/<pid>/maps`	Memory map of the process — every region, its perms, backing file
`/proc/<pid>/fd/`	Symlinks to every open file descriptor — amazing for debugging leaks
`/proc/<pid>/cmdline`	Argv, null-separated
`/proc/<pid>/environ`	Environment, null-separated
`/proc/<pid>/limits`	Rlimits
`/proc/<pid>/stack`	Kernel stack trace (needs CONFIG_STACKTRACE)
`/proc/self/*`	Same as above but for the current process — handy in your own code
`/proc/meminfo`	System memory breakdown
`/proc/cpuinfo`	Per-CPU details
`/proc/loadavg`	1/5/15-min load averages
`/proc/sys/*`	Kernel tunables (also accessible via `sysctl`)

Habit: ls /proc/self/fd/ inside your own program during development. If the count is growing, you have an fd leak. If a fd you expected to close is still there, you know where to look.

`sysfs` — devices and drivers¶

/sys exposes the kernel device model — devices, drivers, buses, classes. You’ll interact with it far less than /proc. When you do, it’s usually:

/sys/class/net/<iface>/ — network interface stats and settings.
/sys/block/<dev>/queue/scheduler — I/O scheduler.
/sys/devices/system/cpu/cpu*/cpufreq/ — CPU frequency governor (a real perf lever for ML workloads: performance vs powersave).
/sys/fs/cgroup/ — the cgroup v2 tree.

cgroups (v2) — the resource-limit system¶

cgroups (control groups) let the kernel account for and limit resource usage of a group of processes: CPU time, memory, I/O, PIDs. cgroup v2 replaces the older v1 layout and is what modern Linux (systemd, Docker, Kubernetes) uses by default in 2026.

Structure: a unified hierarchy rooted at /sys/fs/cgroup/. Each directory is a cgroup, each subdirectory is a child. Control files are named <controller>.<setting>:

/sys/fs/cgroup/
├── cgroup.procs                # PIDs in this group
├── cpu.max                     # "quota period" — e.g., "50000 100000" = 0.5 CPU
├── memory.max                  # hard memory limit
├── memory.current              # current usage
├── io.max                      # per-device I/O limits
└── <sub-cgroups>/

When your Kubernetes pod says resources.limits.memory: 2Gi, kubelet writes 2147483648 to memory.max in the pod’s cgroup. When the process exceeds it, the kernel’s OOM killer fires and kills the process — that’s your OOMKilled status.

Practical debugging you’ll do: cat /sys/fs/cgroup/<your-cgroup>/memory.events shows how many times you hit the limit and were throttled or OOM’d. This is the source of truth; the container runtime just relays it.

Namespaces — containers under the hood¶

A namespace is a kernel-enforced view of a global resource. Linux has (as of 2026): pid, mount, network, uts (hostname), ipc, user, cgroup, time. Each process is in exactly one of each. Two processes in the same PID namespace see each other; two processes in different PID namespaces don’t.

A container is nothing more than a process launched in fresh namespaces + a cgroup + a root filesystem (chroot or pivot_root). Docker’s magic is 5% clever engineering and 95% careful orchestration of these primitives.

The syscalls:

clone(CLONE_NEWPID | CLONE_NEWNET | ...) — create a new process in fresh namespaces.
unshare(CLONE_NEWNS | ...) — detach the current process from a namespace into a new one.
setns(fd, ...) — join an existing namespace (given by an fd from /proc/<pid>/ns/).

Fun exercise for later: implement a 100-line “container” in C — unshare a mount namespace, chroot into a directory, drop caps, exec bash. You’ll never fear Docker again.

`io_uring` — the state of the art async I/O interface (with caveats)¶

io_uring is Linux’s modern asynchronous I/O interface (kernel 5.1+, 2019). Two shared ring buffers between kernel and user space:

SQ (submission queue): user pushes I/O requests here.
CQ (completion queue): kernel pushes results here.

One syscall (io_uring_enter) can submit and reap many operations. With IORING_SETUP_SQPOLL, a kernel thread polls the SQ and no syscall is needed at all for submission — you get sub-10ns per operation amortized.

The performance case (2023-2025 benchmarks)¶

20-40% p99 latency improvement vs epoll for high-connection-count servers when SQPOLL is enabled (kernel-internals.org, Alibaba, Cloudflare blog posts).
5M+ IOPS achievable on NVMe with a single io_uring-based process (Jens Axboe’s fio benchmarks).
At 100K concurrent connections, epoll spends 15-25% CPU in syscall overhead; io_uring’s batching drops that to single digits.
Zero-copy send gives 10-15% throughput gain for messages >4KB.
PostgreSQL, RocksDB, ScyllaDB, ceph are all adding io_uring paths.

The security case (2023-2025 reality — the plot twist)¶

Here’s the counter-story your bootcamp instructor probably didn’t tell you.

Google’s Security Blog (2023): 60% of the Linux kernel exploits submitted to their VRP in 2022 were io_uring bugs. Google paid roughly $1M in io_uring bug bounties in a single year.
Google disabled io_uring on their production servers.
ChromeOS disabled io_uring entirely.
Android disabled io_uring for apps via the seccomp-bpf filter.
Docker removed io_uring from its default seccomp profile — you need to override it to use io_uring in a container.
GKE Autopilot restricts io_uring by default.
Continued stream of EoP (Elevation of Privilege) CVEs through 2024-2025 (see Android Security Bulletins Feb/Mar/Dec 2025).

The 2026 verdict for you¶

Learn it. Understand it. Do not build your first serious network server on it. Reasons:

In many production environments where you’d actually deploy (Docker, K8s Autopilot, hardened corporate VMs), it’s disabled or restricted. Your beautiful io_uring server won’t run.
The security surface is genuinely wider than epoll’s. If you’re not sure your ops team wants you shipping code that uses it, ask first.
The performance win is workload-dependent. Streaming a single big connection? Epoll may actually win (see GitHub axboe/liburing issue #536: epoll 1565K qps vs io_uring 506K qps at 64B buffer for that pattern). Many small connections with mixed I/O? io_uring wins big. Benchmark your actual workload.
Real-world writeups (“The Speed Engineer” on Medium, Oct 2025: 6-month Go/epoll → Rust/io_uring rewrite — gains much smaller than expected, operational complexity high) suggest you should default to epoll and reach for io_uring after profiling.

Build the Phase 5 project on epoll. Do an io_uring port as a stretch goal. That’s the honest 2026 answer.

eBPF — the observability revolution¶

eBPF lets you attach small programs to kernel hooks (syscall entry, network packet arrival, tracepoints) that run in a sandboxed VM inside the kernel. This is how modern Linux observability, networking, and security tooling works: bpftrace, bcc, Cilium, Falco, all of Netflix and Facebook’s kernel monitoring.

For a beginner in 2026:

Start with bpftrace (bpftrace.org). It’s a high-level tracing language; one-liners like bpftrace -e 'tracepoint:syscalls:sys_enter_open { printf("%s\n", str(args->filename)); }' will change how you debug.
Brendan Gregg’s ebpf page (brendangregg.com/ebpf.html) is still the canonical hub.
For production tools, the ecosystem has moved from bcc (Python-driven, heavy) to libbpf + CO-RE (“Compile Once, Run Everywhere”) — portable, small, production-friendly.
Reddit r/eBPF sentiment (2024): “Most guides are outdated; they will point you to BCC or bpftrace which are ok for one-offs but the ecosystem moved on to libbpf/CO-RE for production tools.”

You will not write eBPF programs in this phase. You will use bpftrace one-liners and read execsnoop/opensnoop/biolatency/tcpconnect output to understand what your programs actually do at the kernel level.

What most people get wrong about this¶

They read one benchmark post and conclude io_uring is strictly better than epoll. It’s not, especially in 2026. It’s newer, more powerful, and legitimately faster in a subset of workloads — but it’s also disabled in a growing subset of production environments for security reasons. The honest engineering move is: learn epoll first (it’s not going anywhere), understand io_uring’s model, benchmark before choosing.

Practice this week¶

cat /proc/self/status from within one of your own programs. Print VmRSS before and after a big malloc+touch.
Find your process’s cgroup: cat /proc/self/cgroup. Then cat /sys/fs/cgroup/<that-path>/memory.current. Compare with VmRSS.
Install bpftrace and run execsnoop-bpfcc (from bpfcc-tools). Watch every execve on your system for 30 seconds. Notice how much noise there is.
Read man 7 namespaces and man 7 cgroups cover to cover. Boring. Do it anyway.

References¶

man 5 proc, man 7 cgroups, man 7 namespaces, man 2 io_uring_setup, man 2 io_uring_enter.
Jens Axboe, “Efficient IO with io_uring” — the original design PDF, still the best intro.
Brendan Gregg, Systems Performance, 2nd ed. — the reference for perf, eBPF, and Linux performance methodology.
Julia Evans, “How containers work” zine — the best 30-page tour of namespaces+cgroups.
kernel.org’s cgroup v2 documentation — dry but definitive.

Return to README.md · Next: 06_binary_and_linker.md