File I/O and File Descriptors

A file descriptor is a small non-negative integer. That’s it. It’s a per-process index into the kernel’s open-file table. 0 is stdin, 1 is stdout, 2 is stderr, and after that the kernel gives you the lowest unused integer every time you open something. Getting this data structure into your bones — and understanding that “file” here means socket, pipe, terminal, device, and actual file all uniformly — is what makes Unix Unix.

You’ll spend this file learning three things: (1) the raw syscall interface (open/read/write/close) versus the buffered stdio wrappers (fopen/fread/fwrite/fclose), (2) descriptor manipulation (dup2, O_CLOEXEC, fcntl), and (3) modern zero-copy paths (mmap, sendfile, splice) that are non-negotiable for anything performance-sensitive in 2026.

Raw syscalls vs stdio — when to reach for which

Concern	Raw (open/read/write)	Stdio (fopen/fread/fprintf)
Buffering	None — every call is a syscall	Line- or block-buffered in user space
Portability	POSIX	ISO C
Sockets/pipes	Yes	Painful; buffering fights you
Format strings	You build them	printf family for free
Errno reporting	Direct	ferror(FILE*) + errno
Overhead per small write	~700-1500 ns per call	~10 ns until buffer flush

Rule of thumb:

• Anything that touches a socket, pipe, or device: raw syscalls. Buffering hides bytes at bad moments.

• Anything that’s a text file with formatted output (logs, config files, small tools): stdio is fine and much nicer to write.

• Anything performance-critical on large files: raw syscalls with large buffers (64KB-1MB), or mmap, or sendfile.

The trap: mixing them on the same file descriptor. Once you’ve called fdopen(fd, "r") to wrap a raw fd in a FILE*, do not touch the raw fd again. Stdio has its own buffer; you’ll see stale bytes or lose bytes. If you must switch, fflush first.

The open() flags you actually need to know

int fd = open(path, O_RDONLY);
int fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0644);
int fd = open(path, O_WRONLY | O_CREAT | O_APPEND, 0644);
int fd = open(path, O_RDWR | O_CREAT | O_EXCL, 0644);       // fail if exists
int fd = open(path, O_RDONLY | O_NONBLOCK);                  // no blocking
int fd = open(path, O_RDONLY | O_CLOEXEC);                   // close on exec

Learn cold: O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC, O_APPEND, O_EXCL, O_NONBLOCK, O_CLOEXEC, O_DIRECT, O_SYNC.

O_CLOEXEC is not optional in modern code. Without it, every fd you open will leak into any child process you exec. In a server this is a security hole (child inherits your database connection) and a resource leak. Set it. Every time. Or use openat with O_CLOEXEC explicitly.

Atomic append with O_APPEND

When multiple processes write to the same log file, they interleave lines and corruption ensues — unless every writer opens it with O_APPEND. With O_APPEND the kernel guarantees that the seek-to-end + write happens atomically as a single operation for writes up to PIPE_BUF (usually 4096) bytes. This is the reason nginx, syslog, and every well-behaved logger uses O_APPEND.

Not with O_APPEND: each writer must flock the file or serialize through a single logging process. Painful and slow. Just use O_APPEND.

dup2 and fd redirection

dup2(oldfd, newfd) makes newfd refer to the same open file as oldfd, closing newfd first if needed. This is how shells do >, <, 2>&1. The typical pattern in a child before exec:

int fd = open("out.log", O_WRONLY | O_CREAT | O_TRUNC, 0644);
dup2(fd, STDOUT_FILENO);       // stdout now points at out.log
dup2(fd, STDERR_FILENO);       // stderr too
close(fd);                     // original fd no longer needed
execvp(argv[0], argv);

You will write this exact pattern in your mini-shell project. Memorize the shape.

mmap — files as memory

mmap asks the kernel to map a region of a file (or anonymous memory) into your address space. After that, reading ptr[i] is a memory access that the page-fault handler transparently turns into disk I/O on first touch.

int fd = open("bigfile", O_RDONLY);
struct stat st; fstat(fd, &st);
void *p = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
// use p as if it were a plain byte array of size st.st_size
munmap(p, st.st_size);
close(fd);

When mmap wins:

• Random access to large files (databases, model weights, mmap’d tensors).

• Sharing read-only data across many processes (kernel keeps one copy in the page cache).

• Skipping the user-space buffer copy that read mandates.

When mmap loses:

• Sequential streaming over huge data — read with a 1MB buffer is often equal or faster because the page-fault path has overhead too.

• When you need to handle I/O errors gracefully — mmap turns disk errors into SIGBUS, which is much harder to recover from than a read returning -1.

• On files that may be truncated by another process — you’ll segfault mid-access.

Anchor for the ML engineer: llama.cpp mmaps model weight files. This is why loading a 30GB model is essentially instant — the OS lazy-loads pages as inference touches them. Combined with MAP_POPULATE (Linux-only) or madvise(MADV_WILLNEED), you get controlled prefetch.

Zero-copy: sendfile and splice

Traditional file → socket copy involves four buffers: disk → kernel page cache → user buffer → kernel socket buffer → NIC. That’s 2 copies through user space that do nothing useful.

sendfile(out_fd, in_fd, offset, count) moves bytes directly from one kernel fd to another. nginx serving a static file uses this. On modern Linux the copy can even be DMA’d straight to the NIC (with SO_ZEROCOPY + kernel-tls, though the setup is fiddly).

splice(in_fd, ..., out_fd, ..., len, flags) is the more general primitive — moves bytes between fds using a kernel pipe as the internal buffer. Works between any two fds where at least one is a pipe.

When you’ll reach for these: proxying, file servers, log shipping. In an ML inference server, less often — payloads are usually small enough that the copy cost is dwarfed by compute. But for a model artifact server or a large-batch response, they matter.

fcntl — the swiss army knife

fcntl(fd, ...) does about a dozen things. The ones you’ll use:

• fcntl(fd, F_GETFL) | O_NONBLOCK; fcntl(fd, F_SETFL, ...) — make an fd non-blocking after the fact.

• fcntl(fd, F_SETFD, FD_CLOEXEC) — set close-on-exec after open (prefer O_CLOEXEC at open time).

• fcntl(fd, F_DUPFD_CLOEXEC, 3) — like dup but atomic with cloexec, avoiding a race.

• File locks with F_SETLK / F_SETLKW — usable but tricky; prefer flock for simple cases.

What most people get wrong about this

They think close(fd) cannot fail and ignore its return value. It can fail — on NFS, on distributed filesystems, on buggy USB drives — with EIO, meaning some buffered write did not actually make it to storage. Most of the time you don’t care; but if you’re writing a database or a critical log, checking close’s return value (and calling fsync before it) is the difference between “we lost the customer’s data” and “we didn’t.”

Practice this week

1. Write a cat clone in ~30 lines using raw read/write. Then a stdio version. Time both on a 1GB file.

2. Write a cp clone using: (a) read/write, (b) mmap, (c) sendfile. Benchmark all three. Notice which wins in which regime.

3. Verify O_APPEND atomicity: fork 10 children, all writing 1024 bytes at a time to the same log file. With O_APPEND: no torn lines. Without: torn lines.

4. Run ls -l /proc/self/fd/ inside a child that inherited a parent’s fd. Confirm it’s there. Add O_CLOEXEC. Confirm it’s gone.

References

• man 2 open, man 2 read, man 2 write, man 2 mmap, man 2 sendfile, man 2 splice.

• APUE ch. 3 (File I/O), ch. 5 (Standard I/O), ch. 14 (Advanced I/O).

• TLPI ch. 4-5, ch. 45 (mmap), ch. 63 (sendfile/splice).

• Linus’s classic rant on why O_DIRECT is broken — read it once for perspective, then ignore O_DIRECT until you’re writing a database.

---

Return to README.md · Next: 03_process_control.md