A Production Server Walkthrough

Every serious systems engineer has read the source of at least one real network server end-to-end. It’s the fastest way to bridge the gap between the toy epoll loop you can write yourself and the shape of code that runs at scale. For this walkthrough you’re going to spend about 45 minutes reading redis (or, equivalently, its community fork valkey) — not skimming, actually tracing one request lifecycle. When you finish you will have seen the concrete answer to every architectural question in files 04-07.

Why redis and not nginx? nginx is battle-hardened and huge (~150k LOC, dozens of modules, heavy macro use). Reading it cold is a two-week project. Redis’s core network layer is roughly 3-4k LOC across four files, none of them clever, all of them commented. Why not libuv? libuv is a library, not a server — it lacks the accept-loop and command-dispatch parts you specifically want to see. Why not sqlite? sqlite is a masterpiece but it’s an embedded library with no network layer at all. Redis is the pedagogical sweet spot: real production code, small enough to read in one sitting, structured the same way every high-performance network server is structured.

Redis or valkey?

In March 2024 Redis Inc. changed the Redis license from BSD-3-Clause to a dual SSPL/RSALv2 model, and the community immediately forked Redis 7.2.4 as valkey, donated to the Linux Foundation with maintainer and corporate support from AWS, Google, Oracle, Ericsson, and Snap. Valkey 7.2 was a drop-in wire-compatible replacement; Valkey 8.0 shipped September 2024 with per-slot multithreading gains; Valkey 8.1 in April 2025; Valkey 9.0 in 2026 with hash-field TTL and 2000-node cluster support. Redis 8 (with the newer AGPLv3 addition) also continues to develop.

For this walkthrough either works — the network layer is essentially identical. Valkey (github.com/valkey-io/valkey) is the community-active BSD-licensed lineage in 2026; Redis (github.com/redis/redis) is the original. Clone one:

$ git clone --depth=1 https://github.com/valkey-io/valkey.git
$ cd valkey/src
$ ls *.c | wc -l   # ~120 files

The network code we’re reading is a stable subset that has barely moved between forks; both directories look the same for our purposes.

The files, in the order you’ll open them

Order	File	~LOC	What’s here
1	src/server.c	~7000	main(), initServer(), serverCron(), top-level orchestration
2	src/ae.c	~600	The tiny cross-platform event loop abstraction
3	src/ae_epoll.c	~150	Linux epoll backend (also ae_kqueue.c, ae_select.c)
4	src/networking.c	~4000	Accept, per-client read/write, command dispatch
5	src/anet.c	~600	Thin, portable socket wrappers used everywhere

That’s it. Under 12k lines total, and only the first ~2k of networking.c matter for the request lifecycle. You can absolutely read this in a focused 45-minute session.

The reading protocol — trace one request

Here’s the trap most people fall into: they open server.c and start reading top-to-bottom. It’s 7000 lines of definitions, callbacks, cron jobs, and cluster/replication paths you don’t care about yet. You’ll get bored and quit. Do this instead — pick a single request lifecycle and follow it, top to bottom, one function at a time.

The lifecycle you’re tracing: a client connects, sends PING, gets back +PONG\r\n, disconnects. Every event-loop server does the same five things:

Step 1 — Bootstrap and event loop creation

Open server.c. Search for int main(. You’ll find it about 3/4 through the file. Skim past option parsing until you get to initServer(), called near the end of main. Read initServer() — particularly the calls to:

• aeCreateEventLoop(...) — allocates the event loop struct. Follow it briefly into ae.c.

• listenToPort(...) — opens the listen socket(s). Follow into anet.c’s anetTcpServer if you want to see the bind/listen wrapper.

• createSocketAcceptHandler(...) — registers the accept-side callback with the event loop.

• aeSetBeforeSleepProc(...) / aeSetAfterSleepProc(...) — hooks that run before and after epoll_wait.

Near the very end of main() you’ll find aeMain(server.el). That’s the whole program. Everything else is callbacks.

Step 2 — The event loop itself

Jump to ae.c. Read aeMain(). It’s roughly:

void aeMain(aeEventLoop *eventLoop) {
    eventLoop->stop = 0;
    while (!eventLoop->stop) {
        aeProcessEvents(eventLoop, AE_ALL_EVENTS |
                                   AE_CALL_BEFORE_SLEEP |
                                   AE_CALL_AFTER_SLEEP);
    }
}

That is it. All of redis is the body of that while loop. Now read aeProcessEvents() — the meaty ~150 lines. Notice the shape:

1. Compute next scheduled-timer expiry.

2. Call beforeSleep hook (this is where redis flushes the AOF, sends buffered replies, etc.).

3. Call aeApiPoll — the platform-specific epoll_wait/kqueue/select (see next step).

4. Call afterSleep hook.

5. For each returned event, dispatch to the fd’s readable or writable file-event callback.

6. Fire any expired timer events.

Exit-question moment. After you’ve read this function you should be able to answer: how does redis structure its event loop main function? Answer: two-hook “before/after sleep” reactor over a platform-abstracted poll primitive.

Step 3 — The epoll backend

Open ae_epoll.c. This is under 200 lines. Read it top to bottom — it’s small enough. Note:

• aeApiCreate calls epoll_create1(1024) (the 1024 is the hint, not the max; the kernel ignores it since Linux 2.6.8).

• aeApiAddEvent / aeApiDelEvent wrap epoll_ctl with EPOLL_CTL_ADD or MOD depending on prior state.

• aeApiPoll calls epoll_wait and translates flags back into redis’s own AE_READABLE/AE_WRITABLE bits.

Notice redis uses level-triggered epoll — no EPOLLET. Why? Because redis reads the whole client input buffer in one go inside its readable callback, so the LT re-notification never actually happens. LT is simpler and equally fast under that discipline. This is a real design decision worth noting in your notes.

Now flip open ae_kqueue.c (for BSD/macOS) and ae_select.c. Same interface, different backends. Confirm: the event loop is portable because it’s parameterized on 3 small functions per platform. That’s how you build cross-platform C.

Exit-question moment. Where does redis choose between epoll/kqueue/select? Answer: at compile time, in config.h, via #ifdef HAVE_EPOLL / HAVE_KQUEUE; the chosen ae_XXX.c is #included directly by ae.c near the top.

Step 4 — Accept and the client lifecycle

Open networking.c. Search for acceptTcpHandler — the callback that runs when the listen fd becomes readable. Read it:

1. Loops calling anetTcpAccept (from anet.c, which wraps accept4 on Linux).

2. For each accepted fd, calls acceptCommonHandler → createClient(fd).

Read createClient (a few hundred lines up in the same file). Note:

• Allocates a client struct with input buffer, output buffer, current-command state, argc/argv, connection object.

• Sets TCP_NODELAY, SO_KEEPALIVE via anetKeepAlive.

• Registers the client fd for AE_READABLE with callback readQueryFromClient.

• Stashes the client* in the event loop’s file-event data so lookups are O(1).

Now read readQueryFromClient — this is where every request lands. Shape:

1. read(fd, ...) into client->querybuf.

2. If read returned 0 → peer closed → freeClient(c), done.

3. If EAGAIN → return; loop will re-fire on next readable edge.

4. Otherwise, call processInputBuffer(c) which parses the RESP protocol from querybuf, handles partial commands (leaves them in the buffer, returns to caller — the buffer persists across reads), and for every complete command calls processCommand(c).

5. processCommand does the dispatch to pingCommand, getCommand, etc.

6. Replies are not written directly. They’re appended to client->buf / client->reply and the fd is flagged so beforeSleep will register it for AE_WRITABLE and eventually write.

Exit-question moment. How does redis handle partial reads? Answer: querybuf is a persistent per-client buffer; processInputBuffer parses as many complete commands as it can and leaves any leftover fragment. On the next read it’s appended to and parsing resumes. There is no “one read = one command” assumption anywhere.

Step 5 — Client state and graceful shutdown

Where is client-alive state kept? Answer: in the client struct declared in server.h, one per connection, held in server.clients (a linked list). The event-loop file-event array holds a pointer to it via aeCreateFileEvent(..., clientData=c). The lifetime is create-on-accept, free-on-freeClient (called on error or peer close), with a deferred-free queue for cases where a callback can’t safely delete the struct it’s inside.

Graceful shutdown. Search for prepareForShutdown in server.c. It:

1. Sets server.shutdown_asap in a signal handler (SIGTERM/SIGINT).

2. The event loop notices at the top of aeProcessEvents via a check on eventLoop->stop.

3. Before exiting: flushes AOF, closes clients gracefully (writes any pending replies), unlinks the PID file, calls exit(0).

Exit-question moment. How does redis exit gracefully? Answer: a signal handler flips a flag, the event loop tests the flag on its next iteration, and prepareForShutdown runs a documented cleanup sequence before exit(0). There is no signal-in-the-middle-of-work drama because the signal handler does nothing but set a flag — async-signal-safety by construction.

The five exit questions — answer these before you leave

Write short answers in your notes. If any question feels shaky, re-read the relevant section.

1. How does redis structure its event loop main function? — aeMain is a while (!stop) aeProcessEvents(...) loop; aeProcessEvents runs before-sleep hook → poll → after-sleep hook → dispatch events → fire timers.

2. Where does redis choose between epoll/kqueue/select? — ae.c uses #ifdef HAVE_EPOLL / HAVE_KQUEUE to #include one of ae_epoll.c / ae_kqueue.c / ae_select.c; the choice is at compile time and driven by autoconf/config.h.

3. How does redis handle partial reads? — A persistent per-client querybuf accumulates bytes; processInputBuffer parses zero or more complete RESP commands out of it and leaves any fragment for the next read.

4. Where is the “keep client alive” state kept? — In a client struct (one per connection) allocated in createClient and linked into server.clients. Pointer is stored in the event-loop file-event data so lookups are O(1).

5. How does redis exit gracefully? — SIGTERM/SIGINT handler sets a flag; the event loop checks it and runs prepareForShutdown, which flushes persistence, drains client output, cleans up, and exits. Signal handler is minimal and async-signal-safe.

Bonus reading targets (skip on first pass)

Once the request lifecycle is clear, these are worth an evening each:

• aof.c — Append-Only File persistence. How redis flushes writes to disk without blocking the event loop (background thread + write buffer + fsync policy).

• replication.c — how a replica handshake and stream flows through the same event loop.

• t_string.c, t_hash.c, etc. — how command implementations are structured. Each is small and independent.

• cluster.c — the cluster protocol. Ignore first pass; it’s a whole other book.

What most people get wrong about this

They read production code top-to-bottom instead of tracing one request lifecycle. server.c is 7000 lines; you cannot hold that in your head cold. But you can hold seven functions in your head cold, one from each file, forming the path a PING takes from wire to reply. Every production codebase is like this — a spine of hot-path code buried in a mountain of policy, replication, admin commands, and edge cases. Find the spine first; explore the mountain later.

Second common mistake: reading redis’s source while thinking about redis’s data structures. Ignore the data structures on this pass. You’re reading redis to understand event-loop networking, not dict.c or ziplist.c. Save those for when you’re reading it as a data-structures reference (a great use in a different month).

Practice this week

1. Clone valkey (or redis). find src -name '*.c' | xargs wc -l | sort -n and confirm the file sizes above.

2. Trace a PING from client wire to server reply, following the five files in order. Write the five exit-question answers in your own words — not copy-pasted from here — in a walkthrough_notes.md in your notes repo.

3. Set a breakpoint in gdb on readQueryFromClient — run redis-server, redis-cli PING from another shell, watch it hit. Step through one iteration. This makes the abstract concrete in a way pure reading doesn’t.

4. Draw the reactor + client-struct + querybuf on paper. If you can draw it, you understand it.

5. Compare ae_epoll.c (LT) to your own Rung 5 server (ET). Note the trade-off in a paragraph.

References

• valkey source — github.com/valkey-io/valkey — the community-active BSD-licensed fork, Linux Foundation home. Recommended for this walkthrough.

• redis source — github.com/redis/redis — the original, now under SSPL/RSALv2. Same network code as valkey for our purposes.

• antirez, “Redis internals: dict.c” — the seminal series of blog posts on his old site (archived). Great context but tangential to network code.

• Josiah Carlson, Redis in Action (Manning) — user-facing but includes helpful chapters on the design.

• valkey RFCs — github.com/valkey-io/valkey-rfc — the design discussions for the multithreading work in 8.x.

---

Return to README.md · Next: 09_debugging_concurrent_c.md