Phase 4 Projects

Two deliverables. Both are chosen because (a) they force you to use every syscall you learned in this phase, (b) they produce artifacts you can point at in studies, and (c) they’ll bite you with real bugs — not tutorial-clean happy paths. Ship both by end of M7.

Project 1: msh — a mini shell

Scope: ~<phone_number_or_numberic_id_or_random_id_11> LOC of C. Build a POSIX-ish shell that a human can actually use for an hour without hating.

Required features

  1. Read-eval-print loop with a prompt. Reads a line, tokenizes, executes, repeats.

  2. External command execution via fork + execvp, with waitpid to reap.

  3. Pipelines: ls -l | grep foo | wc -l — arbitrary depth, wired correctly with pipe/dup2.

  4. Redirection: >, >>, <, 2>, 2>&1.

  5. Background jobs: trailing &. Don’t waitpid synchronously; reap in a SIGCHLD handler.

  6. Signal handling:

    • Ctrl-C (SIGINT) sent to the foreground job’s process group, not to the shell itself.

    • Ctrl-Z (SIGTSTP) suspends the foreground job; fg/bg builtins resume it.

    • SIGCHLD handler reaps children and updates job status. Async-signal-safe only.

  7. Job control builtins: jobs, fg, bg, kill %N. Track jobs in a table.

  8. Cd, exit, export: because these must be built in (they change shell state).

  9. PATH search: rely on execvp, or roll your own via $PATH.

Stretch goals

  • Line editing (readline or your own with termios).

  • Command history.

  • Simple globbing (*.c).

  • Environment variable expansion ($HOME, ${VAR:-default}).

  • Command substitution $(cmd) — harder than it looks; you’ll fork a subshell.

Design guidance

  • File layout: msh.c (main + REPL), parser.c (tokenize + parse a command line into a struct), exec.c (execute a parsed command tree), jobs.c (job table + SIGCHLD handling), builtins.c.

  • Struct for a parsed command: { char **argv; int fd_in; int fd_out; int fd_err; bool background; struct command *next_in_pipe; }.

  • Process groups: after fork, in the child, setpgid(0, 0) to create a new pgrp for a foreground job; then tcsetpgrp(STDIN_FILENO, pgrp) in the parent to hand the terminal to it. This is the tricky part of Unix job control; APUE ch. 9 walks it step by step. Don’t skip it.

  • Reaping: your SIGCHLD handler loops waitpid(-1, &status, WNOHANG | WUNTRACED) until it returns 0. WUNTRACED lets you see stopped children, not just exited ones.

Testing

  • echo hi | tr a-z A-Z — basic pipe.

  • sleep 5 & then jobs — background execution and job listing.

  • cat > /tmp/out then Ctrl-C — signal to the fg job, shell survives.

  • sleep 100 then Ctrl-Z, then bg, then jobs, then kill %1.

  • ls /nonexistent 2> /tmp/err; cat /tmp/err — error redirection.

Anti-goals

  • Do not implement complete POSIX shell semantics. That’s a career. Get the 20% of the syntax that covers 80% of daily use.

  • Do not use system() internally. That’s cheating; you’re building the thing system() uses.

Exit criteria

  • 10 minutes of hands-on use without a crash.

  • valgrind ./msh on a scripted session shows no leaks and no invalid accesses.

  • The SIGCHLD handler is genuinely async-signal-safe (no printf, no malloc).

  • You can explain, on a whiteboard, every syscall your shell makes between prompt-in and prompt-out for a simple pipeline.

References:

  • Stephen Brennan, “Write a Shell in C” — the friendliest starting tutorial. Read it, close it, don’t copy from it.

  • APUE ch. 9 (Process Relationships) and ch. 18 (Terminal I/O) for job control.

  • bash’s source code (specifically execute_cmd.c) as a reference when yours does something surprising. Do not read it start-to-finish; it’s a monster.

Project 2: mtail — a tail -f clone using inotify

Scope: ~300 LOC. Watch a file for new content and print it as it arrives — the essential half of tail -f. Add multiple-file support if you have time.

Why this project

It looks trivial. It’s not. Doing it correctly forces you to handle:

  • File rotation (the log file gets renamed, a new one created — you must follow the correct file).

  • Truncation (file shrinks; you must reset your read offset).

  • Files that don’t exist yet.

  • Non-blocking reads on a regular file (O_NONBLOCK on a regular file doesn’t actually make read non-blocking; you must use select/poll on the inotify fd instead).

Required features

  1. Print the last N lines (default 10) on start (tail’s default).

  2. Then poll for new content and print it as it arrives (-f behavior).

  3. Use inotify to sleep efficiently — no polling loop with sleep(1).

  4. Handle rotation: when the file’s inode changes (rename+create) or the file is deleted+recreated, re-open the new file.

  5. Handle truncation: stat shows current size < your last read offset → reset and reread from start (or from EOF depending on semantics you choose).

  6. Multiple files: mtail file1 file2 ... — prefix each output line with the filename.

Design guidance

  • inotify basics: inotify_init1(IN_CLOEXEC | IN_NONBLOCK) gives you an fd. inotify_add_watch(ifd, path, IN_MODIFY | IN_MOVE_SELF | IN_DELETE_SELF) returns a watch descriptor. Read struct inotify_events from the fd.

  • Event loop: a simple while(1) { poll on inotify fd; drain events; on IN_MODIFY read the file's new data; on IN_MOVE_SELF/IN_DELETE_SELF close and re-open the file after a short delay. }.

  • Reading new data: track per-file offset. On each IN_MODIFY, seek to that offset, read until EOF, print, update offset.

  • Robustness: what if the file doesn’t exist when mtail starts? Watch the parent directory for IN_CREATE with matching name, then start tailing.

Stretch goals

  • -n <N> for last N lines.

  • -F semantics: keep retrying after delete (as opposed to -f which stops).

  • Coloured output per file (like multitail).

  • A stats mode: count lines per second per file.

Testing

  • Run mtail /tmp/log &. In another shell: for i in $(seq 100); do echo "line $i" >> /tmp/log; sleep 0.1; done. All 100 lines should appear.

  • mv /tmp/log /tmp/log.old; touch /tmp/log; echo hi >> /tmp/log — mtail should follow to the new file.

  • > /tmp/log; echo new >> /tmp/log — truncate case, mtail should notice and print new.

  • Compare behavior to tail -F (capital F on GNU) on the same test cases.

Exit criteria

  • Behaves correctly across rotation and truncation for a 1-hour continuous run against a file being written to by a script.

  • No busy-waiting: top shows near-zero CPU when nothing is happening.

  • strace -c shows the process spends its time in read/ppoll, not nanosleep or spinning.

  • You can explain what happens when a file is rename()d: the inode stays, so the fd stays valid — but future writes go to the new file at the old path, and you must re-open by path to see them.

References:

  • man 7 inotify, man 2 inotify_init1, man 2 inotify_add_watch.

  • GNU coreutils tail.c source — read the -f/-F section for the reference behavior. It’s dense C, but educational.

  • TLPI ch. 19 (Monitoring File Events).

Grading yourself

At the end of M7, look at your two projects and honestly answer:

  • Would you be comfortable demoing this to a senior engineer on your team? If yes, you’re ready for Phase 5. If “almost,” fix the almost.

  • Can you explain every syscall your program makes for a given input? If no, you copy-pasted somewhere. Go back and understand it.

  • Did you write tests for the tricky bits (SIGCHLD races, rotation)? If no, add them now. This is the habit that separates you from a junior dev.

The applied ML engineer in you may protest: “why am I building a shell? I want to write inference kernels.” The answer is that every serious ML infrastructure project you’ll touch — Ray, vLLM, Triton, Kubeflow — launches processes, manages fds, handles signals, and coordinates via IPC. The shell teaches those primitives in miniature. Phase 5 turns them into networked servers. This progression is not accidental.

Return to README.md · Next phase: ../06_concurrency_and_networking/README.md