10 — vLLM V1 Source Map (Code Is Truth)

Papers give you the vocabulary. Blog posts give you the concepts. Code is the ground truth. This document is a file-by-file guide to reading vLLM V1 productively — the ~30 files that actually matter, in the order you should read them, with what to look for in each.

Repo: https://github.com/vllm-project/vllm · 66K+ stars (mid-2026) · V1 architecture default since mid-2025. Version pin: always work against a specific commit. This map targets the V1 architecture layout as of 2026. Files may move; the concepts do not.

Orientation — the two mental levels

vLLM has two layers:

V0 (legacy, vllm/engine/, vllm/core/, vllm/worker/): the original design, single-process, synchronous engine, custom BlockSpaceManagerV1/V2. Read only for historical context.

V1 (current, vllm/v1/): multi-process, isolated EngineCore, async by design, unified KV cache manager. This is what you read.

Rule: if a file path in a blog post or issue doesn’t start with vllm/v1/, it’s referring to V0 unless it’s a genuinely shared module (kernels, models, sampling primitives).

The reading plan (7 sessions × ~1 hour)

Session 1 — The engine loop (the heart)

Start here. Everything else is called by this loop.

File

Read for

Key symbols

vllm/v1/engine/core.py

The main busy loop. This is the heart of V1.

EngineCore, run_engine_core, run_busy_loop, _process_engine_step, step

vllm/v1/engine/async_llm.py

The async wrapper the API server talks to.

AsyncLLM, generate (~line 315), output_handler (~line 366), add_request

vllm/v1/engine/core_client.py

IPC between API server and EngineCore (ZMQ).

EngineCoreClient, get_output_async, add_request_async

What to look for: how run_busy_loop alternates between polling for new requests and calling step(). Follow one request from AsyncLLM.generate() → ZMQ send → EngineCore.add_request() → next step() picks it up → outputs stream back through the ZMQ output queue → output_handler yields to the awaiting coroutine.

Mental model to build: the API server and the engine are two separate processes. They talk exclusively via serialized messages. This decoupling is why V1 is fast — the API server never blocks the GPU loop.

Session 2 — The scheduler

This is where continuous batching, chunked prefill, and preemption live.

File

Read for

Key symbols

vllm/v1/core/sched/scheduler.py

The scheduler. Iteration-level batching.

Scheduler, schedule (~line 361-438), update_from_output, _try_schedule_new_request

vllm/v1/core/sched/output.py

The struct passed to the executor per step.

SchedulerOutput

vllm/v1/request.py

Per-request state machine.

Request, RequestStatus (WAITING / RUNNING / PREEMPTED / FINISHED)

What to look for: the schedule() function’s decision order —

  1. Decode-first: for every RUNNING request, budget one token.

  2. Then admit WAITING requests up to the remaining token budget (this is chunked prefill).

  3. If HBM full: preempt (recompute).

The token budget (max_num_batched_tokens) is the master knob you learned about in 04_chunked_prefill.md. Trace where it’s consulted.

Exercise: find where the scheduler decides whether to preempt or wait. Note the strategy.

Session 3 — KV cache management (PagedAttention lives here)

File

Read for

Key symbols

vllm/v1/core/kv_cache_manager.py

Top-level KV manager, prefix caching entry point.

KVCacheManager, get_computed_blocks (~line 157), allocate_slots, free

vllm/v1/core/single_type_kv_cache_manager.py

The per-attention-type manager (full attention, sliding-window, hybrid).

SingleTypeKVCacheManager, find_longest_cache_hit (~line 236)

vllm/v1/core/block_pool.py

The physical block pool + free list.

BlockPool, get_new_blocks, free_blocks

vllm/v1/core/kv_cache_utils.py

Block hashing for prefix cache.

BlockHashType, hash_request_tokens

What to look for:

  • Prefix cache lookup: get_computed_blocks walks the token hashes of an incoming request against the global hash → block dict. The number returned is the number of blocks skipped in prefill.

  • Block allocation: on a miss, BlockPool.get_new_blocks(n) pops from the free list.

  • Eviction: when the free list is empty, LRU eviction over blocks with refcount 0.

  • Refcounting: shared prefix blocks have refcount > 1. Deallocation only frees when refcount hits 0.

Exercise: trace a request that shares a 500-token system prompt with an existing sequence. Count exactly how many prefill FLOPs are skipped.

Session 4 — The worker (GPU side)

File

Read for

Key symbols

vllm/v1/worker/gpu_worker.py

The worker process, wraps model runner.

Worker, execute_model, initialize_cache

vllm/v1/worker/gpu_model_runner.py

Forward pass, sampling, CUDA graph capture.

GPUModelRunner, execute_model, _dummy_run (graph capture), _prepare_inputs

vllm/v1/worker/gpu_input_batch.py

The batch fed to the model.

InputBatch, CachedRequestState

What to look for:

  • How execute_model receives a SchedulerOutput, builds an InputBatch with block tables, calls the model, samples, and returns ModelRunnerOutput.

  • CUDA graph capture: search for capture_model — note how graphs are captured for a fixed set of batch sizes (1, 2, 4, 8, …, max_num_seqs) and how runtime batches are padded to the next captured size.

  • Attention metadata: how AttentionMetadata carries block tables into the attention kernel.

Exercise: find where padding happens for CUDA graphs. Understand why padding is cheaper than launching non-captured kernels.

Session 5 — Attention backends and sampling

File

Read for

Key symbols

vllm/v1/attention/backends/flash_attn.py

FA2/FA3 backend.

FlashAttentionBackend, forward

vllm/v1/attention/backends/flashinfer.py

FlashInfer backend (paged).

FlashInferBackend

vllm/v1/attention/backends/triton_attn.py

Triton fallback — read this for a readable reference impl.

TritonAttentionBackend

vllm/v1/sample/sampler.py

Sampling: greedy, top-k, top-p, penalties.

Sampler, forward

vllm/v1/sample/logits_processor.py

Logit processors: structured output plug-in point.

LogitsProcessor

What to look for:

  • How block tables enter the attention kernel — the kernel gathers KV from non-contiguous physical blocks. This is the whole point of PagedAttention.

  • The sampler: it batches sampling across sequences with different sampling params. Look at how top-p is implemented on GPU.

Session 6 — Speculative decoding and structured output

File

Read for

Key symbols

vllm/v1/spec_decode/

Speculative decoding: draft, verify, accept.

NgramProposer, EagleProposer, MedusaProposer

vllm/v1/spec_decode/eagle.py

EAGLE-3 integration.

EagleProposer, propose_draft_tokens

vllm/v1/structured_output/

XGrammar / Outlines integration.

StructuredOutputManager, Grammar

What to look for:

  • How the drafter runs inside the scheduler step: propose k tokens, verify in one forward pass, accept the longest correct prefix, roll back rejected tokens (this includes freeing their KV blocks).

  • How XGrammar’s token bitmask is applied to logits in the sampler (masked_fill_(-inf) pattern).

Session 7 — Executor and multi-GPU

File

Read for

Key symbols

vllm/v1/executor/abstract.py

The executor interface.

Executor

vllm/v1/executor/multiproc_executor.py

Multi-worker (TP) executor.

MultiprocExecutor, execute_model

vllm/v1/executor/ray_distributed_executor.py

Ray-based multi-node executor.

RayDistributedExecutor

What to look for: how the scheduler stays on one process but the model is sharded across workers. Broadcast pattern for SchedulerOutput → all workers execute → gather sampled tokens.

The full V1 request lifecycle (memorize this)

HTTP POST /v1/chat/completions
  → api_server.py handler
  → tokenize prompt, apply chat template
  → AsyncLLM.generate(req_id, tokens, sampling_params)
     → AsyncLLM.add_request()
     → EngineCoreClient sends AddRequest over ZMQ to EngineCore process

[in EngineCore process]
  run_busy_loop:
    poll input queue → Scheduler.add_request(new_reqs)
    schedule():
      - decode existing RUNNING requests (budget 1 token each)
      - admit WAITING: KVCacheManager.get_computed_blocks(prefix cache hit?)
      - allocate physical blocks via BlockPool
      - fill batch to token_budget (chunked prefill mixes here)
    → SchedulerOutput { req_ids, block_tables, positions, ... }
    → Executor.execute_model(SchedulerOutput)
      → GPUModelRunner._prepare_inputs()
      → model.forward() with paged attention (block tables gathered from HBM)
      → Sampler with logit processors (structured output masks applied)
      → returns ModelRunnerOutput { sampled_token_ids, ... }
    → Scheduler.update_from_output(sampled_tokens)
      - append tokens to sequences
      - mark FINISHED for stop/EOS/max_tokens
      - free blocks (refcount → 0 blocks return to pool)
    → EngineCore pushes RequestOutput to output queue via ZMQ

[back in API server process]
  AsyncLLM.output_handler receives RequestOutput
  → yields to the coroutine awaiting generate()
  → SSE stream sends delta chunks to HTTP client
  → on last chunk, coroutine completes, request removed

Being able to draw this from memory is the exit criterion.

Key contributors to know (as of 2026)

Follow their GitHub activity; their PRs are the actual technical roadmap.

  • Woosuk Kwon (@WoosukKwon) — original vLLM author, PagedAttention.

  • Zhuohan Li (@zhuohan123) — co-author, scheduler work.

  • Simon Mo (@simon-mo) — project co-lead, community.

  • Kaichao You (@youkaichao) — V1 architecture, compiler/graph work.

  • Lily Liu (@LiuXiaoxuanPKU) — speculative decoding, EAGLE integration.

  • Nick Hill (@njhill, RedHat) — API server, async engine internals. He’s the technical reviewer on Aleksa Gordić’s inside-vLLM post — read his PRs.

  • Cody Yu (@comaniac) — KV cache manager, prefix caching.

  • Robert Shaw (@robertgshaw2-neuralmagic) — quantization integration.

  • Alex Marques (@alexm-neuralmagic) — performance/kernel work.

  • SangBin Cho (@rkooo567) — distributed executor.

V1 tracking issue: https://github.com/vllm-project/vllm/issues/8779 — subscribe to this. It’s the master TODO for V1 evolution.

Good-first-issue clusters (where to land your first PR)

Search the repo for label:good first issue, but the recurring themes:

  1. Model architecture support: adding a new model family. Copy the closest existing model file in vllm/model_executor/models/, adapt config, add tests. Great first PR because it’s mechanical but demonstrates you understand the model loading path.

  2. Sampler/logit processor improvements: new sampling method (e.g., DRY, XTC), new penalty type. Isolated file, easy to test.

  3. Benchmark script improvements: add a metric, add a dataset, fix a bug in vllm bench serve. Low-risk, high-value, teaches you the metrics layer.

  4. Docs: the concepts docs (docs/source/) are perpetually incomplete. A well-written page on a subsystem you now understand is welcome and gets you known to maintainers.

  5. Bug reproductions: when you hit a bug during your mini-engine build, file a minimal repro. Even without a fix, a great repro is valuable.

Path to a merged PR: use → benchmark → file great issue → fix small issue → own a feature. Do not skip steps.

External deep-dive resources (in reading order)

  1. vLLM V1 alpha release blog — https://vllm.ai/blog/2025-01-27-v1-alpha-release. The design rationale straight from the team.

  2. Aleksa Gordić’s “Inside vLLM” post — https://www.aleksagordic.com/blog/vllm (also mirrored at https://vllm.ai/blog/2025-09-05-anatomy-of-vllm). Technically reviewed by Nick Hill. The single best deep-dive on V1. Read this alongside the code.

  3. Ubicloud’s “Life of an inference request (vLLM V1)” — https://www.ubicloud.com/blog/life-of-an-inference-request-vllm-v1. Good complement to Aleksa’s post.

  4. vLLM docs, Concepts → Architecture — https://docs.vllm.ai/en/latest/design/. The maintained architecture docs.

  5. vLLM meetup talks on YouTube — “vLLM Meetup” playlist. Kwon, Mo, Youkaichao have all given hour-long talks. Better than any tutorial.

Reading discipline

  • Every session, one commit hash. Note it in your lab notebook. Files move; commit hashes don’t.

  • Trace one request end-to-end per session. Don’t wander. Pick a scenario (chat completion, batch inference, streaming) and follow it.

  • After each session, redraw the diagram from memory. If you can’t, re-read.

  • Contribute reading notes back. If you understand a file, propose a docstring-only PR clarifying it. Trivially mergeable, and now you’re a vLLM contributor.

vLLM’s code is not clean, not simple, and moves weekly. That’s normal for an OSS project moving this fast. Your goal isn’t to understand every line. Your goal is to know where to look when a specific question comes up. This map is that lookup table.