Systems Performance Engineering — learning map

Eight books, 75 chapters. Click a chapter to read it. Inside a chapter, the arrows move to the previous and next chapter. The figures and diagrams from each chapter are placed in the text where they belong. Every chapter also opens with a drawn animation, a structure diagram, and a three step study guide.

New to these topics? Start here

Foundations — 22 animated pictures and 7 structure diagrams, no assumed background. Bytes, cache lines, cores, threads, pointers, locks, network calls, replication, consensus, and the two numbers that decide speed. Ten minutes, then every chapter below is readable.

The stack, in one picture

Every chapter in the map sits in one of six layers. Read a layer from the bottom up and you have the whole machine; read a component and you land on the chapter that teaches it.

L6ApplicationThe workload you run — jobs and services, not the machine.
Serving and routingFregly Ch 15Prefill and decode splitFregly Ch 17Adaptive enginesFregly Ch 19Inference at scaleFregly Ch 16Training and tuningFregly Ch 13
L5AI logicHow the model meets the machine — kernels, math, lowerings.
Kernels and occupancyFregly Ch 6Memory access patternsFregly Ch 7Kernel efficiencyFregly Ch 9Pipelining and CUDA graphsFregly Ch 12Quantization and KV cacheFregly Ch 18Triton and the compilerFregly Ch 14
L4Distributed servicesHow machines work together: call, copy, agree.
gRPC under the hoodgRPC Ch 4Failure detectionPetrov Ch 9Leader electionPetrov Ch 10Replication and consistencyPetrov Ch 11Anti-entropy and gossipPetrov Ch 12ConsensusPetrov Ch 14
L3Systems runtimeThe language, the memory model, the primitives.
Ownership and movesRust Ch 4Traits and genericsRust Ch 11ConcurrencyRust Ch 19Atomics and orderingRust Atomics Ch 2Locks and channelsRust Atomics Ch 9Futures and asyncAsync Rust Ch 6Runtimes and wakersAsync Rust Ch 8
L2ToolchainHow source becomes instructions.
LLVM IRLLVM Essentials Ch 2Optimization passesLLVM Hsu Ch 9TableGenLLVM Hsu Ch 4Clang front endLLVM Hsu Ch 5IR instrumentationLLVM Hsu Ch 12
L1InfrastructureThe machine and the fabric under everything.
Hardware overviewFregly Ch 2OS, Docker, KubernetesFregly Ch 3Distributed networkingFregly Ch 4GPU storage I/OFregly Ch 5Processor and OS primitivesRust Atomics Ch 7Scaling outwardFregly Ch 20

Read the layers bottom up: hardware first, then the toolchain, the language, the network, the kernels, and the workload on top. Each chip opens the chapter that teaches it.

The system these books add up to

The chapters describe parts of one machine. This is that machine, as it runs in production: each box is a topic in the map below, and each arrow is a boundary where something is lost.

Every chapter in this map is one box or one arrow in this picture. Read it left to right: a request enters, work is split, memory is shared, and a device runs it.

The serving system the eight books add up toClientsSDK, HTTP, gRPCGatewayauth, quota, routingAPI serverrequest shapingPrefill workersprompt, build KVKV cache storepaged, sharedDecode workerstoken loopWeight storeobject storageGPU poolSMs, HBM, NVLinkInference serviceDevice fleetmTLSroutepromptbuild KVwrite KVread KVkernelsload
request comes from heredecides where work goesdoes the workholds statemoves bytesyour boundary

The API routes work to both pools: prefill takes the prompt, decode takes the token loop. Prefill writes the cache and decode reads it, and the GPU pool runs kernels for both. The dashed boundary is a trust and failure boundary: what is inside is your service, what is outside you do not control.

Read it in: Fregly 15, multinode inferenceFregly 17, disaggregated prefill and decodeFregly 19, adaptive engines

1. GPU architecture and kernels

Hardware overview: SMs, HBM, NVLink, PCIe
Fregly Ch 2
GPU architecture, CUDA basics, occupancy
Fregly Ch 6
Memory access patterns, coalescing, profiling
Fregly Ch 7
Occupancy tuning, warp efficiency, ILP
Fregly Ch 8
Kernel efficiency, arithmetic intensity, roofline
Fregly Ch 9
Intra-kernel pipelining, warp specialization, thread block clusters
Fregly Ch 10
Inter-kernel pipelining, streams, stream-ordered allocation
Fregly Ch 11
CUDA graphs, dynamic scheduling, device-side launch
Fregly Ch 12
CUTLASS and CuTe
Fregly Ch 9Fregly Ch 10
Triton kernels
Fregly Ch 14

2. Distributed systems

Failure models, partial failure, Two Generals, FLP
Petrov Ch 8
Failure detection, heartbeats, phi-accrual
Petrov Ch 9
Leader election
Petrov Ch 10
CAP, consistency models, linearizability, CRDTs
Petrov Ch 11
Anti-entropy, gossip, Merkle trees
Petrov Ch 12
Distributed transactions, 2PC/3PC, consistent hashing
Petrov Ch 13
Consensus: Paxos, Raft, PBFT
Petrov Ch 14
gRPC fundamentals and protobuf
gRPC Ch 1gRPC Ch 2
Communication patterns, streaming
gRPC Ch 3
gRPC internals: HTTP/2 framing, encoding
gRPC Ch 4
Interceptors, deadlines, metadata, load balancing
gRPC Ch 5
gRPC in production
gRPC Ch 7
NCCL, collectives, InfiniBand, fabric tuning
Fregly Ch 4
GPU storage I/O, GPUDirect
Fregly Ch 5

3. Systems language (Rust)

Ownership, moves, references, lifetimes
Rust Ch 4Rust Ch 5
Traits, generics, closures, iterators
Rust Ch 11Rust Ch 14Rust Ch 15
Error handling, crates, modules
Rust Ch 7Rust Ch 8
Threads, channels, Send/Sync
Rust Ch 19
Async and futures — language level
Rust Ch 20
Unsafe code and raw pointers
Rust Ch 22
FFI — calling into C and CUDA
Rust Ch 23
Atomics
Rust Atomics Ch 2
Memory ordering
Rust Atomics Ch 3
Building a spin lock
Rust Atomics Ch 4
Building channels
Rust Atomics Ch 5
Building Arc
Rust Atomics Ch 6
Cache lines, false sharing, the processor
Rust Atomics Ch 7
OS primitives: futex, parking
Rust Atomics Ch 8
Building mutexes and condvars
Rust Atomics Ch 9
Concurrency and parallelism models
Async Rust Ch 1Async Rust Ch 2
Event queues, syscalls, epoll/kqueue/IOCP
Async Rust Ch 3Async Rust Ch 4
Fibers and green threads
Async Rust Ch 5
Futures in Rust
Async Rust Ch 6
Coroutines and async/await desugaring
Async Rust Ch 7
Runtimes, wakers, reactor-executor
Async Rust Ch 8
Self-referential structs and Pin
Async Rust Ch 9
Building your own runtime
Async Rust Ch 10

4. PyTorch internals

Profiling, tuning, and scaling PyTorch
Fregly Ch 13
torch.compile, Inductor, Triton, XLA backends
Fregly Ch 14

5. LLM inference systems

Multinode inference, parallelism, decoding, routing
Fregly Ch 15
Speculative decoding
Fregly Ch 15
Profiling and debugging inference at scale
Fregly Ch 16
Disaggregated prefill and decode
Fregly Ch 17
KV-cache tuning, paged attention
Fregly Ch 18
Dynamic and adaptive engine optimizations
Fregly Ch 19

6. Quantization and compilers

FP8, NVFP4, GPTQ, AWQ, SmoothQuant, KV-cache quantization
Fregly Ch 9Fregly Ch 18Fregly Ch 19
LLVM IR syntax and IRBuilder
LLVM Essentials Ch 1LLVM Essentials Ch 2LLVM Essentials Ch 3
LLVM build system and tooling
LLVM Hsu Ch 1LLVM Hsu Ch 2
LIT testing
LLVM Hsu Ch 3
TableGen
LLVM Hsu Ch 4
Clang architecture and AST
LLVM Hsu Ch 5LLVM Hsu Ch 7
PassManager and AnalysisManager
LLVM Hsu Ch 9
Processing LLVM IR, writing passes
LLVM Hsu Ch 10LLVM Hsu Ch 11
IR instrumentation and PGO
LLVM Hsu Ch 12

7. Infrastructure and operations

OS, Docker, Kubernetes tuning for GPU nodes
Fregly Ch 3
Scaling to very large GPU clusters
Fregly Ch 20

8. Reliability and benchmark practice

Profiling and tuning at scale
Fregly Ch 16
Optimization checklist (175+ items)
Fregly Appendix

Suggested reading order

1. Fregly 6-9 -> write Triton kernels
2. Fregly 13-14 -> PyTorch and compiler
3. Fregly 15-19 -> inference, then read the vLLM source
4. Blandy 1-15 -> Rust basics
5. Blandy 19, 22 + Bos 1-6 -> concurrency
6. Samson 1-10 -> async runtime, then build the proxy project
7. Fregly 2-5 -> hardware and fabric
8. Fregly 10-12 -> advanced CUDA
9. Petrov 8-14 and Indrasiri 1-5 -> distributed theory and RPC
10. Pandey 1-3, then Hsu -> compilers, only if you target that work
11. Fregly 3, 20 -> infrastructure

Not in these eight books

AMD ROCm / HIP
no book — use AMD docs
MLIR
no book — use MLIR docs + Triton source
Kubernetes operators, CRDs, Terraform
no book — lowest priority
Books.
Fregly — AI Systems Performance Engineering — Chris Fregly
Petrov — Database Internals — Alex Petrov
gRPC — gRPC: Up and Running — Kasun Indrasiri & Danesh Kuruppu
Rust — Programming Rust, 2nd ed. — Blandy, Orendorff, Tindall
Rust Atomics — Rust Atomics and Locks — Mara Bos
Async Rust — Asynchronous Programming in Rust — Carl Fredrik Samson
LLVM Essentials — LLVM Essentials — Sarda & Pandey
LLVM Hsu — LLVM Techniques, Tips, and Best Practices — Min-Yih Hsu
Chapter text and figures were extracted from the PDFs. Running heads, page numbers, and figure whitespace were removed. Code listings keep their monospace layout. Nothing was reworded.