Book V
ML systems and infrastructure
Accelerators, distributed training, inference systems, reliability, cost, and full ML architecture.
6 chapters · 30 entriesChapters
6Hardware and performance
Count work and memory, understand accelerator limits, then read a trace.
Distributed training and parallelism
Build from collectives and sharded matrix operations to multi-axis parallel plans.
Training plans and reliability
Size a run, choose a layout, and preserve consistent state through failures.
Inference systems and cost
Design a serving system, estimate cost, and choose measured optimizations.
ML platforms and operations
Version data and models, then design serving, monitoring, budget, and freshness as one operating system.
Applied ML system design
Apply the lifecycle to high-stakes classification and human-review systems.