Book V · Chapter 1
Hardware and performance
Count work and memory, understand accelerator limits, then read a trace.
Read first: Neural network foundations
Chapter contents
6 entries · read in order- 01 Transformer compute and memory accounting✓ Concept
- 02 GPU memory hierarchy: HBM, SRAM, and roofline reasoning✓ Concept
- 03 Accelerator network topology for distributed ML✓ Concept
- 04 Profiling distributed ML workloads✓ Concept
- 05 Optimize an accelerator workload from a trace✓ Question
- 06 Chinchilla scaling, MoE, and fused Triton kernels✓ Concept