Book V · Chapter 4
Inference systems and cost
Design a serving system, estimate cost, and choose measured optimizations.
Read first: Inference and decoding
Chapter contents
5 entries · read in order- 01 Design a production LLM inference service✓ Question
- 02 Train and serve a reasoning model under a fixed compute budget✓ Question
- 03 Annotated upper-IC mock: reasoning under a fixed budget✓ Guide
- 04 How would you reduce LLM inference cost by 10x?✓ Question
- 05 How to think about LLM inference cost✓ Guide