Book III · Chapter 4
Product and experimentation practice
Choose metrics, run valid tests, diagnose disagreement, and make a ship decision.
Read first: Evaluation design and validity
Chapter contents
6 entries · read in order- 01 Causal inference for ML decisions✓ Concept
- 02 Delayed labels, selective labels, and feedback loops✓ Concept
- 03 Choose metrics for an ML product✓ Question
- 04 Design an A/B test for a new ML model✓ Question
- 05 How do you A/B test a chatbot?✓ Question
- 06 Your offline metric improved but the online metric got worse. Debug it.✓ Question