Book IV · Chapter 2

Tokens, position, and long context

Understand input representation, positional information, and context extension.

Priority

Core

Difficulty

Intermediate

Useful for

RS, RE, MLE, AS

Interview rounds

ML breadth, LLM systems

Read first: Transformer architecture and attention