Embedding spaces and similarity metrics

Summary

An embedding maps an object to a vector; its training objective determines which vector relationships should predict relevance. Choose whether length, direction, or absolute distance should affect retrieval before choosing the index. Normalization changes the ranking objective unless the model and task already make length irrelevant.

Common similarity metrics

For two vectors $u, v \in R^{d}$ :

Metric	Formula	When to use
Dot product	$u^{⊤} v$	When magnitudes are meaningful (e.g., learned matrix-factorization scores). Indexable with MIPS algorithms.
Cosine	$\frac{u ^{⊤} v}{∥ u ∥ ∥ v ∥}$	Default for embeddings where direction encodes meaning and magnitude is a noise/popularity confound.
Euclidean / L2	$∥ u - v ∥_{2}$	When distances have physical meaning (image patches in pixel space, geographic coordinates).
Negative Euclidean²	$- ∥ u - v ∥^{2}$	Equivalent to dot product on L2-normalized vectors plus a constant.

Cosine requires nonzero vectors and explicitly discards positive scale. Some models train with normalized similarities; others use unnormalized scores. Follow the model’s documented scoring contract instead of assuming all embedding models share one metric.

L2 normalization

For nonzero $u, v$ , define $u = u / ∥ u ∥$ and $v = v / ∥ v ∥$ . Then

u^{⊤} v = cos (u, v), ∥ u - v ∥^{2} = 2 - 2 cos (u, v) .

Thus normalized dot product, cosine, and normalized squared distance induce the same exact ranking, including ties. The identity follows by expanding the squared difference and substituting both unit norms.

Take query $q = (1, 0)$ and candidates $A = (1.8, 1.6)$ , $B = (0.9, 0.2)$ . Raw dot product favors A: $1.8 > 0.9$ . Cosine favors B: approximately $0.976 > 0.747$ . Their normalized chord distances are approximately $0.218 < 0.711$ .

Metric geometry

Normalization removes the length advantage that made A win the raw dot product

Derivation

Before: magnitude can win

Raw dot product rewards length as well as alignment.

After: direction decides

Normalization preserves each angle and sets every length to 1.

Read it this way: raw dot product favors the longer A. Unit normalization preserves angles but discards length. Cosine, normalized dot product, and normalized chord distance then agree on B.

Check: Why does B win both normalized dot and nearest L2 even though A wins the raw dot product?

Read it this way: For q=(1,0), A=(10,10) beats B=(1,0) by raw dot product. Cosine reverses the order because B points exactly along q.
After normalization, can increasing only A’s original length improve its cosine?

Normalize stored candidates once when cosine is the intended objective. Normalize each nonzero query or divide scores by its norm when actual cosine values are needed. Declare a rejection or fallback policy for zero vectors; adding an arbitrary epsilon changes the score definition.

Change the Requirement

Retain Meaningful Magnitude

New task: Serve a model trained to encode relevance through both vector length and direction.

Picture and change: Use the raw dot product rather than projecting onto the unit sphere.

Check: The same fixture now ranks A above B, matching its unnormalized objective.

Use when: Training and evaluation support that scoring contract. Length may be useful signal or an unwanted association; geometry alone cannot determine which.

Trade Exact Search for Candidate Speed

New task: Retrieve from an approximate index under a latency budget.

Picture and change: Candidate generation may omit the exact nearest item; reranking cannot recover an absent candidate.

Check: If exact top-one is B but the index returns only A, top-one recall is zero despite correct reranking.

Use when: Measured latency and recall justify approximation under the same metric, filters, and data version.

Geometry of learned embeddings

Clusters, linear analogies, anisotropy, and distance concentration depend on the model and input distribution. Whitening or centering changes geometry; validate relevance afterward rather than assuming improvement. Compare dimensions through measured retrieval quality, memory, and latency, not a universal preferred range.

Indexing for fast retrieval

For $N$ items and queries:

Method	Decision to measure
Exact scan	$O (N d)$ arithmetic; establishes the same-metric reference ranking
HNSW	Graph-search breadth, build cost, recall, and memory
IVF with product quantization	Number of searched partitions and quantization error
ScaNN	Candidate-search and quantization settings under the workload

Approximate latency and recall are empirical workload properties, not universal logarithmic guarantees or fixed percentages.

Common pitfalls

Mixed objectives. Normalizing only some candidates invalidates the unit-vector equivalence.
Mixed coordinate systems. Independently trained encoders need a validated shared-space or alignment contract.
Unproved bias claims. Removing length changes scores; it does not establish fairer or better retrieval.

Reconstruct

Can exact reranking fix a missing relevant candidate?

No. Reranking only reorders supplied candidates. Measure candidate coverage separately from ranking quality, including filtered and tail-query populations.