Matrix factorization for recsys

Summary

A missing rating supplies two indices: a user row and an item row. Matrix factorization shares learned rows across many observed ratings rather than storing an independent prediction per cell. Its low-rank assumption is a modeling choice, not a guarantee of matrix completion.

Let $R \in R^{m \times n}$ , $U \in R^{m \times k}$ , and $V \in R^{n \times k}$ . The shared coordinate count is $k$ , and $R \approx U V^{⊤}$ . With zero biases, fixture rows $u_{A} = (2, 1)$ and $v_{3} = (1.5, 1)$ predict $2 (1.5) + 1 (1) = 4$ . These are chosen vectors, not a fitted solution for all displayed ratings.

Matching user and item coordinates multiply, then sum to the missing rating

Derivation

Read it this way: Keep user A and item 3 fixed through every stage. Their matching factor rows contribute 3 and 1. The predicted missing rating is 4, not a newly observed preference.

Check: Which two factor rows predict the missing A/item-3 entry? How do their matching coordinates produce 4?

Read it this way: The example user vector (1,2) and item vector (3,1) predict 1x3+2x1=5. This is a model estimate, not an observed rating.
Which observation would be required to call five a recorded rating?

Entry in user A’s row	Observation	Explicit-rating loss
Item 1	Rating 5	Included
Item 3	Unknown	Excluded
Item 4	Rating 1	Included

The objective

Minimize regularized squared error on observed ratings $Ω = {(u, i) : r_{u i} observed}$ :

U, V min (u, i) \in Ω \sum (r_{u i} - u_{u}^{⊤} v_{i})^{2} + λ (∥ U ∥_{F}^{2} + ∥ V ∥_{F}^{2}) .

The observed set $Ω$ excludes unknown entries, and $λ \geq 0$ controls the stated L2 penalty. With biases, use $\overset{r}{^}_{u i} = μ + b_{u} + b_{i} + u_{u}^{⊤} v_{i}$ , defining global mean and user/item offsets. Fit these together; adding $μ = 3, b_{A} = 0.2, b_{3} = - 0.1$ afterward changes four to $7.1$ . That exceeds a 1-5 rating scale and is not automatically a calibrated prediction.

Change observed signal

New task: replace explicit ratings with clicks whose exposure is not fully observed.

Picture and change: a confidence-weighted implicit model uses

U, V min u, i \sum c_{u i} (p_{u i} - u_{u}^{⊤} v_{i})^{2} + λ (∥ U ∥_{F}^{2} + ∥ V ∥_{F}^{2})

Here $p_{u i}$ is an interaction indicator and $c_{u i} \geq 0$ is modeling confidence, not an identified exposure probability. Unlike explicit-rating loss, this objective includes all pairs, usually with lower confidence for unobserved interactions.

Check: a separate two-item fixture has scores $(0.8, 0.2)$ , preferences $(1, 0)$ , and confidences $(5, 1)$ . Its unregularized loss is $5 (1 - 0.8)^{2} + (0 - 0.2)^{2} = 0.24$ , not the positive-only value $0.20$ . A never-shown item’s weak zero preference remains an assumption, not evidence of dislike.

Use when: revise the objective and evaluation when observation semantics change.

Cold start

Change item availability

New task: score a new item without a trained ID row.

Picture and change: use a separately validated feature encoder or content/context fallback, rather than a nonexistent lookup row.

Check: if an encoder supplies $(1, 0)$ , user A’s zero-bias score is $2 (1) + 1 (0) = 2$ . Increasing $k$ alone cannot produce a trained row for an unseen ID. An encoder can compute a vector without proving cold-item accuracy.

Use when: evaluate the cold-item slice and fallback separately from seen-ID performance.

Training

Squared-error factorization is nonconvex jointly, but convex in either factor when the other is fixed. ALS solves conditional ridge problems; SGD updates selected rows. For one pair’s squared residual and two-row L2 penalty, hold biases fixed and save both old rows. Set $e = r - μ - b_{u} - b_{i} - u_{old}^{⊤} v_{old}$ , then compute simultaneously

u_{new} = (1 - 2 η λ) u_{old} + 2 η e v_{old}, v_{new} = (1 - 2 η λ) v_{old} + 2 η e u_{old} .

Using the updated user row for the item gradient changes this step. Sampling and regularization scaling must match the full objective.

Properties

For orthogonal $Q$ , replacing $(U, V)$ with $(U Q, V Q)$ preserves predictions and L2 norms. The fixture rows can become $(1, - 2)$ and $(1, - 1.5)$ while retaining dot product four. Tune $k$ and regularization against matched baselines; coordinates are not uniquely identifiable tastes.

Connection to two-tower models

Two-tower retrieval replaces free rows with feature encoders $f_{θ}, g_{ϕ}$ . An ID-only tower still lacks trained parameters for new IDs. Sampled retrieval losses differ from squared-error ratings and all-pair implicit fitting.

When to use plain MF in 2026

MF can be a rating baseline, prototype, or embedding initializer. Choose architecture from task, latency, features, and measured validation, not a universal deployment-prevalence claim.

Common pitfalls

Dot-product prediction does not validate within-type cosine; normalization changes scores. Compare bias baselines and ranking metrics when top-K quality matters; RMSE answers a different question.

Can a PCA rotation identify unique causal tastes?

Answer

No. It chooses coordinates without removing equivalent factorizations or establishing causal meanings.

References: Koren et al. (2009), Hu et al. (2008).

Two-tower retrieval. Neural generalization.
Embedding spaces. How the latent factors are used.