Skip to content
mlmentorship

How would you do cold-start for a new user?

Use permitted context, item content, optional preferences and bounded exploration while measuring feedback uncertainty and whole-cohort outcomes.

Published · 6 min read ·Role-specific ·Advanced

ML breadth · active recall

Practice before you read

8 minutes. Explain the mechanism, why it works, when it fails, and one alternative.

Back to my workbook How practice works

ML breadth · closed-book attempt

How would you do cold-start for a new user?

Explain the mechanism, why it works, when it fails, and one alternative.

Problem

Context. A new user can receive recommendations from eligible strategies A and B before any personal behavior is available. The teaching fixture below models saves after confirmed exposure. Task. Propose initial recommendations, update the stated probability model after the observations, and identify what remains uncertain. Explain how feedback could change the mix without assuming a fixed interaction count establishes reliable personalization. Constraints. Use only permitted context and optional preferences; preserve eligibility and safety boundaries. Respect the reward's observation window, and distinguish a matured no-save from dislike or unobserved exposure. A new user can receive eligible recommendations from strategies A and B. Use optional preferences, permitted request context and item content before behavior exists. Decide how early feedback should change the mix without claiming certainty or bypassing safety. For a teaching model, strategy A has independent Bernoulli rewards with stable save probability $p$. A success means a save within 24 hours after confirmed exposure; a matured no-save is failure for this reward. That failure does not establish dislike. Start with $p\sim\operatorname{Beta}(1,1)$ and observe success, success, failure. What changes, and what remains unknown?

08:00recommended time

Closing or reloading clears the scratchpad. Only score, weak rubric dimensions, attempt count, and retry date can be stored locally.
Reading depth

Design recommendations for a user with no interaction history and explain how limited early feedback should update decisions.

Problem

Context. A new user can receive recommendations from eligible strategies A and B before any personal behavior is available. The teaching fixture below models saves after confirmed exposure.

Task. Propose initial recommendations, update the stated probability model after the observations, and identify what remains uncertain. Explain how feedback could change the mix without assuming a fixed interaction count establishes reliable personalization.

Constraints. Use only permitted context and optional preferences; preserve eligibility and safety boundaries. Respect the reward’s observation window, and distinguish a matured no-save from dislike or unobserved exposure.

A new user can receive eligible recommendations from strategies A and B. Use optional preferences, permitted request context and item content before behavior exists. Decide how early feedback should change the mix without claiming certainty or bypassing safety.

For a teaching model, strategy A has independent Bernoulli rewards with stable save probability . A success means a save within 24 hours after confirmed exposure; a matured no-save is failure for this reward. That failure does not establish dislike. Start with and observe success, success, failure. What changes, and what remains unknown?

Picture: Update Belief, Not a Session Counter

The Beta prior contributes one unit to each parameter, not two observed interactions. Add the two successes to the first parameter and the failure to the second.

The mean rises from 0.50 to 0.60, but probability mass remains spread across possible values. Untried B keeps its own uncertainty; A’s three observations do not establish B’s value.

Three mature observations, still uncertain

Three observations reshape uncertainty; they do not prove a fixed success rate

Three outcomes update a distribution, not a certaintyStarting from Beta(1,1), two successes and one failure give Beta(3,2). Its mean is 0.6, but substantial uncertainty remains.Evidence reshapes a distributionPrior: every probability has equaldensityTwo successes; one failureMultiply by p x p x (1-p), thennormalizeMean 0.6; uncertaintyremains
Derivation
Uniform Beta prior and the posterior after two successes and one failure The dashed Beta(1,1) prior has constant density one. The solid Beta(3,2) posterior has density twelve times p squared times one minus p. Both integrate to one. The posterior mean is 0.60 and its peak is at two thirds, with substantial uncertainty remaining. Prior: Beta(1,1) After S, S, F: Beta(3,2) Density 0 1 2 0 0.5 1 Mean 0.60 Strategy A's save probability p
Read it this way: the solid curve shifts belief toward larger save probabilities without collapsing to one value. These are illustrative distributions under the stated reward assumptions, not a production user classifier.
Check: Why do two successes and one failure yield Beta(3,2), not certainty about the save rate?
Read it this way: Starting from Beta(1,1), two successes and one failure give Beta(3,2). Its mean is 0.6, but substantial uncertainty remains.
Do three observations make every probability except 0.6 impossible?

Decision Contract

First filter eligible items/actions using safety, availability, consent and privacy rules. Then blend available preference, context, content and behavior evidence using calibrated confidence, not a universal interaction count. Bound exploration against an acceptable fallback. Keep exposure, strategy/policy, selection probability, context, feedback window and missingness with each update. Pending or unexamined impressions are not observed dislikes.

Change 1: Onboarding Causes Abandonment

New task. Preference questions produce richer profiles among completers but increase abandonment.

Picture and change. Compare optional/minimal questions against context/content fallback, assigning users before onboarding. Retain abandoners in whole-cohort retention and successful-recommendation outcomes.

Check. Better recommendations among completers can coexist with worse results for all assigned users.

Use when. Collecting signal has product cost.

Change 2: The Item Is New

New task. A new item has content but no engagement history.

Picture and change. Retrieve it through eligible content features and consider bounded exploration with quality and item/creator coverage checks.

Check. A boost cannot bypass eligibility or guarantee value.

Use when. The cold side is the catalog, not the user.

Failure: Unshown Means Disliked

If popular items receive every impression, unseen categories have no feedback. Treating their missing clicks as negative preferences reinforces the same selection without learning about alternatives.

Transfer Check

Does higher engagement in session ten prove that learning improved cold-start?

Reasoned Answer

No. Returners differ from users who left, and intent may change. Compare identified cohorts with explicit follow-up maturity and censoring.

Signal Sources and Exploration Limits

Content features and two-tower models can connect eligible items with stated or observed interests before item-specific interaction exists. Similar-user aggregates can supply priors only when their use is permitted and relevant. Available demographics do not imply permission or useful prediction.

An epsilon-greedy or Thompson-sampling strategy needs eligible actions, defined rewards, sufficient exposure support and a nonstationarity policy. The Beta example assumes stable, independent rewards; actual slates, repeated exposures and changing intent can violate that assumption. No algorithm guarantees faster learning or retention. Continue only justified exploration after early sessions; reset or adapt when evidence becomes stale.

Evaluate New-User Outcomes

Define success and time to useful recommendations before testing; report users who never reach the threshold. Compare new-user strategies with cohort assignment, uncertainty, sample/power planning and privacy/safety/quality guardrails. Measure first-session outcomes, mature day-N retention and time-to-event outcomes with explicit censoring. Use the full assigned cohort, not only onboarding completers or returning users.

Sources

Related: Design YouTube’s recommender, Design Spotify’s homepage, System design case study: building personalized search ranking.

Found an error or stale claim? Report a correction or send a private note.