Domain adaptation

Summary

Domain adaptation transfers a predictor between source and target populations under explicit assumptions about what changed. Specify the shared label space, sampling unit, available target supervision, time period, and deployment slices. Unlabeled target inputs can reveal an input mixture without identifying a changed labeling rule.

Our teaching fixture has two input bins, fixed predictor errors $(0.1, 0.4)$ , and different population mixtures. Compute target risk before choosing a correction.

Shift types

Covariate shift: $P (X)$ changes while $P (Y ∣ X)$ is stable.
Label shift: $P (Y)$ changes while $P (X ∣ Y)$ is stable.
Concept shift: $P (Y ∣ X)$ changes; unlabeled target data alone is generally not enough.

Multiple factors can move together; a drift test on inputs does not establish an unchanged conditional distribution.

The same input distribution can hide a different label rule

Derivation

Rows index X and columns index Y. Check the stated conditional against the cells. Identical target input marginals cannot identify which opposite label rule holds.

Check: How can two targets have identical observed X distributions but require different predictions?

Read it this way: Both domains can show x=0 and x=1 equally often. Their opposite labels make this concept shift, not merely covariate shift.
Can a method checking only the input counts detect this reversed rule?

Work the target risk

Input bin	Source mass	Target mass	Conditional error	Target/source weight
0	0.8	0.2	0.1	0.25
1	0.2	0.8	0.4	4

Source risk is $0.8 (0.1) + 0.2 (0.4) = 0.16$ . Target risk is $0.2 (0.1) + 0.8 (0.4) = 0.34$ . Under stable $P (Y ∣ X)$ and target support contained in source support,

R_{t} (f) = E_{s} [\frac{p _{t} ( X )}{p _{s} ( X )} ℓ (f (X), Y)] .

The weighted source calculation gives $0.8 (0.25) (0.1) + 0.2 (4) (0.4) = 0.34$ . This verifies the algebra, not the real-world invariance assumption. For 80/20 source rows, the weight-based effective sample size is

$\frac{( 80 ( 0.25 ) + 20 ( 4 ) ) ^{2}}{80 ( 0.25 ) ^{2} + 20 ( 4 ) ^{2}} = \frac{400}{13} \approx 30.77.$

Large weights can reduce precision; clipping them changes the estimate and can introduce bias.

Receive target labels

New task: use a small representative labeled target sample to test the assumed relationship.

Picture and change: reserve separate target evaluation; use adaptation labels for calibration or fitting, not both fitting and final scoring.

Check: target conditional error rates $(0.1, 0.7)$ would instead give risk $0.2 (0.1) + 0.8 (0.7) = 0.58$ . Input reweighting alone still returns $0.34$ , so it cannot repair that changed error relationship. Allow sampling uncertainty before concluding that a finite sample establishes concept drift.

Use when: labeled evidence can discriminate assumptions that target inputs alone cannot test.

Lose support or the labeling invariant

New task: deploy where the source has no observations or the target label rule differs.

Picture and change: restrict claims, collect justified labels, or abstain rather than inventing density ratios.

Check: if source mass on bin 1 is zero, its target mass $0.8$ has no source-label support. With error $0.1$ in bin 0 and bounded loss, target risk can range from $0.02$ to $0.82$ . Separately, uniform inputs can accompany either $Y = X$ or $Y = 1 - X$ . Predictor $f (X) = X$ then has risk zero or one despite identical input distributions.

Use when: overlap or conditional invariance fails; matching feature distributions is not proof of transfer.

Approaches

Importance weighting requires the named invariant and support.
Target fine-tuning requires representative supervision and separate evaluation.
Feature alignment requires preserving label-relevant distinctions.
Self-training can reinforce confident mistakes; validate its assumptions and calibration.
Domain-specific normalization or adapters change the fitted model, not identifiability by themselves.
Robust objectives compare the environments actually observed, not every possible future shift.

Under label shift, a fixed classifier statistic can satisfy $q_{t} = C π_{t}$ . Columns of $C$ are its stable prediction distributions given each class; $π_{t}$ contains target class priors. For binary true-positive/false-positive rates $0.8/0.2$ and target positive-prediction rate $0.5$ , the target prior is

$π_{t} (1) = \frac{0.5 - 0.2}{0.8 - 0.2} = 0.5.$

If the two conditional rates coincide, this statistic cannot identify that prior.

Evaluation

Compare source-only and target-supervised baselines on a true target holdout. Report target slices, calibration, source retention, uncertainty, and negative transfer. Reusing target evaluation labels for adaptation requires a new evaluation split.

In an interview

Can unlabeled target inputs distinguish the two opposite label rules above?

Answer

No. Their input likelihoods agree exactly. Target labels or additional structural assumptions are necessary.

Common confusions

Aligned features or lower adaptation loss do not establish preserved class structure. Biased target labels can worsen transfer even when their training loss decreases.