Skip to content
mlmentorship

Fine-tuning vs prompting: the deep version

Past the basic decision tree. The senior answer covers SFT, LoRA, DPO, continued pretraining, and the operational trade-offs each introduces.

Published · 7 min read ·Role-specific ·Intermediate

ML breadth · active recall

Practice before you read

8 minutes. Explain the mechanism, why it works, when it fails, and one alternative.

Back to my workbook How practice works

ML breadth · closed-book attempt

Fine-tuning vs prompting: the deep version

Explain the mechanism, why it works, when it fails, and one alternative.

Problem

Context. A team has a base language model, task examples, and a behavior gap that requires evaluation before adaptation. The intervention-choice question addresses whether fine-tuning is needed at all. Task. Specify the training objective, data construction, trainable parameters, evaluation, and release plan for an appropriate fine-tuning method. Compare alternatives and explain what evidence would change the choice. Constraints. Respect compute, data rights, privacy, and serving requirements. Separate behavior learning from changing factual knowledge, guard against leakage and regressions, and do not presume adaptation always helps.

08:00recommended time

Closing or reloading clears the scratchpad. Only score, weak rubric dimensions, attempt count, and retry date can be stored locally.

Asked in: LLM-team interviews; deeper variant of the standard question.

Choose a fine-tuning method and a defensible training plan once a model-adaptation requirement has been established.

Problem

Context. A team has a base language model, task examples, and a behavior gap that requires evaluation before adaptation. The intervention-choice question addresses whether fine-tuning is needed at all.

Task. Specify the training objective, data construction, trainable parameters, evaluation, and release plan for an appropriate fine-tuning method. Compare alternatives and explain what evidence would change the choice.

Constraints. Respect compute, data rights, privacy, and serving requirements. Separate behavior learning from changing factual knowledge, guard against leakage and regressions, and do not presume adaptation always helps.

What the L4 candidate misses

L4: “Fine-tune the model on examples.” Misses that “fine-tune” covers different training objectives (continued pretraining, SFT, preference optimization) and a separate choice about which parameters to update (full fine-tuning or a parameter-efficient method such as LoRA).

What an L5 answer sounds like

“There are three common training objectives, plus a separate choice about update scope:

  1. Continued pretraining: continue the next-token-prediction objective on a domain corpus. Use case: shifting the model’s underlying knowledge or vocabulary (e.g., medical, legal, code in a niche language). Expensive, requires lots of unlabeled data.

  2. Supervised fine-tuning (SFT): train on (prompt, ideal-response) pairs. Use case: teaching the model the format, style, or specific behaviors. Standard for instruction tuning. Moderate cost, moderate data needs (10K-1M pairs).

  3. Preference optimization (DPO, IPO, RLHF): train on pairwise preferences (A > B). Use case: shaping nuanced behavior that’s hard to specify in SFT pairs (helpfulness, conciseness, tone). DPO has largely displaced RLHF in 2026 due to simpler training.

  4. Parameter-efficient methods (LoRA, QLoRA, prefix tuning) are not a fourth objective. They change how you update a model: train a small set of additional parameters while freezing the base model. Use case: many specialized models on the same base, or training under tight memory constraints. The quality trade-off is task-dependent, while the memory and serving benefits can be large.

The decision tree:

  • Need new knowledge → continued pretraining + SFT, or RAG (usually RAG is cheaper).
  • Need consistent format / behavior → SFT.
  • Need nuanced quality (helpful, on-tone) → SFT then DPO.
  • Need many task-specific variants of the same base → LoRA.
  • Need just a behavior tweak → prompting first, fine-tuning only if prompting can’t get there.”

Choose a training signal and an update scope as separate decisions

Training signal and update scope form two independent axesSupervised or preference signals can update all parameters or a parameter-efficient subset. Choosing one axis does not choose the other.Choose supervision and update scopeindependentlylabelsfulllabelsLoRApairsfullpairsLoRAEach supervision choice has bothscope choices.
Derivation
Choose a training signal and an update scope, not one item from a mixed listThree objective rows use domain text, prompt/ideal-response pairs, and preference triples. Each crosses full base-weight updates and LoRA adapter updates with the base frozen. The SFT-plus-LoRA cell is highlighted. The rows are alternatives, not a mandatory sequence.Objective and update scope are independentchoicesFULLLoRA1. Domain textnext-token lossbase WTRAINABLEW FROZENadapter A,BTRAINABLE2. Prompt + ideal responseSFT lossbase WTRAINABLEW FROZENadapter A,BTRAINABLE3. Prompt + preferred / rejectedpreference lossbase WTRAINABLEW FROZENadapter A,BTRAINABLEHighlighted cell: SFT objective + LoRAscopeRows are alternatives, not a requiredthree-stage sequence.
SFT tells you which loss and examples train the model. LoRA tells you where trainable updates live. Their highlighted intersection is one valid combination.
Check: Can preference optimization use LoRA? Why are those not competing alternatives?
Read it this way: Supervised or preference signals can update all parameters or a parameter-efficient subset. Choosing one axis does not choose the other.
Does choosing preference pairs force either full-scope or LoRA updates?

This is L5. You’ve named the objectives and update scope, given use cases, and a decision tree.

What an L6 answer adds

“…operational realities:

Fine-tuning has lock-in cost. You’re tied to the base model checkpoint. When the provider deprecates it (typical 12-18 months for hosted models), you re-tune. When a new model family launches, you re-tune to evaluate. Plan for this in your roadmap; teams that don’t get caught flat-footed when their tuned model goes EOL.

Data quality dominates fine-tuning quality. A small (~10K) carefully-curated SFT dataset typically beats a large (~1M) noisy one. The team’s time spent on data is the highest-leverage activity. Synthetic data (LLM-generated) is increasingly common and works well when filtered by a strong judge.

DPO is operationally simpler than RLHF. No reward model, no PPO. Pairwise preference data is enough. Most teams in 2026 default to DPO; RLHF is reserved for the most polished products at the largest labs.

LoRA quality is close to full fine-tuning for most adaptation tasks. Memory savings are substantial: you can train LoRA adapters on a single GPU for models that need 8 GPUs to full-fine-tune. Multiple LoRAs can be served from one base model with adapter swapping at request time, useful for multi-tenant systems.

Continued pretraining is rarely worth it unless you have a domain corpus measured in tens of billions of tokens and a budget. Most teams better-served by RAG + SFT for domain adaptation.

The most common fine-tuning mistake: training on a small dataset that doesn’t reflect the production input distribution. The model overfits to your eval examples and underperforms on real traffic. Mitigate by sampling SFT data from production logs (with privacy controls) when possible.”

Tells that get you a strong-hire vote

  • You separate the three training objectives (continued pretraining, SFT, preference optimization) from parameter-efficient update methods.
  • You bring up DPO over RLHF for operational simplicity.
  • You discuss LoRA’s serving advantages (multi-tenant, memory).
  • You mention lock-in cost of fine-tuning.
  • You insist that data quality dominates data quantity.

Tells that get you down-leveled

  • Treating “fine-tuning” as one thing.
  • Recommending continued pretraining as a default.
  • No knowledge of LoRA / QLoRA.
  • No awareness of DPO.
  • Suggesting RLHF when DPO is sufficient (or vice versa, recommending DPO when the task needs on-policy training).

Common follow-up

“When would you use full fine-tuning over LoRA?”

The L6 answer:

“When the adaptation requires changing the model substantially: cross-language adaptation, a major behavioral shift, or when the LoRA quality gap is unacceptable for the use case. Also when serving infrastructure can’t support adapter-based serving (some inference stacks don’t have efficient LoRA support, though most do in 2026). For most fine-tuning tasks in 2026, LoRA is the default and full fine-tuning is the exception.”

Related: When would you fine-tune vs prompt vs RAG?, RLHF and DPO, Mixed precision training.

Found an error or stale claim? Report a correction or send a private note.