Summary
Locate the changing knowledge source before choosing acoustic retraining. Text, pronunciation, acoustics and endpointing need different update evidence.
Problem
Product names change weekly; the team has text and pronunciations, but little new audio. For two hypothetical transcript candidates, use fixed acoustic and external-LM log scores:
| Candidate | Acoustic score | LM score |
|---|---|---|
| Rare name | -5 | -1 |
| Common word | -2 | -4 |
Decode with . Can text-side preferences change the winner without changing acoustic weights?
Picture: Find Where New Knowledge Enters
Learning objective
A separate lexicon can change without retraining the acoustic core
Derivation
Which component remains fixed in this modular adaptation?
Calculate the Changed Preference
| Fusion weight | Rare name | Common word | Selected candidate |
|---|---|---|---|
| Common | |||
| Rare |
The acoustic evidence and parameters stayed fixed. Only the scoring preference changed; this is not measured accuracy improvement or proof the rare name was spoken. If that candidate never enters a fixed n-best list, rescoring cannot select it.
Decision: Version the Whole Update
Specify acoustic/token units, compatible phone/token inventory, lexicon, LM score convention, fusion weight, search and artifact versions. Graph rebuilding, publication, regression tests and compatible rollback can remain necessary when acoustic weights do not change. Compare rare-term recall, false insertions, WER slices, delay, maintenance and update time under matched data and workloads.
Change 1: Only Text and Pronunciations Are Available
New task. Add domain terms without paired-audio retraining data.
Picture and change. Test compatible lexicon, LM or contextual-bias updates with candidate-support and negative checks.
Check. New-term recall must not hide unrelated false insertions.
Use when. Vocabulary changes faster than labeled audio.
Change 2: The Microphone Changed
New task. Errors follow channel conditions, not new vocabulary.
Picture and change. Inspect front-end/acoustic coverage and collect representative adaptation and evaluation audio.
Check. A stronger term bias cannot restore missing acoustic information.
Use when. Failure tracks noise, channel or device.
Failure: Fixed Acoustic Weights Are Called a Free Update
An incompatible graph, missing candidate or untested bias can break delivery despite unchanged acoustic weights.
Transfer Check
Is an end-to-end acoustic core with external LM rescoring a contradiction?
Reasoned Answer
No. Joint core training and modular inference controls describe different boundaries.
Hybrid ASR
Acoustic scores combine with HMM topology, pronunciation lexicon and LM, often through WFST decoding. Explicit modules enable local diagnosis/control but cost alignment, graph and component maintenance. Separate objectives do not automatically optimize final transcript quality jointly.
End-to-end ASR
CTC, RNN-T and attention objectives learn audio-to-token mappings with different alignment/history assumptions. Joint representations can reduce some boundaries; external fusion, biasing, endpointing, confidence and safety controls can remain. Data needs, multilingual quality, hallucination and diagnosis depend on the actual system, not its label.
In an interview
- Name whether acoustics, pronunciation, frequency or endpoint policy changes.
- Locate the update in both decompositions.
- Check labeled audio, domain text, control, languages and existing infrastructure.
- Validate streaming, rare terms, false biasing and update rollback.
- Compare matched beyond-WER outcomes and operating cost.
Common confusions
Neither “hybrid always needs less data” nor “end-to-end has no decoder” is a reliable selection rule. Accuracy and adaptation speed require workload-specific evidence.
Sources
Related: CTC, RNN-T, encoder-decoder architectures, and streaming ASR.