RAG: retrieval-augmented generation

Summary

Retrieval-augmented generation supplies external evidence as model context without requiring that evidence to be stored in model weights. Success requires authorized evidence to survive retrieval and selection, then support the actual response claims. A resolving citation alone proves neither support nor truth.

The architecture

Offline, parse documents into chunks while retaining source IDs, versions, boundaries, and access metadata. Build the chosen indexes and update or delete entries when their sources change. Keep each plan name and retention value together; preserve heading or neighbor references when splitting would separate them.

Online, apply the requester’s access policy, retrieve a bounded candidate set, select bounded context, and generate claim-linked citations. The pictured hybrid retrieval and reranking are possible choices, not mandatory stages for every corpus or latency budget. Query rewriting can also change intent, so evaluate it rather than assuming improvement.

For a hypothetical fully authorized corpus:

Ask about plan B, with candidate limit two and context limit one. Retrieval returns P1/P2; selection retains P2 with its version and an allowed access decision. The answer “thirty days” linked to P2 follows the evidence path. A seven-day answer citing P2 resolves to an existing passage but fails support. If P2 never enters the candidate set, merely reranking P1 cannot recover it.

Change Matching Needs

New task: Retrieve both exact error identifiers and paraphrased descriptions.

Picture and change: Compare lexical and dense candidate sets, then fuse deduplicated document identities with their provenance. Tune candidates, fusion, and limits on development queries, not the final test set.

Check: A lexical ERR-42 match remains eligible rather than being vetoed solely by a low dense score. Inspect whether subsequent candidate/context cutoffs retain its supporting passage.

Use when: Query slices need complementary matching signals, rather than a universally mandated hybrid pipeline.

Change Evidence Availability

New task: The authorized corpus lacks the answer.

Picture and change: Return the defined no-evidence or partial-answer outcome instead of inventing a supporting passage.

Check: Neither P1 nor P2 supports a claim about plan C.

Use when: Coverage or authorization prevents a grounded answer.

Measure retrieval coverage at the actual candidate limit separately from context support, answer correctness, citation resolution, and abstention behavior. Use judged query slices and report denominators; first-hit utility and relevant-passage coverage are distinct targets. Time and cost budgets must include retrieval, selection, generation, and any verification actually used.

Revoking access or deleting a document must invalidate old index entries and context caches before serving its text again. Preserve source/version IDs through citations and recheck request authorization; a cached passage is not a permanent access grant. Retrieved instructions are untrusted data, not policy.