Retrieval-Augmented Generation for Large Language Models: A Survey

core: three RAG paradigms succeed each other — Naive (Retrieve-Read), Advanced (pre-retrieval + post-retrieval optimization), Modular (rewritable Search / Memory / Predict / Rerank modules and patterns like Rewrite-Retrieve-Read, DSP, ITER-RETGEN, FLARE, Self-RAG). Each paradigm fixes a subset of the previous one's three named drawbacks — none fix all three.

Three RAG paradigms succeed each other — Naive (Retrieve-Read) → Advanced (pre-retrieval + post-retrieval optimization) → Modular (rewritable Search / Memory / Predict / Rerank modules). The first pane demos the Sam Altman query with and without RAG. The second pane walks the three paradigm flows as ASCII diagrams. The third pane demos three pre-retrieval transformations on synthetic queries. The fourth pane shows the four augmentation loops. The fifth pane lists the three Naive RAG drawbacks plus the RAG vs PE vs FT quadrant.

primer — what you need before you start

the task. Gao et al.'s RAG survey organizes 1000+ retrieval-augmented generation papers into three paradigms. RAG = "use a vector index to inject external knowledge into an LLM's prompt" — the dominant production pattern for knowledge-grounded LLM apps.

how it was done before. Long context windows (200K+ tokens) looked like they would replace RAG. They didn't: latency, cost, and the attention-sink problem make real-time RAG still cheaper. The paper makes a taxonomy of how the field has solved the same three problems (precision, hallucination, redundancy) differently across paradigms.

glossary — terms you will see.
vector index — a database of text chunks plus their embeddings; query-by-similarity.
retrieval — query the index, get top-k chunks most similar to the question.
prompt — concatenate question + chunks; ask the LLM to answer using the chunks.
hallucination — the LLM generates a fact not supported by any retrieved chunk.
pre-retrieval optimization — rewrite / expand / disambiguate the question before the index call.
post-retrieval optimization — re-rank, compress, filter the retrieved chunks before generation.
Modular RAG — replaceable modules: Search, Memory, Predict, Rerank, Read, etc.

symbol table — what the letters mean.
$q$ — user query.
$D = \{d_1, d_2, ..., d_K\}$ — top-K retrieved documents.
$\text{RAG}(q)$ — the whole pipeline: retrieve + read.
$w$ — chunking window; how much text per indexed chunk.
$\text{Rewrite}(q)$ — pre-retrieval transformation.
$\text{Rerank}(D, q)$ — post-retrieval re-scoring of $D$ by relevance to $q$.
$\text{FLARE}$ — Forward-Looking Active REtrieval Augmented generation; gates retrieval by token-level confidence.

tone legend — what the [brackets] mean.
[obs] observation   [modular] modular component
[naive] naive RAG   [advanced] advanced RAG
[eq] equation   [take] key takeaway
[sys] system / meta   [limit] paper-reported limit

Six panes cover, in pedagogical order: a Sam Altman example showing RAG wins and loses; the three paradigm flows; pre-retrieval transformations; the four augmentation loops; the three Naive RAG drawbacks + the RAG / PE / FT quadrant. Each pane ends with key takeaways so you can stop after any section and still know what you learned.

point: a user asks about Sam Altman's sudden dismissal and reinstatement. Naive RAG retrieves three news-article chunks on the event. Without RAG the model punts on the event ("unable to provide comments on future events"); with RAG the model grounds the answer in the chunks ("significant internal disagreements within OpenAI"). The same query demonstrates Naive RAG's failure mode and Advanced RAG's fix.

Query, three chunk titles, "without RAG" output, and "with RAG" output. Retrieval pipeline is schematic.

retrieve retrieved chunk LM output drawback / loss Naive RAG Advanced RAG (with chunks)

point: Naive RAG is a linear Retrieve-Read chain. Advanced RAG adds pre-retrieval (chunking, metadata, query rewrite, HyDE) and post-retrieval (re-rank, context compression) but stays chain-like. Modular RAG inherits both but breaks the fixed sequence into replaceable modules and patterns (Rewrite-Retrieve-Read, Generate-Read, Recite-Read, DSP, ITER-RETGEN, FLARE, Self-RAG).

Three ASCII flow diagrams printed sequentially — each paradigm's topology is the structural claim, the printed block IS the diagram.

Naive RAG (Retrieve-Read chain) Advanced RAG (pre + post) Modular RAG (replaceable modules) retrieve read generate

point: paper §III.C lists three pre-retrieval query-transformation techniques — Query Rewrite (RRR) rewrites short queries into long ones to broaden recall; HyDE has the LLM generate a hypothetical answer and uses that as the retrieval query; Step-back Prompting steps back to a higher-level abstraction. Each before/after demo shows the same query at increasing Recall@K (schematic).

Synthetic queries — these are paper-defined techniques, not §1's verbatim chunks. The ASCII shows the transformation shape, not the actual retrieval scores.

transformation step Recall@K delta (schematic) retrieval call query

point: paper §V categorizes retrieval-augmentation into three loop shapes (Iterative, Recursive, Adaptive), each fixing a different Naive-RAG failure mode. Naive (once) retrieves a single pass; Iterative (ITER-RETGEN) regenerates after each retrieve until the answer is complete; Recursive decomposes the query into sub-questions and retrieves each; Adaptive (FLARE / Self-RAG) gates retrieval on token-level confidence and emits reflection tokens ([Retrieve] / [IsRel] / [IsSup] / [IsUse]).

Four loop diagrams printed sequentially. The topology is the claim — only the labels change between panels.

Naive (once) Iterative (ITER-RETGEN) · Recursive Adaptive (FLARE / Self-RAG) retrieve read generate / decompose / gate

point: Naive RAG fails on three named axes — Retrieval Challenges (precision / recall miss relevant chunks), Generation Difficulties (hallucination beyond retrieved docs), Augmentation Hurdles (redundancy, disjoint output, over-reliance on retrieved context). Advanced RAG fixes parts of the first and third; Modular RAG absorbs the rest, including fine-tuning at the high-adaptation end of the Figure 4 quadrant.

drawback Advanced RAG fix Modular RAG fix still open

A survey does not give quantitative head-to-head numbers — it gives the loss-comparison: which paradigm fixes which drawback, and where each still leaves the original problem standing.