Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

core: Self-RAG adds four reflection tokens to the LM vocabulary — [Retrieve] / [IsRel] / [IsSup] / [IsUse] — so a single model decides on-demand whether to fetch passages and critiques its own segments before ranking.

The first pane walks the four reflection tokens and the 10-step inference pipeline. The second pane demos conventional RAG failing on the same prompt. The third pane demos Self-RAG's three-step retrieval, parallel critique, and ranked best segment. The fourth pane prints the main results, ablation, and human-evaluation tables. Traces are teaching schematics; numbers are paper-report.

primer — what you need before you start

the task. Self-RAG is a retrieval-augmented generation (RAG) system that can refuse to retrieve, judge whether retrieved passages are relevant, judge whether its own draft is supported, and rank its own candidate segments. All four decisions are emitted as next-token predictions.

how it was done before. Vanilla RAG retrieves top-k chunks and stuffs them into the prompt — no per-chunk judgment, no per-statement verification, no per-segment ranking. Hallucinations propagate. Self-RAG closes the loop by training the model to be its own retriever, critic, and ranker.

glossary — terms you will see.
retrieval-augmented generation (RAG) — LLM answer generation conditioned on a query to a vector index.
parametric knowledge — facts stored in the LM's weights, not in any external index.
reflection tokens — four special tokens ([Retrieve], [IsRel], [IsSup], [IsUse]) the model emits to mark its own reasoning.
critic — in this paper, the model itself; the four tokens replace an external judge model.
PopQA / ASQA — short-form QA and long-form QA benchmarks used in the paper.
support — whether a retrieved passage backs a specific claim in the draft answer.

symbol table — what the letters mean.
$x$ — the input question.
$D$ — set of retrieved passages (variable size, decided by [Retrieve]).
$y$ — the generated answer text, split into segments.
$y_t$ — the t-th segment of the answer.
$[\text{Retrieve}]$ — binary: yes / no / continue.
$[\text{IsRel}]$ — per-passage binary: relevant / irrelevant.
$[\text{IsSup}]$ — per-statement ternary: fully / partially / no support.
$[\text{IsUse}]$ — 5-point utility score (1–5).
$w_{\text{rel}}, w_{\text{sup}}, w_{\text{use}}$ — adjustable inference-time weights for each critique token.

tone legend — what the [brackets] mean.
[obs] observation   [crit] critique / reflection
[retrieve] retrieval decision   [use] usage / utility
[eq] equation   [take] key takeaway
[sys] system / meta   [limit] paper-reported limit

Four panes cover, in pedagogical order: the four reflection tokens and 10-step pipeline; conventional RAG failing; Self-RAG retrieval + critique + ranking; main results + ablation + human eval. Each pane ends with key takeaways so you can stop after any section and still know what you learned.

point: the LM emits reflection tokens inline with the next-token prediction, no extra model calls. At inference the algorithm runs through [Retrieve] → on-Yes: [IsRel] / [IsSup] / [IsUse] → weighted-rank; on-No: skip retrieval, still emit [IsUse].

[Retrieve] (on-demand fetch) [IsRel] (relevance) [IsSup] (support) [IsUse] (utility 1–5) pipeline step reflection tokens / inference

point: conventional RAG retrieves a fixed three passages for every query, feeds them to the LM, and never revises. For "How did US states get their names?" the three retrievals are heterogeneous in topic, the LM confabulates Christopher Columbus as a state namesake, and reports "No information in passage" despite having just answered.

Prompt, three retrieved chunk titles, the LM output, and the contradiction diagnosis. Retrieval is schematic; LM output is verbatim including the hallucinated Columbus claim.

retrieve (always) retrieved chunk LM output hallucination / contradiction

point: same prompt, but the model emits [Retrieve]=Yes, fetches three passages, runs [IsRel] + [IsSup] on each in parallel, then [IsUse] ranks. d₃ ("California's name has its origins in a 16th-century novel Las Sergas de Esplandián") wins — fully supported, utility 5 — and the best segment is the one grounded in that passage.

Prompt, three chunk titles, every reflection-token prediction per chunk, the ranking line, and the final best segment.

[Retrieve] = Yes [IsRel] (relevant / irrelevant) [IsSup] (full / partial / no support) [IsUse] 1–5 best segment

point: Self-RAG-7B / 13B beats ChatGPT and most retrieval-augmented baselines on PopQA, ARC-Challenge, PubHealth, ASQA-citation and biographies — but ChatGPT still leads on TriviaQA (74.3 vs 69.3) and PubHealth (75.3 vs 73.1), and ASQA citation precision (70.3) only crosses ChatGPT after tuning the [IsUse] weight at test time. Ablation shows the critic matters most (No Critic C → ASQA 18.1 vs full 32.1); retrieval on-demand beats always-retrieve-top1 (32.1 vs 28.6).

Self-RAG ChatGPT Toolformer / SAIL / Alpaca baselines ablation rows paper-report numbers

Stat cards are paper-report values, not computed from the printouts above.

point: paper-acknowledged limits. The training corpus needs GPT-4 to generate reflection tokens — not reproducible without the same API budget. At inference, segment-level beam search is slower than a single LM pass. ChatGPT still wins TriviaQA (74.3) and PubHealth (75.3). Human-eval agreement is 92.5 on PopQA but only 70.0 on Bio — long-form factuality judgment is contested.

paper-acknowledged limit ChatGPT still leads on some tasks GPT-4 dependency / inference cost