ReAct: Synergizing Reasoning and Acting in Language Models

core: Reasoning and acting are one generation loop: the model thinks in language, acts on the world, and lets observations revise the next thought.

primer — what you need before you start

the task. ReAct targets tasks where a language model needs to look things up and act in the world — answering multi-hop questions (HotpotQA), verifying claims (FEVER), or navigating text-based environments (ALFWorld). The model writes a stream of text; sometimes that text is a thought, sometimes it's an action that hits an external API.

how it was done before. Two camps studied this separately: chain-of-thought prompting (let the model "think out loud" then answer) was strong on knowledge but hallucinated facts; pure action-emitting agents were grounded but couldn't reason across multiple steps. ReAct fuses the two — thought steers what to search, search returns observations that revise the next thought.

glossary — terms you will see.
agent — the language model that decides what to do next, one token at a time.
environment — the outside world the agent acts on (Wikipedia API, ALFWorld text game, etc.).
thought — text the model writes to itself; never reaches the environment. Used for planning, decomposition, synthesis.
action — text the model emits that triggers something in the environment: a search query, a click, a "finish" call.
observation — text the environment prints back into the context (search result snippet, ALFWorld reply, etc.).
trajectory — the whole sequence of (thought, action, observation) tuples for one task. The paper's unit of analysis.
few-shot — paper uses 1-6 worked examples in the prompt, no fine-tuning. This is "prompting", not training.
HotpotQA — multi-hop question answering benchmark (paper §3).
FEVER — fact verification: SUPPORTS / REFUTES / NOT ENOUGH INFO (paper §3).
ALFWorld — text adventure game benchmark (paper §4).

symbol table — what the letters mean.
$c_t$ — the context (everything written into the prompt so far).
$\hat{a}_t$ — action the agent decides at step t.
$o_t$ — observation the environment returns at step t.
$c_{t+1}=(c_t,\hat{a}_t,o_{t+1})$ — the recursion: new context = old context + new action + new observation.
$c_{t+1}=(c_t,\hat{a}_t)$ — when the agent emits a thought (no observation comes back), context updates without $o$.
$\mathcal{A}$ — the action space (search, lookup, finish, etc.).
$\mathcal{L}$ — the language space (anything the model can write).
$\hat{\mathcal{A}}=\mathcal{A}\cup\mathcal{L}$ — union of both; the agent's actual output space includes both actions and thoughts.

tone legend — what the [brackets] mean.
[obs] observation   [arch] architecture / paper claim
[thought] the agent's inner monologue   [action] external step the agent takes
[react] the ReAct interleaved step   [form] formal derivation
[take] key takeaway   [sys] system / meta   [warn] caveat

Six panes cover, in pedagogical order: the problem (CoT hallucinates, Act-only can't reason); the ReAct mechanism (thought-action-observation loop); what knowledge is required (HotpotQA, FEVER); what decision tasks look like (ALFWorld, WebShop); results (ReAct vs CoT head-to-head); and the limit (prompting-only, no thought editing here). Each pane ends with key takeaways so you can stop after any section and still know what you learned.

point: CoT-only is a closed box — facts can hallucinate, errors propagate. Act-only can query the world but has no high-level plan, so a failed search is not rewritten. Below: one teaching multi-hop question, four prompting styles. Schematic — not a paper figure reproduction.

schematic teaching trace, not the actual case.

Standard CoT only Act only ReAct

point: at time the agent sees and samples the next step. ReAct enlarges the action space to : a Thought in does not change the environment and has no Observation; an environment action in returns . A thought updates context to .

Thought (no env change) Action Observation context

point: HotpotQA (multi-hop QA) and FEVER (SUPPORTS / REFUTES / NOT ENOUGH INFO) are question-only. The model gets no gold paragraphs. It can use three action types (search / lookup / finish) for many interleaved steps. HotpotQA heuristic: back off after about 7 steps without an answer. The API is weak on purpose, to force explicit language reasoning. Knowledge-task Thought-Action-Obs is dense.

retrieval sentences are schematic, not Wikipedia reprints.

search lookup finish obs / page

point: in household text games like ALFWorld, expert traces can exceed 50 steps. Thoughts appear only at forks: decompose, track progress, guess a location, handle a closed drawer. Act-only keeps taking from an empty cabinet. WebShop is a noisy real product page.

ALFWorld is a text game; the pane is a text Obs log. Structure borrowed: plan, look in the wrong place, obs empty, revise, put.

sparse Thought Act stuck ReAct revises env obs

point: on knowledge tasks, solo ReAct does not uniformly beat CoT: FEVER Acc 60.9 > CoT 56.3, HotpotQA EM 27.4 < CoT 29.4; CoT-SC is 33.4 / 60.4. Opposite backoffs win: ReAct→CoT-SC 35.1 on HotpotQA, CoT-SC→ReAct 64.6 on FEVER. On decision tasks, 1-2 shots beat IL/RL traces. Tables print paper-report numbers, not a digitized figure.

ReAct CoT / other prompting Act / IL paper-report numbers

The four cards are paper-report results, not computed from the printout above. Point values from the paper's scatter plot are not reproduced here; only this qualitative line is kept.