Yao et al. study a split in how language models are used. Chain-of-thought reasons inside a closed context; acting systems emit API calls or environment steps with no explicit plan. ReAct generates both on one trajectory. A Thought stays in language and does not change the world; an Action queries Wikipedia or steps through a text environment and returns an Observation that conditions later thoughts.
The paper does not claim a uniform win. On knowledge tasks, ReAct and CoT are complementary: solo ReAct beats CoT on FEVER and trails it on HotpotQA, and the best numbers come from backing off one into the other. On decision tasks, one or two in-context traces outperform imitation and reinforcement learning trained on thousands to hundreds of thousands of demonstrations. The session below is a teaching schematic, not a paper figure. The argument and the tables are in the paper; this site does not host the PDF.
teaching explainer · Yao et al. · ICLR 2023 · PaLM-540B few-shot · arXiv:2210.03629 · project | traces are schematic session logs; numbers marked paper-report come from the tables
0overview
core: Reasoning and acting are one generation loop: the model thinks in language, acts on the world, and lets observations revise the next thought.
Reason + Act
Alternate reasoning traces and task actions on the same trajectory, instead of thinking the answer through, or emitting actions only.
query outside
Actions let the model query Wikipedia or step through a text environment, then write observations back into context.
split the trace
Internal knowledge and freshly read sentences can be checked apart; a thought can be edited to pull a bad plan back.
The paper has thought editing (a human may rewrite a thought). This page does not demo that.
1problem: reasoning and acting were studied apart
point: CoT-only is a closed box — facts can hallucinate, errors propagate. Act-only can query the world but has no high-level plan, so a failed search is not rewritten. Below: one teaching multi-hop question, four prompting styles. Schematic, not Figure 1.
schematic teaching trace, not the paper figure or original question.
Q (schematic, not from the paper) · The physicist who proposed general relativity was born in a city in which country?
Standard CoT only Act only ReAct
2mechanism: fold language space into the action space *
point: at time the agent sees and samples the next step. ReAct enlarges the action space to : a Thought in does not change the environment and has no Observation; an environment action in returns . A thought updates context to .
Thought (no env change) Action Observation context
3knowledge tasks: ground with a Wikipedia API
point: HotpotQA (multi-hop QA) and FEVER (SUPPORTS / REFUTES / NOT ENOUGH INFO) are question-only. The model gets no gold paragraphs. It can use three action types (search / lookup / finish) for many interleaved steps. HotpotQA heuristic: back off after about 7 steps without an answer. The API is weak on purpose, to force explicit language reasoning. Knowledge-task Thought-Action-Obs is dense.
retrieval sentences are schematic, not Wikipedia reprints.
search lookup finish obs / page
4decision tasks: sparse Thought over long action chains
point: in household text games like ALFWorld, expert traces can exceed 50 steps. Thoughts appear only at forks: decompose, track progress, guess a location, handle a closed drawer. Act-only keeps taking from an empty cabinet. WebShop is a noisy real product page.
ALFWorld is a text game; the pane is a text Obs log. Structure borrowed: plan, look in the wrong place, obs empty, revise, put. Not a paper figure.
sparse Thought Act stuck ReAct revises env obs
5results and limits
point: on knowledge tasks, solo ReAct does not uniformly beat CoT: FEVER Acc 60.9 > CoT 56.3, HotpotQA EM 27.4 < CoT 29.4; CoT-SC is 33.4 / 60.4. Opposite backoffs win: ReAct→CoT-SC 35.1 on HotpotQA, CoT-SC→ReAct 64.6 on FEVER. On decision tasks, 1-2 shots beat IL/RL traces. Tables print paper-report numbers, not a digitized figure.
ReAct CoT / other prompting Act / IL paper-report numbers
71%
ALFWorld · ReAct best of 6
40.0 vs 30.1
WebShop SR: ReAct vs Act (paper: ~+10 abs. pp vs prior best)
56%
share of CoT failure modes that are hallucination
3k
finetune traces: small model can beat large-model prompting
The four cards are paper-report results, not computed from the printout above. Figure 3 point values are not invented here; only this qualitative line is kept.
Schematic traces are not paper figures. Paper-report numbers: Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, ICLR 2023, arXiv:2210.03629. This site does not host the PDF.