Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models

core: LATS adapts Monte Carlo Tree Search to language agents, replacing the world model that classical MCTS requires with environment feedback, and using the same pretrained LM as both the agent and the value function — making search tractable in interactive tasks where you can revert to any prior context by simply re-feeding the LM its earlier context.

Six panes follow: primer; the five-axis framework overview (reasoning × acting × planning × self-reflection × external memory); the six MCTS operations with UCT selection; the value function (LM score + self-consistency) and a HotPotQA trace evolution; main results across HotPotQA, HumanEval, and WebShop; and ablation plus the paper's own limits. Traces are schematic; numbers marked paper-report come from the paper tables.

primer — what you need before you start

the task. A language agent is given an input (a question, a programming spec, a shopping instruction) and must produce a trajectory of language actions that ends in a correct answer. The simplest agents (ReAct) emit one thought + one action at a time, autoregressively. That is fast, but it commits to the first plausible step and can fail on multi-step problems where the right path requires trying a few alternatives before committing. LATS frames the same loop as Monte Carlo Tree Search: keep a tree of candidate trajectories, sample several actions at each step, score each candidate with a learned (here, LM-evaluated) value function, and keep searching until a budget runs out or a correct trajectory is found.

how it was done before. Three families. Reasoning-only (CoT, ToT, RAP) search over internal thoughts but never see environment feedback, so they hallucinate on tasks where the answer requires retrieval or tool use. Acting-only (ReAct, Reflexion) interleave thoughts with environment actions but commit to one trajectory — no tree search, no comparing alternatives. Search-with-acting is what LATS adds: take an MCTS-style search loop, drop the world-model requirement (LM tasks let you reset to any prior context), and use real environment observations as the rollout signal.

glossary — terms you will see.
agent — an LM policy that emits thoughts (language) and actions (tool calls / API calls) given a state.
state — the LM context so far: the original input plus the accumulated action-observation history.
thought — a piece of language the agent emits to reason about what to do next; it never directly affects the environment.
action — a call to the environment (search API, code execution, web click) that returns an observation.
observation — environment feedback returned after an action; it becomes part of the state.
MCTS — Monte Carlo Tree Search: a heuristic search algorithm that builds a decision tree by repeated selection, expansion, simulation, and backpropagation steps.
UCT — Upper Confidence bounds applied to Trees: the selection rule that balances exploitation (high value) and exploration (rarely visited children).
selection — walk down the tree from root to leaf by repeatedly picking the child with highest UCT.
expansion — at a leaf, sample several new actions and add their resulting states as children.
evaluation — assign each new child a scalar value (how promising this state looks).
simulation — from the new state, walk further (often greedily) until reaching a terminal state.
backpropagation — take the terminal reward and update the value and visit count of every state along the rollout path.
reflection — on a failed terminal, prompt the LM to summarise what went wrong; store it and inject it into the next rollout's context.
self-consistency — agreement across multiple samples at the same state; more agreement tends to mean more reliable.
value function — the function V(s) that scores how promising a state is; in LATS this is λ · LM(s) + (1-λ) · SC(s).
state reversion — the property that you can reset to any prior state by re-feeding the earlier context; this is what lets LATS drop the world model classical MCTS requires.
rollout budget — the maximum number of trajectories K the search may explore before stopping.
benchmarks in this paper — HotPotQA (multi-hop QA over Wikipedia), HumanEval / MBPP (programming from docstrings), WebShop (web shopping over 1.18M products), Game of 24 (math).

symbol table — what the letters mean.
$s_t$ — state at depth $t$; the LM context so far.
$a_t, o_t$ — action and observation at depth $t$.
$V(s)$ — value of state $s$; how promising this state looks.
$N(s)$ — visit count: how many rollouts have passed through state $s$.
$N(p)$ — visit count of state $s$'s parent.
$w$ — exploration weight in UCT (paper default 1).
$\lambda$ — weight blending LM score and self-consistency in V(s).
$n$ — number of children sampled at each expansion (paper default 5).
$K$ — total rollout budget for the search (HotPotQA 50, WebShop 30, HumanEval 8).
$L$ — max depth per rollout (HotPotQA 7, WebShop 15).
$r$ — terminal reward from the environment (success / failure / partial).
$p_\theta$ — the pretrained LM, used as the action generator.
$p_V$ — the same LM prompted to score a state; gives the LM(s) part of V(s).
$p_{\text{ref}}$ — the same LM prompted to reflect on a failed trajectory.
$c_t$ — the running context (the model input built from past observations and actions).

the UCT selection rule (paper). $\text{UCT}(s) = V(s) + w \cdot \sqrt{\frac{\ln N(p)}{N(s)}}$. Exploitation is the first term (high-value children get picked again); exploration grows logarithmically with the parent's visits and shrinks with this child's visits, so under-visited children get a temporary boost.

the value function (paper). $V(s) = \lambda \cdot \text{LM}(s) + (1 - \lambda) \cdot \text{SC}(s)$. $\text{LM}(s)$ is the LM prompted to reason about state $s$ and end with a score; $\text{SC}(s)$ is the agreement across multiple samples at that state. The paper defaults $\lambda = 0.5$ on HotPotQA and Game of 24, $\lambda = 0.8$ on programming and WebShop.

tone legend — what the [brackets] mean.
[obs] observation   [arch] paper claim / design property
[mcts] tree-search operation   [val] value-function signal
[res] paper-reported result row   [abl] ablation row / lost comparison
[limit] paper-acknowledged limit   [take] key takeaway
[sys] system / meta   [pan] ASCII tree pane label

Six panes cover, in pedagogical order: framework overview (which five axes LATS is the first to cover); the six MCTS operations with the UCT rule; the value function and a HotPotQA trace; main results across three domains; and ablation plus the paper's own limits. Each pane ends with key takeaways so you can stop after any section and still know what you learned.

point: across seven prior prompting frameworks (CoT, ReAct, ToT, RAP, Self-Refine, Beam Search, Reflexion), LATS is the only one that covers all five design axes at once: internal reasoning, environment acting, search-based planning, verbal self-reflection, and external memory of past attempts. CoT covers reasoning only; ReAct covers reasoning and acting; ToT and RAP add planning and memory but no acting; Reflexion adds self-reflection and memory but no planning; Self-Refine and Beam Search cover reasoning and self-reflection only. The combination is what lets LATS be applied without modification to programming (test-driven), QA (search-driven), web navigation (click-driven), and math (no-environment). All numbers below are paper-reported.

design property ASCII tree pane label key takeaway (brackets)

point: LATS is MCTS with six operations per iteration. selection walks from the root to a leaf by picking the child with the highest UCT at each level; UCT balances exploitation (the value term) against exploration (the visit-count term). expansion samples n=5 new actions from the LM at that leaf and adds the resulting states as children. evaluation scores each new state with V(s), a blend of an LM self-score and a self-consistency score. simulation walks further from each new state until reaching a terminal or hitting the depth limit, gathering a reward. backpropagation updates N(s) and V(s) along the rollout path using the terminal reward. reflection only on failure: the LM is asked to summarise what went wrong, and that reflection is stored in memory and injected into the next rollout's context. The whole loop runs K times or until success.

tree-search operation design property key takeaway (brackets)

point: V(s) is the blend of two signals about state s. The LM score is a scalar that the same pretrained LM produces when prompted to reason about how promising s is. The self-consistency score measures agreement among the n actions that were sampled at s — if the same state keeps producing the same actions, those actions are likely good. λ trades them: λ=0.5 is the HotPotQA default, λ=0.8 is the programming and WebShop default, λ=1.0 reduces to LM only (no consistency term). On a schematic HotPotQA multi-hop trace, the first rollout takes a wrong turn (searches the wrong entity), gets a failure reward, and triggers reflection; the reflection is written into memory and injected into the next rollout's context; the second rollout uses the reflection to plan a better search and succeeds. The empirical gain from reflection on top of LATS is only +0.05 EM on HotPotQA — much smaller than Reflexion's +0.19 over ReAct, because search already covers many trajectories reflection would otherwise need to retry.

value-function signal tree-search operation design property key takeaway (brackets)

point: across HotPotQA acting (GPT-3.5, EM), HumanEval programming (GPT-3.5 and GPT-4, pass@1), and WebShop web navigation (GPT-3.5, average score and success rate), LATS wins on every benchmark. HotPotQA: ReAct 0.32, ToT-ReAct 0.41, RAP-ReAct 0.38, Reflexion 0.51, LATS 0.65, LATS+CoT-SC 0.71. HumanEval GPT-3.5: CoT 46.9, ReAct 56.9, ToT 54.4, RAP 63.1, Reflexion 68.1, LATS 83.8. HumanEval GPT-4: base 80.1, Reflexion 91.0, LATS 92.7 (state of the art at the time). WebShop: ReAct 53.8 score / 27.0% SR, ReAct best-of-30 56.5 / 30.0, Reflexion 64.2 / 38.0, LATS 75.9 / 48.0, human 82.0 / 70.7. WebShop LATS at 75.9 is comparable to gradient-based RL baselines like BCP at 63.5 — gradient-free, no fine-tuning. Lost comparisons: CoT alone on HotPotQA (0.34) beats ReAct (0.32), RAP-CoT (0.55) beats RAP-ReAct (0.38) — simple integration of search with acting actually loses to reasoning-only search; ToT-ReAct (0.41) likewise loses to ToT-CoT (0.55). On WebShop LATS still trails human performance by 6+ score points. All numbers are paper-reported, averaged across runs.

paper-reported result lost comparison / anti-result key takeaway (brackets)

point: on HotPotQA (ReAct base, n=5, k=50) the ablation sheet tells a sharp story. Dropping self-reflection costs 0.05 EM. Dropping the LM value function entirely costs 0.26 EM — by far the biggest single switch — because without it the only signal is terminal rewards, which are sparse and binary. Replacing MCTS with DFS costs 0.21 EM; selection and backpropagation are real contributions, not bookkeeping. Adding self-consistency on top of LATS only adds +0.01 EM, suggesting the LM score already captures most of the consistency signal. Two paper-acknowledged limitations follow. Higher compute cost: with k=50 trajectories and n=5 children per expansion, plus reflection rollouts, LATS is one to two orders of magnitude more expensive than ReAct or Reflexion — limiting practicality under tight budgets. State-reversion assumption: LATS assumes you can reset to any prior state by re-feeding the earlier context, which holds for text-based tasks but not for environments with side effects (real money, real network state). The paper's WebShop section adds a third caveat — on WebShop "generated reflections are often generic and do not provide useful feedback, resulting in a tendency for the agent to become stuck in local minima" — explaining why reflection gains are smaller there.

ablation row paper-acknowledged limit key takeaway (brackets)