ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

core: DFSDT lets an LLM expand reasoning on RapidAPI tool calls: a depth-first decision tree where a wrong API can be aborted via Finish by Giving Up and a sibling node tried next.

Three reasoning strategies with the same instruction, side by side. The DFSDT pane (right) walks a depth-first decision tree; the ReACT pane (middle) commits to one API per step and cannot recover from a bad call; the CoT pane (left) hallucinates without any API at all. The trace below is a teaching schematic.

primer — what you need before you start

the task. ToolLLM targets tool-using agents. Given a natural-language instruction that requires external information (flights, weather, recipes, …), the model must figure out which API to call, what parameters to pass, how to chain multiple calls, and how to interpret the results.

how it was done before. ReAct-style agents commit to one API call per step and backtrack only by remembering the past. If the first call was wrong, the rest of the chain suffers. ToolLLM makes backtracking explicit: when a call fails, the agent can `Finish by Giving Up` and try a sibling.

glossary — terms you will see.
API — application programming interface; here, a remote HTTP service the agent can call.
ToolBench — the paper's instruction-tuning dataset; 16k+ real RapidAPI endpoints.
DFSDT — Depth-First Search Decision Tree; the paper's core algorithm.
Finish with Final Answer — successful terminal; the agent commits to an answer.
Finish by Giving Up — unsuccessful terminal; the agent abandons a node and tries a sibling.
RapidAPI — a public marketplace of real APIs; ToolBench queries these.
ToolEval — the paper's automatic evaluator; two metrics: pass rate + win rate.

symbol table — what the letters mean.
$T$ — the search tree; nodes are (reasoning, action, observation) tuples.
$d_{\max}$ — depth limit (how deep DFS can go before giving up).
$w_{\max}$ — width limit (how many siblings to try at any node).
$\text{Finish}_{\text{succ}}$ — successful terminal action.
$\text{Finish}_{\text{giveup}}$ — give-up terminal action.
$\text{Trajectory}$ — the path from root to terminal.

tone legend — what the [brackets] mean.
[action] API call step   [obs] API response
[tree] tree reasoning structure   [final] terminal action
[eq] equation   [take] key takeaway
[sys] system / meta   [limit] paper-reported limit

Five panes cover, in pedagogical order: the four action types + terminal conditions; a single-pane trace; the full DFSDT tree; a RapidAPI worked example; results vs ReAct vs CoT. Each pane ends with key takeaways so you can stop after any section and still know what you learned.

point: same RapidAPI instruction across three reasoning strategies. CoT does not call any API; ReACT commits to one API per step and cannot backtrack once the first call returns an error; DFSDT expands a tree, aborts via Finish by Giving Up, and finds a sibling API that works. The ReACT trace is a teaching schematic of Normal → Normal → Error → Error → Fail against DFSDT's tree.

DFSDT pane Round 1 (Thought + API call + Observation) is the literal instruction trace; middle rounds are compressed to x N. CoT and ReACT panes are schematic — the same instruction, three strategies, no per-pane trace.

CoT (no API) ReACT chain DFSDT tree node API call API response Finish with Final Answer Finish by Giving Up FAIL

point: DFSDT expands a reasoning tree pre-order. At each node the model emits : either an API call (advance) or one of two terminal actions — Finish with Final Answer or Finish by Giving Up. Giving Up abandons the current branch so the algorithm can try a sibling. The expected cost is in OpenAI API calls. When the model does not retract any action, DFSDT degrades to ReACT.

DFS expansion API call (advance) API response Finish by Giving Up Finish with Final Answer

point: a friend wants a birthday surprise for an admirer of actress Hailee Steinfeld. DFSDT enters the tree at the root, calls get_extra_character_details to retrieve Hailee's age and recent movies, and after several rounds the algorithm reaches one of two terminals: Finish with Final Answer (return the suggested gift) or Finish by Giving Up (abandon this branch, try a sibling API).

user instruction, Round 1 Thought / API Name / Arguments / Observation, the final-answer Arguments text, and the Finish by Giving Up action name. Middle rounds are marked x N — no expanded middle steps were given.

Thought API call (RapidAPI) Observation (API response) Finish with Final Answer Finish by Giving Up

point: ToolLLaMA + DFSDT-Retriever reaches 70.0 average pass rate across four generalization splits, beating ChatGPT-DFSDT (64.8) but trailing GPT-4-DFSDT (71.0). On out-of-distribution APIBench, the same model posts 16.77 / 51.16 / 40.59 AST accuracy on HuggingFace / TorchHub / TensorHub — beating Gorilla-RS-BM25 on HuggingFace (16.77 vs. 15.71) and TorchHub (51.16 vs. 50.00). But Vicuna and Alpaca fine-tuned on ToolBench score 0.0 pass rate on every split.

ReACT DFSDT ChatGPT GPT-4 paper-report numbers

The four cards are paper-report values, not computed from the printout above. Win-rate ties are split into win + lose in the tie-break footnote; most splits of DFSDT vs. ChatGPT-ReACT show win < 50% on those rows — ToolLLaMA is competitive but not dominant.

point: paper-acknowledged limits. The tree search costs OpenAI API calls per task. When the model does not retract any action, DFSDT degrades to ReACT. Solution path annotation itself can hallucinate API names — ChatGPT generates the trace, the pipeline filters some, the rest is training data. On the tie-split win rate, most DFSDT vs. ChatGPT-ReACT splits drop below 50%.

paper-acknowledged limit DFSDT → ReACT (Appendix A.4) Tie-split win rate < 50%