DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines

core: DSPy replaces hand-crafted prompt strings with three abstractions — signatures are natural-language typed declarations (question -> answer), modules are parameterized LM calls built on signatures (Predict, ChainOfThought, ReAct), and teleprompters are optimizers that bootstrap demonstration examples from a training set. A compiler then optimizes every Predict in the program jointly.

The first pane demos the signature syntax with the Guarani example. The second pane walks the ChainOfThought class and the built-in module tree. The third pane walks the compiler's three stages. The fourth pane prints GSM8K main results. The fifth pane prints HotpotQA multi-hop results and the paper-acknowledged limits. Traces are teaching schematics; numbers are paper-report.

primer — what you need before you start

the task. DSPy targets the engineering of LM programs. Instead of writing fragile prompt strings, you declare typed signatures and let the compiler figure out the right prompts and demos for your metric.

how it was done before. Two prior recipes: (1) hand-tune one prompt at a time, with no feedback loop; (2) DSP-style programming languages that need a programmer to update the prompt string manually. DSPy closes the loop with optimizers that automatically find prompts and demos that maximize a metric.

glossary — terms you will see.
signature — typed input → typed output, with a docstring.
module — a callable that uses signatures internally; forward() is the PyTorch-style entry point.
Predict — the basic signature call: zero-shot prompt, parse output.
ChainOfThought — Predict + "Let me think step by step" prefix + a reasoning field.
ReAct — Predict with a tools=[] argument, multi-step.
teleprompter — optimizer that compiles a program for a metric.
BootstrapFewShot — teleprompter that rejection-samples traces, keeps the good ones.
metric — a function that scores an output; the optimizer maximizes it.

symbol table — what the letters mean.
$M$ — the metric; the objective the compiler maximizes.
$D$ — the training set of (input, expected_output) examples.
$\theta$ — the parameters DSPy tunes: which demos go in which prompt.
$\pi_\theta$ — the resulting program; after compilation, behaves consistently on the metric.
$\text{Trace}(x)$ — a record of (input, prompt, output) for a single forward pass.

tone legend — what the [brackets] mean.
[obs] observation   [mod] module behavior
[sig] signature   [tp] teleprompter / optimizer
[eq] equation   [take] key takeaway
[sys] system / meta   [limit] paper-reported limit

Five panes cover, in pedagogical order: signature syntax with a concrete example; module tree (Predict, ChainOfThought, ReAct); the three-stage compiler; GSM8K results; HotpotQA results and the paper's own limits. Each pane ends with key takeaways so you can stop after any section and still know what you learned.

point: a signature is a typed natural-language declaration of input/output fields. The string question -> answer is enough to declare a QA module; context, question -> search_query declares a query generator; document -> summary, key_points: list[str] declares a multi-output summarizer. Predict consumes the signature and an LM handles the rest.

signature (typed declaration) Predict call LM output structural / notation

point: modules are parameterized LM calls built on signatures. Predict is the drop-in; ChainOfThought modifies a signature by prepending a rationale output field and wraps Predict with the modified signature; ReAct adds a Thought / Action / Observation loop. Modules compose with PyTorch-style forward.

module class signature inside module built-in module name composition pattern

point: the compiler runs three stages. Stage 1 generates candidate instructions, field descriptions, and demonstrations for each Predict. Stage 2 optimizes parameters — BootstrapFewShot rejection-samples traces and keeps good ones; BootstrapFinetune uses demos to fine-tune LM weights. Stage 3 modifies control flow — ensembling bootstrap K copies and replaces each Predict with a majority vote.

compiler stage teleprompter / optimizer name module affected runtime caveat

point: on GSM8K math word problems, vanilla GPT-3.5 zero-shot reaches 24.0 dev / 25.2 test; CoT with bootstrap and ensemble reaches 88.3 dev / 81.6 test — CoT itself is worth +26 (24.0 → 50.0), bootstrap adds +30 (50.0 → 80.3), ensemble adds +8 (80.3 → 88.3). On Llama2-13b-chat, vanilla zero-shot starts at 7.0 dev / 9.4 test; CoT with bootstrap reaches 43.3 dev — the smaller model closes a meaningful share of the gap.

result row compilation method headline result (brackets) paper-report number

point: on HotpotQA multi-hop, vanilla fewshot reaches 34.3 EM on GPT-3.5 dev; the compiled multihop module with bootstrap and ensemble reaches 54.7 EM — a +20.4 jump from compilation alone. Llama2-13b multihop ensemble reaches 50.0 Ans Dev. The cost: 50% of HotpotQA test set was not evaluated due to budget; demonstrations don't transfer across LMs without re-bootstrapping; compilation runs minutes to tens of minutes.

result row headline result (brackets) paper-acknowledged limit