DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

core: DeepSeekMath 7B continues pre-training from a code checkpoint with a 120B-token math corpus mined from Common Crawl. It reaches 51.7% on the competition-level MATH benchmark without external toolkits, closing the gap to GPT-4 (52.9%) and Gemini-Ultra (53.2%). Two contributions: a 4-iteration data-selection pipeline that scales OpenWebMath 7×, and GRPO — a critic-free PPO variant that estimates the baseline from group rewards.

Five panes: the iterative data pipeline, the GRPO algorithm, the DeepSeekMath-Base 7B benchmarks, the RL boost on top, and the paper's own limits.

primer — what you need before you start

the task. DeepSeekMath targets math reasoning in language models. The bet: focused math pre-training + GRPO RL fine-tuning gets a 7B model to within 1% of GPT-4 on competition-level math, without external toolkits.

how it was done before. Minerva (2022) used retrieval over a math corpus; it was 540B parameters. DeepSeekMath gets the same quality at 7B via much better data filtering + RL. The paper's main contributions are the iterative data selection loop and the GRPO algorithm.

glossary — terms you will see.
Common Crawl — public web crawl; the source corpus for most LLMs.
fastText classifier — fast linear text classifier; used to filter math pages.
DeepSeekMath-Base — 7B model after continued pre-training on 120B math tokens.
DeepSeekMath-RL — Base + GRPO RL fine-tuning; the strongest variant.
MATH — competition-level math benchmark (AMC, AIME, Olympiad style).
GSM8K — grade-school math word problems.
miniF2F — formal math olympiad problems.
OpenWebMath — open-source 14B-token math corpus; DeepSeekMath is 7× its size.

symbol table — what the letters mean.
$\pi_\theta(y \vert x)$ — the policy: probability of completion y given problem x.
$\pi_{\text{ref}}$ — frozen reference policy (DeepSeekMath-Base).
$\hat{A}_i$ — advantage of sample i within a group: $\frac{r_i - \text{mean}(r)}{\text{std}(r)}$.
$\text{GRPO}(\theta)$ — group-relative PPO loss: same clipped surrogate, group-relative advantage.
$\mathcal{J}$ — group of rollouts for the same prompt.
$r_i$ — reward of rollout i (correct = +1, format = small bonus, wrong = -1).

tone legend — what the [brackets] mean.
[obs] observation   [pipe] data pipeline
[grpo] algorithm   [rl] reinforcement learning step
[eq] equation   [take] key takeaway
[sys] system / meta   [limit] paper-reported limit

Six panes cover, in pedagogical order: iterative data pipeline; GRPO algorithm; DeepSeekMath-Base benchmarks; RL boost; the paper's own limits. Each pane ends with key takeaways so you can stop after any section and still know what you learned.

point: four iterations of a fastText classifier loop. Step 1 trains a binary fastText classifier on OpenWebMath (positive) + 500K random CC pages (negative). Step 2 recalls math-related pages from CC and keeps the top-K by classifier score. Step 3 groups CC domains by URL and flags domains with >10% pages collected as math domains. Step 4 traverses links from the math domains to grow the seed corpus.

After four iterations: 35.5M math pages, 120B tokens. The third iteration hits 98% saturation, so data collection stops there. The final mix is 56% DeepSeekMath Corpus + 4% AlgebraicStack + 10% arXiv + 20% GitHub code + 10% natural language.

seed corpus fastText classifier math domain URL annotation

point: PPO needs a critic. GRPO drops the critic. For each prompt, sample a group of outputs from the current policy, compute the group mean and standard deviation of rewards, and use the standardized reward as the advantage. The policy gradient then follows PPO's clipped ratio form but with no value function. Same clipped surrogate objective, no separate baseline network.

The crucial difference is the advantage estimator. PPO fits a value head on rollouts; GRPO estimates the baseline from group statistics. The KL term uses k3 estimator to avoid computing the partition function.

clipped ratio group-relative advantage KL penalty

point: DeepSeekMath-Base 7B is initialized from DeepSeek-Coder-Base-v1.5 7B and trained for 500B tokens on the math-heavy mix. It leads every open-source 7B–34B model on the eight listed benchmarks. Notably, on competition-level MATH it surpasses Minerva 540B (33.6%), which is 77× larger.

Numbers are paper-report: chain-of-thought prompting, no tool use. The corpus mix is the key lever — same checkpoint, same architecture, four different corpora (MathPile 8.9B, OpenWebMath 13.6B, Proof-Pile-2 51.9B, DeepSeekMath 120B) produce very different outcomes.

DeepSeekMath-Base 7B Mistral / Llemma tool-integrated (Python)

point: GRPO applied on top of DeepSeekMath-Instruct 7B lifts every benchmark. GSM8K 82.9% → 88.2%, MATH 46.8% → 51.7%, MGSM-zh 73.2% → 79.6%, CMATH 84.6% → 88.8%. With Python tool use the gains add another 5–7 points. Self-consistency over 64 samples reaches 60.9% MATH — approaching GPT-4 (52.9%) and Gemini-Ultra (53.2%).

The Instruct baseline already crosses 80% on GSM8K, so RL adds ~5 points. MATH is the harder regime: 46.8% → 51.7% is a 5-point absolute gain and the largest relative gain (11%) on the listed benchmarks.

DeepSeekMath-RL DeepSeekMath-Instruct closed-source comparison

point: math pre-training primarily improves mathematical reasoning. Natural-language and code benchmarks see smaller gains, and the corpus is English-and-Chinese-centric. The data loop saturates by the third iteration — structural limit on further data scaling at this pipeline's design. GRPO depends on the reward signal having non-trivial variance within each group; for tasks with mostly-correct samples the advantage collapses toward zero and learning stalls.

Code training before math training improves math performance — a partial answer to the question of whether code is "for math" or "for reasoning in general". The paper evaluates up to 7B; scaling to larger LMs is open. Tool use remains essential for the highest scores on competition-level math.

limit open direction