Selected Papers
Papers
Open-access papers, commentary here. Full text at the source; this site hosts no PDFs.
19 entries
2026 · arXiv preprint · Trustworthy AI for Good (AI4GOOD) workshop · arXiv
Can escalation channels redirect reward hacking toward defect disclosure?
Escalation channels turn a coding agent's reward-hack response into defect disclosure: a structured `submit_escalation_report` tool given at the moment of conflict cuts reward hacking 23.6% to 5.3% (mixed-effects OR 9.2, 95% CI 5.0 to 16.8) across 8 frontier models in 5 families and adds +10.1pp of defect detection coverage over monitoring alone, without detectable solve-rate or cost overhead.
2023 · arXiv preprint · ICML 2024 · UIUC / Lapis Labs · arXiv
Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models
LATS adapts Monte Carlo Tree Search to language agents, replacing the world model that classical MCTS requires with environment feedback, and using the same pretrained LM as both the agent and the value function — making search tractable in interactive tasks where you can revert to any prior context.
2026 · arXiv preprint · Google Research / Virginia Tech · arXiv
WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution
WikiSkill co-evolves agent skills with a persistent knowledge base (wiki): raw traces, accumulated patterns, and active procedures live in three distinct layers, and the wiki compounds across iterations while skills are validated and rolled back on rejection.
2022 · NeurIPS 2022 · DeepMind · arXiv
Training Compute-Optimal Large Language Models
Given a fixed training compute budget, large language models are significantly undertrained — the compute-optimal choice scales parameters and training tokens equally, as proven by Chinchilla (70B parameters on 1.4T tokens, same compute as Gopher), which beats Gopher (280B), GPT-3 (175B), and Megatron-Turing NLG (530B) and reaches 67.6% on MMLU.
2017 · NeurIPS 2017 · arXiv:1706.03762v7 · arXiv
Attention Is All You Need
The Transformer replaces recurrence with pure multi-head scaled dot-product attention: six encoder layers + six decoder layers, each containing multi-head self-attention, position-wise feed-forward networks, residual connections, and layer normalization — 28.4 BLEU on WMT 2014 EN-DE and 41.8 on EN-FR at a fraction of prior models' training cost.
2023 · arXiv preprint · Stanford / UC Berkeley / CMU / Amazon / Dash · arXiv
DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
A programming model that compiles declarative LM pipelines into optimized prompts or fine-tunes, replacing hand-crafted prompt strings with parameterized modules (Predict, ChainOfThought, ReAct) and bootstrapping them with teleprompters that search for effective few-shot demonstrations — small models compiled this way match GPT-3.5 on math word problems and multi-hop QA.
2024 · arXiv preprint (Sierra) · arXiv
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
A benchmark that emulates multi-turn tool-agent-user interaction as a POMDP — even GPT-4o function calling succeeds on under half of τ-airline tasks, and pass^8 reliability drops to single digits.
2026 · arXiv preprint · arXiv
A Survey on Evaluation of LLM-based Agents
A taxonomy of agent evaluation across four axes — core capabilities, application-specific benchmarks, generalist-agent platforms, and framework tools — with a structural comparison of representative benchmarks by five cross-cutting dimensions (data curation, environment, interface, metric, safety) and a critique of where current evaluation falls short.
2023 · ICLR 2024 · arXiv
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
A single LM emits reflection tokens to fetch passages on-demand, then critique its own segment for relevance, support, and utility before ranking.
2024 · arXiv · DeepSeek-AI / Tsinghua / Peking · arXiv
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
DeepSeekMath 7B trains on a 120B-token math corpus from Common Crawl and applies GRPO — a critic-free PPO — to reach 51.7% on competition-level MATH, near GPT-4 without external toolkits.
2025 · arXiv · DeepSeek-AI · arXiv
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-R1 demonstrates that pure RL — without any human-labeled reasoning traces — incentivizes emergent long chain-of-thought including self-reflection, verification, and backtracking.
2023 · NeurIPS 2023 · Stanford / CZ Biohub · arXiv
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
DPO bypasses RLHF by deriving the optimal policy in closed form — fine-tuning reduces to a binary classification loss on preference pairs, with no reward model and no rollouts.
2024 · arXiv preprint · arXiv
Retrieval-Augmented Generation for Large Language Models: A Survey
A taxonomy of RAG: Naive (Retrieve-Read), Advanced (pre + post-retrieval), and Modular (rewritable modules) — each generation fixing a subset of the previous one's losses.
2024 · arXiv · Meta AI · arXiv
The Llama 3 Herd of Models
Llama 3 is a herd of multilingual dense Transformers (8B / 70B / 405B) trained on 15.6T tokens with a 128K context window, then post-trained with six alternating rounds of SFT + rejection sampling + DPO to reach GPT-4-class instruction following and helpfulness.
2024 · arXiv · Alibaba · arXiv
Qwen2.5 Technical Report
Qwen 2.5 scales pre-training to 18T tokens across knowledge, code, and mathematics, then applies a two-stage post-training pipeline — SFT + offline DPO + online GRPO with five labeling criteria — to ship seven open-weight dense models (0.5B–72B) plus a 1M-context MoE variant that matches GPT-4o-mini on most benchmarks at lower inference cost.
2024 · arXiv · Renmin University / Huawei · arXiv
A Survey on the Memory Mechanism of Large Language Model based Agents
A unified taxonomy of agent memory across three sources (inside-trial / cross-trial / external), two forms (textual / parametric), and three operations (writing / management / reading), with a survey of evaluation methods and open problems.
2023 · arXiv preprint · arXiv
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
DFSDT lets an LLM expand reasoning on RapidAPI tool calls: a depth-first decision tree where a wrong API can be aborted via Finish by Giving Up and a sibling node tried next.
2023 · NeurIPS 2023 · arXiv
Reflexion: Language Agents with Verbal Reinforcement Learning
Reflexion reinforces language agents with verbal self-reflection and an episodic memory buffer; no weight updates, faster learning from trial-and-error.
2023 · ICLR · arXiv
ReAct: Synergizing Reasoning and Acting in Language Models
Reasoning and acting are one generation loop: the model thinks in language, acts on the world, and lets observations revise the next thought.
No papers match.