Overview
Huang et al. (Tencent AI Seattle Lab + universities, arXiv:2508.05004, Jan 2026) — Fully autonomous framework that generates its own training data from scratch. A single base LLM is split into a Challenger and Solver that co-evolve through interaction.
Mechanism
- Challenger is rewarded for proposing tasks near the edge of the Solver’s current capability
- Solver is rewarded for solving increasingly challenging tasks posed by the Challenger
- Both optimized via GRPO (Group Relative Policy Optimization)
- Challenger reward signal derived from Solver uncertainty (self-consistency of multiple answers)
- Solver fine-tuned on filtered challenging questions using pseudo-labels voted by itself
This creates a self-evolving cycle operating without any human intervention.
Results
| Backbone | Math Benchmarks | General Reasoning |
|---|---|---|
| Qwen3-4B-Base | +6.49 (3 iterations) | +7.54 |
- Model-agnostic framework (works across different backbone LLMs)
- Math-focused training generalizes to complex general-domain tasks (MMLU-Pro, SuperGPQA)
- Acts as effective mid-training method — models first improved by R-Zero achieve higher performance after subsequent fine-tuning on labeled data
Key Innovation
Unlike prior self-challenging approaches that rely on external code executors for verification, R-Zero uses the Solver’s own self-consistency as the verification signal. This enables self-evolution in domains lacking explicit verification oracles (e.g., open-ended reasoning).
Related
- dr-zero-self-evolution — Related self-evolving search agent framework
- crisp-dr-zero-review — Academic review examining the Dr. Zero variant
- reinforcement-learning-introduction — Sutton & Barto’s foundational RL theory underpinning GRPO