Overview
Dr. Zero is a data-free self-evolution framework for search agents that utilizes a proposer-solver feedback loop and external search engine signals to autonomously enhance reasoning capabilities. It addresses the lack of question diversity and high compute requirements in existing methods by introducing an automated curriculum and a novel optimization technique.
Abstract
Dr. Zero (DeepResearch-Zero) introduces a framework enabling search agents to self-evolve without any training data. As high-quality data becomes scarce, data-free self-evolution offers a path where LLMs autonomously generate and solve complex problems to improve their reasoning capabilities. The key innovation is a proposer-solver co-evolution loop where both models are initialized from the same base LLM and improve through mutual feedback.
Methodology
Proposer-Solver Feedback Loop: The proposer generates diverse, challenging questions to train the solver. As the solver evolves, it incentivizes the proposer to produce increasingly difficult yet solvable tasks, creating an automated curriculum.
HRPO (Hop-Grouped Relative Policy Optimization): A novel optimization method that clusters structurally similar questions to construct group-level baselines, eliminating expensive nested sampling while maintaining training stability.
Multi-Turn Tool-Use Rollout: The proposer uses external search engines to generate complex, multi-hop questions rather than simple one-hop queries, dramatically increasing question diversity.
Difficulty-Guided Reward: The proposer is rewarded for generating questions that are challenging but verifiable — trivial questions (all correct) and impossible questions (none correct) receive low rewards.
Key Contributions
- Data-free search agents that match or surpass fully supervised baselines by up to 14.1% on complex QA benchmarks
- HRPO reduces compute requirements for solver training without compromising performance
- Demonstrates that complex reasoning and search capabilities can emerge solely through self-evolution
- Published January 2026, with code available on GitHub (facebookresearch/drzero)
Significance
Dr. Zero challenges the assumption that human-curated training data is necessary for advanced reasoning and search capabilities. The proposer-solver co-evolution paradigm could reduce the cost and effort of training specialized AI agents across domains.
Related
- deepseek-conditional-memory — complementary architecture research on memory mechanisms for LLMs
- optimal-small-ai — related research on efficient AI model strategies
- ai-engineering-foundation-models — production context for foundation model capabilities