World's Fair 2025
Training Agentic Reasoners — Will Brown, Prime Intellect
Overview
This talk argues that reasoning and agents are fundamentally the same concept, with reinforcement learning (RL) being the key to developing powerful agentic systems. The speaker posits that traditional approaches to building agents often involve manual, iterative tuning that mirrors RL processes. By framing agent development through the lens of RL, developers can leverage established algorithms and techniques to create more robust and capable agents, especially for complex, multi-turn tasks.
Who should watch
- AI Engineers
- Product Managers
- Developers building agentic applications
- Researchers exploring advanced AI capabilities
- Those interested in improving agent performance and reliability
- Individuals looking to bridge the gap between agent development and reinforcement learning
Key takeaways
- Reinforcement learning is a critical technique for training capable agentic reasoners, enabling models to improve performance on complex tasks and tool use.
- Building agents often involves manual iteration that is conceptually similar to reinforcement learning, suggesting a unified approach.
- The complexity of real-world tasks requires more than simple prompt engineering; RL provides a framework for systematic improvement.
- Tools like MCP are essential for agents to interact with environments, and RL can train agents to use these tools effectively.
- Developing robust evaluation methods (rubrics) is crucial to prevent reward hacking and ensure agents learn desired behaviors.
- Frameworks are emerging to simplify the application of RL to agent development, making it more accessible to a wider range of developers.
- Multi-turn interactions and tool use are key areas where RL can significantly enhance agent capabilities.
- *The more complex your system, the more things can go wrong, the more that a generic LM API is going to be brittle.*
Notable quotes
The speaker suggests that reasoning and agents are fundamentally the same thing.
*RL is kind of the way around it. It's the trick you can do to take the system that kind of works... and train the model to be better at that thing.*
The process of tuning prompts, harnesses, and tools for agents is akin to performing RL by hand.
Unofficial community note. Prefer the recording for nuance.