Europe 2026
Lessons from Trillion Token Deployments at Fortune 500s — Alessandro Cappelli, Adaptive ML
Alessandro Cappelli , Adaptive ML
Overview
This talk argues that reinforcement learning (RL) is crucial for moving AI models, particularly large language models (LLMs), from pilot stages to production. The speaker posits that the common failure of GenAI pilots stems from the "myth of the last mile," where initial MVPs built on proprietary models or instruction fine-tuning lack systematic improvement pathways. RL, by its nature, allows for the mathematical integration of feedback, leading to more effective model steering and enabling scaled, cost-efficient, and faster deployments.
Who should watch
- AI Engineers
- Product Managers
- Builders working on LLM deployment
- Teams struggling with pilot-to-production transitions
- Those exploring agent development and deployment
Key takeaways
- 95% of GenAI pilots fail to reach production due to the misconception that the hard part is creating an MVP, not the subsequent journey to production.
- Instruction fine-tuning and proprietary models offer limited systematic improvement; RL provides a mathematical framework for integrating feedback.
- RL enables outsized performance, allowing for smaller, cheaper, and faster models that are essential for large-scale enterprise applications and latency-sensitive use cases.
- For agents, RL is particularly well-suited as it was originally designed for training agents in environments, naturally aligning with agentic workflows.
- RL can generate synthetic datasets by leveraging environments and reward signals, addressing the common challenge of scarce agent training data.
- Human feedback in RL can be integrated through LLM judges or by training reward models, reducing reliance on expensive and time-consuming annotation campaigns.
- Reinforcement learning is complex, often requiring orchestration of multiple LLMs, but platforms like Adaptive Engine offer pre-built recipes to manage this complexity.
- *RL is the one algorithm that will let you bring model into production in a systematic and industrialized way.*
Notable quotes
*95% of GenAI pilots fail to reach production.*
*RL is the one algorithm that will let you bring model into production in a systematic and industrialized way.*
Unofficial community note. Prefer the recording for nuance.