Code 2025
From Stateless Nightmares to Durable Agents — Samuel Colvin, Pydantic
Overview
This talk introduces Pydantic AI's integration with durable execution frameworks like Temporal, addressing the challenge of maintaining state and preventing data loss in long-running AI agent workflows. It demonstrates how these frameworks enable agents to recover from failures and resume execution without starting from scratch, significantly improving reliability for complex tasks. The presentation also touches upon Pydantic Evals for model evaluation.
Who should watch
- AI Engineers building complex, multi-step agentic workflows.
- Product Managers seeking to improve the reliability of AI-powered features.
- Developers encountering issues with state management in long-running AI processes.
- Builders interested in robust orchestration for AI agents.
Key takeaways
- Durable execution frameworks like Temporal are crucial for handling long-running AI agent workflows, preventing data loss upon failure.
- Pydantic AI integrates with Temporal and other durable execution backends, allowing agents to be wrapped and managed by these systems.
- Temporal operates on a deterministic workflow and non-deterministic activity model, recording execution history to enable seamless resumption.
- When a workflow fails, Temporal can automatically retry activities or resume the workflow from its last saved state, leveraging cached results for completed steps.
- The integration allows for seamless recovery even if the execution environment (e.g., Kubernetes) restarts the process.
- Temporal's ability to cache results means that previously completed steps in a workflow can be replayed instantly, speeding up resumption.
- The talk showcases a deep research agent example, demonstrating how durable execution and parallel processing (using Python's task groups) can be applied to complex tasks like information gathering and analysis.
- Pydantic Evals can be used to benchmark and compare the performance and cost of different LLMs on specific tasks.
Notable quotes
*The problem is if this dies either because we have some unreliable endpoint within our system or because we're running in the cloud and Kubernetes decides it wants to scale or whatever it might be. If we run this again, we obviously have to start from scratch.*
*Temporal just returned the result, the kind of cached result that it already had for each of these cases. So we're able to effectively zoom forward to the point where it then continues to to call the LLM.*
*Our deep research agent is actually made up of multiple agents. So we have this plan agent which goes off with a prompt and it returns an instance structured data extraction gives you an instance of this pyantic model which is your plan to run it.*
Unofficial community note. Prefer the recording for nuance.