World's Fair 2025
3 ingredients for building reliable enterprise agents - Harrison Chase, LangChain/LangGraph
Overview
Building reliable enterprise agents hinges on three core ingredients: maximizing value when the agent is correct, minimizing cost when it is wrong, and increasing the probability of success. This approach provides a foundational framework for developing agents that are more likely to be adopted and effective within an enterprise setting. The future vision involves numerous agents operating autonomously, coordinating tasks, and requiring careful management.
Who should watch
- AI engineers and developers building agentic systems.
- Product Managers evaluating the feasibility and adoption of AI agents.
- Builders seeking to understand how to create agents that are both valuable and safe for enterprise use.
- Individuals interested in the evolution from chat-based agents to more autonomous, background-operating systems.
- Those concerned with the reliability, predictability, and cost-effectiveness of AI agents in production.
Key takeaways
- The success of an agent can be framed by the equation: (Probability of Success * Value if Successful) / Cost if Wrong. This must exceed the cost of running the agent.
- Increasing agent value can be achieved by targeting high-value verticals like legal or finance, or by enabling agents to perform more substantial, long-term tasks, such as deep research or ambient code generation.
- Agent reliability is enhanced by making more of the agent's processes deterministic, moving beyond simple prompting to structured workflows, especially for predictable sequences of actions.
- Frameworks like LangGraph are designed to bridge the gap between deterministic workflows and agentic behavior, allowing developers to choose the optimal point on this spectrum for their application.
- Reducing the perceived error bars of agent performance through observability and robust evaluation is crucial for enterprise adoption. Tools like LangSmith can provide transparency into agent operations and aid in gaining approval.
- Minimizing the cost of agent errors involves strategies like making changes easily reversible (e.g., code commits) and implementing human-in-the-loop mechanisms, such as pull requests or approval steps.
- Ambient agents, triggered by events rather than direct human commands, represent a scalable future where agents operate autonomously in the background, handling more complex operations and increasing concurrency.
- Ambient agents do not equate to full autonomy; human oversight through approve/reject patterns, tool editing, question-answering, or time-travel debugging remains vital for managing risk and ensuring alignment.
Notable quotes
*The more value it provides when it's right, the more likely it will be to be adopted.*
*The core thing that we've seen... it's hard to make it work reliably, put it in in in production.*
*Ambient does not mean fully autonomous.*
Unofficial community note. Prefer the recording for nuance.