World's Fair 2024
Pydantic is STILL all you need: Jason Liu
Overview
This talk argues that Pydantic remains the essential tool for building reliable applications with large language models (LLMs). The core thesis is that by leveraging Pydantic's schema definition and validation capabilities, developers can move beyond the unreliability of unstructured text outputs and program with data structures, similar to classical software development. This approach enhances composability, reliability, and developer experience when interacting with LLMs.
Who should watch
- AI Engineers
- Product Managers
- Developers building with LLMs
- Anyone frustrated with unreliable LLM outputs
- Those looking to improve data validation and structure in AI applications
Key takeaways
- Pydantic provides a robust way to define structured outputs for LLMs, improving reliability and composability compared to raw text responses.
- The `instructor` library, which integrates Pydantic with LLM clients, has expanded to support multiple languages and various LLM providers beyond OpenAI.
- Validators within Pydantic models can enforce complex business logic and data integrity, with LLMs capable of retrying to correct errors based on informative error messages.
- Structured outputs are beneficial across various AI applications, including Retrieval Augmented Generation (RAG), data extraction, and agentic workflows, by enabling programming with well-defined data structures.
- The use of Pydantic transforms LLM interactions into generating predictable data structures, allowing developers to own the objects, functions, and control flow, effectively bringing classical programming paradigms to generative AI.
- Newer LLMs are faster, making the latency cost of structured output generation more acceptable, and fine-tuning on function calling can significantly reduce failure modes.
- The core Pydantic API remains stable, focusing on a client object and verbs like create, create with iterable, and create with partial, making it adaptable to evolving LLM capabilities.
Notable quotes
*The problem is that by not having schemas and structured responses we tend to lose compatibility, composability and reliability when we build tools and write code that interact with external systems.*
*The real idea here is we're going to be programming with data structures which is something everyone knows how to do rather than trying to like beg and pray to the LLM gods.*
*We are actually turning software 3.0 and making it backwards compatible with existing software.*
Unofficial community note. Prefer the recording for nuance.