World's Fair 2024
Realtime Data Connectivity for AI: Tanmai Gopal
Overview
This talk addresses the challenge of connecting Large Language Models (LLMs) to real-time data and business logic. The core thesis is that LLMs, while capable of complex tasks like coding, often struggle to interact intelligently with structured and unstructured data sources. The proposed solution involves making live data and business logic available to LLMs as tools, enabling them to perform data-driven actions.
Who should watch
- AI Engineers
- Product Managers
- Builders working with LLMs and data
- Those facing challenges in connecting LLMs to internal data sources (calendars, CRMs, ticketing systems, etc.)
- Developers looking to enable LLMs to perform actions based on real-time data.
Key takeaways
- LLMs can write complex code but often fail at simple data interactions, like querying a calendar for meeting counts or a CRM for deal status.
- A key solution is to expose live data and business logic as tools that LLMs can utilize.
- A unified query language, where LLMs can interact with structured data, unstructured data, and APIs in a consistent manner, is crucial. LLMs understand SQL semantics, making it a viable base for such a language.
- An object model for authorization can simplify access control, ensuring consistent security regardless of data source or access method.
- LLMs can be empowered to figure out data access plans themselves by being prompted to write Python code to retrieve and process the necessary data.
- The concept of a "data singularity" is presented, emphasizing the need to bring AI and data together for effective applications.
- The speaker demonstrated a system that successfully generates an email to a top customer, referencing recent movie rentals from a Blockbuster database, by querying multiple data sources.
Notable quotes
*The real pain that I have as I work with LLMs is that they can write a Flappy Bird for me with my face going up and down in 30 seconds but they cant talk to my data intelligently.*
*What if your LLM could talk to everything the same way?*
*When you're asking LLMs to figure out how to retrieve data we just ask it to run Python code fetch the data that we want.*
Unofficial community note. Prefer the recording for nuance.