World's Fair 2025
\"Data readiness\" is a Myth: Reliable AI with an Agentic Semantic Layer — Anushrut Gupta, PromptQL
Overview
The talk argues that the concept of "data readiness" for AI is a myth, as data is rarely perfect and constantly changing. Instead of striving for pristine data, the focus should be on building AI systems that can reliably work with messy, evolving data. This is achieved by creating an agentic semantic layer that learns and adapts to the specific business domain and its nuances over time, much like an experienced human analyst.
Who should watch
- AI engineers and builders working with production data.
- Product Managers and engineers aiming to deploy reliable AI systems.
- Anyone struggling with data quality issues in AI deployments.
- Those exploring semantic layers or knowledge graphs for AI.
- Teams looking to bridge the gap between AI capabilities and business-specific language.
Key takeaways
- The pursuit of perfectly clean and ready data for AI is an unattainable goal, as data domains and schemas frequently change.
- Traditional approaches like data standardization, Master Data Management (MDM), and manual semantic layers or knowledge graphs fail to capture the full complexity and tacit knowledge of a business.
- An agentic semantic layer acts like a continuously learning analyst, improving its understanding of the business domain through user interaction and course correction.
- This approach decouples the LLM's planning capabilities from the deterministic execution of data retrieval and computation, reducing hallucinations.
- The system uses a domain-specific language (DSL) to generate plans that are then executed by a distributed query engine, interacting with various data sources.
- Users can steer the AI's process by editing its generated plans or providing feedback, which the system learns from to improve its semantic layer.
- This method allows AI to handle complex, multi-step queries across disparate systems and learn company-specific jargon and definitions over time.
- The result is a self-improving AI that can achieve high accuracy on complex tasks, reducing deployment time and effort.
Notable quotes
*Data readiness is a myth. How do we still make reliable on no matter how messy data we have?*
*The problem is that the AI does not speak your business business's language.*
*Let's make an AI system that behaves It's like an analyst you just hired today.*
Unofficial community note. Prefer the recording for nuance.