Agentic Excellence: Mastering AI Agent Evals w/ Azure AI Evaluation SDK — Cedric Vidal, Microsoft

This talk introduces Agentic GraphRAG, a method for simplifying data retrieval across both structured and unstructured sources. It proposes using a knowledge graph as an intermediary layer to enhance agentic workflows. By modeling data within a graph, agents can more accurately decompose complex questions, pull relevant information, and perform analytical tasks that go beyond simple semantic search.

This talk addresses the critical need for robust evaluation and benchmarking strategies when deploying large language models (LLMs) into production. It highlights the inherent complexities and potential pitfalls of generative AI, emphasizing that scalability, reliability, and safety are paramount. The presentation introduces practical tools and methods to assess LLM performance, ensuring that models meet enterprise-level requirements before and during deployment.

This talk focuses on mastering AI evaluation, moving from initial development to production. It emphasizes that even the best large language models (LLMs) require a robust testing framework due to issues like hallucinations and performance degradation with changes. Effective evaluation helps answer critical questions about model selection, cost-effectiveness, brand consistency, and ongoing improvement, ultimately reducing development time, costs, and enabling faster iteration.

The talk argues that the concept of "data readiness" for AI is a myth, as data is rarely perfect and constantly changing. Instead of striving for pristine data, the focus should be on building AI systems that can reliably work with messy, evolving data. This is achieved by creating an agentic semantic layer that learns and adapts to the specific business domain and its nuances over time, much like an experienced human analyst.

The core thesis of this talk is that 2025 will finally be the year for robust AI evaluation, driven by the convergence of several key factors. The increasing understanding of AI by business leaders, coupled with budget shifts towards generative AI projects, has set the stage. Crucially, the rise of agentic systems, which make decisions and take actions, introduces significant complexity and risk, making rigorous evaluation a necessity rather than an option.