Building Agentic Applications w/ Heroku Managed Inference and Agents — Julián Duque & Anush Dsouza

The talk argues that the concept of "data readiness" for AI is a myth, as data is rarely perfect and constantly changing. Instead of striving for pristine data, the focus should be on building AI systems that can reliably work with messy, evolving data. This is achieved by creating an agentic semantic layer that learns and adapts to the specific business domain and its nuances over time, much like an experienced human analyst.

This talk explores the concept of agentic enterprise, positing that AI agents could fundamentally reshape organizations within three years. It defines AI agents as autonomous, LLM-based applications capable of planning, tool usage, and dynamic adaptation. The presentation emphasizes that digital assets should evolve to serve as tools or agents themselves, enabling capabilities like multi-step reasoning, adaptability, and computer vision for automation.

This talk focuses on the critical process of evaluating AI agents, emphasizing a shift from ad-hoc testing to a more methodical approach. It highlights that effective evaluation should begin at the earliest stages of AI development, not as an afterthought. The presentation introduces tools and frameworks, particularly the Azure AI Evaluation SDK, to ensure AI agents behave correctly and safely as they gain more independence.

This talk introduces Agentic GraphRAG, a method for simplifying data retrieval across both structured and unstructured sources. It proposes using a knowledge graph as an intermediary layer to enhance agentic workflows. By modeling data within a graph, agents can more accurately decompose complex questions, pull relevant information, and perform analytical tasks that go beyond simple semantic search.

This talk explores Box's journey in building an agentic platform, focusing on how agentic approaches can solve complex data extraction challenges that traditional methods and basic LLM calls struggle with. The core thesis is that an agentic abstraction layer provides a clean, flexible, and evolvable architecture for tackling sophisticated AI tasks, moving beyond simple chatbot interactions to orchestrate complex workflows.