Beyond Code Coverage: Functionality Testing with Playwright MCP — Marlene Mhangami, Microsoft

This talk introduces Pydantic AI's agent optimization capabilities, focusing on the Jeppa library for prompt optimization and managed variables for dynamic configuration. The core idea is to systematically improve agent performance by iteratively refining prompts and configurations, demonstrated through a practical example of analyzing political family ties in MP data. The presentation highlights how these tools can lead to more accurate and efficient AI agents.

This talk addresses the challenges of AI evaluations, which are currently scattered, quickly become outdated, and often lack transparency and verifiability. The speakers propose solutions to democratize the evaluation process, enabling a broader community to contribute to and benefit from robust AI benchmarking. Their work aims to foster more equitable AI development by allowing diverse expertise to shape evaluation standards.

This talk explores the effectiveness of semantic code retrieval for AI coding assistants, specifically benchmarking it against traditional grep-based methods on Claude Code. The core thesis is that while grep is simpler and often sufficient, semantic search, powered by vector embeddings, can significantly improve precision and the agent's ability to locate relevant code, especially for complex tasks. This approach offers a form of cached compute, reducing redundant processing and potentially leading to better accuracy and user satisfaction.

This talk explores strategies for effectively integrating and managing third-party tools within AI agent workflows, specifically focusing on Multi-Call Protocol (MCP) servers. It addresses the common challenges of generic tool descriptions, unexpected agent behavior, and potential performance degradation or security risks when using off-the-shelf tools. The presentation outlines a framework of five best practices to tailor these tools for specific use cases, ensuring agents perform as intended and applications remain stable.

This talk explores the evolution of generative UI for Multi-Component Platform (MCP) applications, moving beyond static components to more dynamic and collaborative interfaces. It posits that current AI models are capable of generating sophisticated UI code, prompting a re-evaluation of how users interact with AI-powered applications. The core thesis is that the future of generative UI lies in collaborative experiences rather than just component-based outputs.