Product building

This talk addresses the security implications of agentic development, a paradigm shift where AI agents assist in the software development lifecycle. It highlights the potential risks introduced by these agents, such as new attack vectors and vulnerabilities, and emphasizes the need for robust security practices tailored to this evolving landscape. The core thesis is that securing agentic development requires a proactive and specialized approach to mitigate emerging threats.

This talk explores the practical challenges and considerations of granting AI agents access to production codebases. It delves into the necessary infrastructure, safety protocols, and the evolving role of human oversight when AI is empowered to make changes in a live production environment. The core thesis revolves around the tension between AI's potential for rapid development and the critical need for robust safeguards.

This talk addresses the challenges of managing and understanding AI agents in production environments. It emphasizes the need for robust monitoring and observability to ensure agents behave as expected and to debug issues effectively. The core

This talk addresses the critical gap between raw AI agent output and a user-friendly experience, arguing that a dedicated rendering layer is essential for LLM pipelines. It emphasizes that simply displaying raw LLM responses fails to meet user expectations for intuitive and effective interaction. The core thesis is that building a robust rendering layer is key to successfully shipping AI-powered products.

This talk addresses the critical gap between the excitement AI developers feel and the apprehension users often experience with AI-powered applications. It emphasizes that successful AI product adoption hinges on thoughtful user experience (UX) design, focusing on building user trust, ensuring privacy and security, and guiding users through novel interaction patterns. The core thesis is that teaching users how to effectively interact with AI capabilities is essential for them to derive value from new applications.

This talk argues that the core challenge in software engineering is shifting from writing code to managing complexity, designing systems, and ensuring correctness. While AI models now excel at generating code syntax, the future of software development lies in leveraging AI for higher-level tasks like architectural design, complex problem decomposition, and inductive reasoning. The economics of code production are changing, making code generation nearly free and emphasizing the need for new processes and evaluation methods.

Benoit Schillings, VP of Research at Google DeepMind, outlines the frontier of AI engineering research, emphasizing the shift from foundational model development to sophisticated engineering for product integration and real-world impact. The agenda focuses on building robust, scalable, and reliable AI systems that can be productized and deployed effectively. This involves a deep dive into the engineering challenges and opportunities at the forefront of AI research.

This talk explores the concept of "imagination engineering," proposing that as AI models become increasingly capable of executing tasks, the primary bottleneck for innovation will shift to generating novel and ambitious ideas. The speaker shares personal experiments using AI to capture and organize streams of consciousness, build personal websites from thoughts, and analyze the commonalities among highly creative historical figures. The core thesis is that thinking and building in public, amplified by AI, can unlock new levels of creativity and knowledge generation.

This talk argues that while AI model intelligence has rapidly advanced, its real-world usefulness is hampered by a lack of business-specific context. The core thesis is that a "context layer" is the missing infrastructure for production-ready AI agents, analogous to how humans learn and operate within a company's knowledge, skills, and norms. This layer aims to transform implicit knowledge into machine-usable context, enabling AI to perform effectively in business environments.

This talk emphasizes the critical need for rigorous evaluation of AI skills before deployment, arguing that shipping skills without proper testing can lead to unpredictable behavior and degraded performance. The speaker highlights the distinction between agents developers use for personal productivity and agents built for consumers, noting that end-users lack the context to troubleshoot skill invocation issues. The core thesis is that comprehensive evaluations are essential for ensuring skill reliability, managing costs, and determining when skills can be retired as models improve.

This talk explores the design principles behind creating secure and scalable agent sandbox clouds. It begins by explaining why AI models need the ability to execute code or use tools to handle tasks with verifiable rewards, such as math and coding problems. The presentation then delves into the evolution of sandbox technologies, from basic process isolation to advanced virtualization, emphasizing the critical need for robust security to protect against malicious or overzealous code execution in both research and product environments.

This talk presents five techniques for reducing AI agent hallucinations and token waste, focusing on code-based solutions rather than prompt engineering. These methods aim to improve accuracy and catch failures before they impact users. The techniques include semantic tool selection, graph retrieval augmented generation (RAG), multi-agent validation, neuro-symbolic guardians, and runtime guardians.

The developer role is shifting from solely writing code to designing and managing complex systems, augmented by AI tools. This evolution means developers are becoming more efficient, acting as orchestrators rather than just coders. The core idea is to leverage AI to scale development efforts, enabling individuals to achieve significantly higher productivity while maintaining control through robust guardrails.

This talk introduces design patterns for building trust in AI agents, drawing parallels to human collaboration and decision-making processes. The core thesis is that managing AI agents effectively involves applying principles similar to how teams of humans are managed, particularly in complex or ambiguous situations. The presentation emphasizes that AI agents, like people, benefit from clear guidance, access to knowledge, and structured evaluation methods to ensure reliable and trustworthy outputs.

This talk addresses the critical challenge of explaining complex AI products to potential customers and stakeholders. The core thesis is that many AI products fail not due to technical shortcomings, but because their value proposition is not clearly communicated. The speaker emphasizes that effective storytelling, focusing on customer pain points and tangible transformations, is essential for product success in a crowded market.

This talk outlines a repeatable framework for designing and building AI systems from initial idea to production. It emphasizes that in the current AI landscape, defining product requirements, system design, and evaluation criteria are the critical, challenging aspects, rather than just coding. The framework consists of four phases: product requirements, system design, evaluation and monitoring, and optimization.

This talk addresses the challenge of transitioning frontier machine learning research into production-ready software. It proposes a three-pronged approach focusing on research legibility, code structure, and decomposition strategies to improve the velocity and efficiency of teams bridging the gap between ML research and software engineering. The core idea is to treat this transition as a systems and process problem.

This talk introduces spec-driven development as a structured approach to software engineering, emphasizing the creation of detailed specifications and design documents before writing any code. This methodology is particularly effective when working with AI coding assistants, guiding them to produce higher-quality, more accurate code. The core idea is to leverage AI as an intern that requires clear direction, with spec-driven development providing that essential guidance.

This talk details OpenGov's journey in building and scaling OG Assist, an AI agent integrated across their government software products. The core thesis revolves around the strategic decision to bet on the Effect TypeScript library for building a robust agent loop, enabling significant advancements in development and production capabilities. The presentation highlights practical applications, architectural choices, and the iterative process of improving AI agent performance in a real-world enterprise environment.

This talk argues that traditional benchmarks are insufficient for evaluating agentic AI systems in production. Agentic systems, which plan, call tools, and execute workflows, require a shift in evaluation focus from model capability to system behavior. The core thesis is that production telemetry and reliability metrics are paramount for dependable outcomes, moving evaluation from a pre-deployment phase to a continuous operational capability integrated into the system's control plane.

This talk outlines a production AI playbook, a framework designed to guide the deployment of AI systems at an enterprise scale. It addresses common pitfalls encountered when moving AI from demos to production, emphasizing the need for structured approaches to evaluation, observability, data foundation, orchestration, and governance. The framework aims to ensure AI systems are measurable, accountable, and reliable in real-world applications.

This talk introduces Google DeepMind's Gemma family of open models, highlighting their capabilities and the benefits of using open-source AI for greater ownership and customization. The presenters emphasize that while proprietary models like Gemini offer cutting-edge performance, open models like Gemma provide crucial advantages for specific use cases, such as running models on local hardware, handling sensitive data, and adapting models to unique requirements.

This talk introduces a pipeline designed to automate the process of turning product observability data into actionable code changes. Instead of engineers manually reviewing dashboards and creating pull requests (PRs) for issues, the system aims to automatically detect product signals, diagnose problems, and generate PRs for review or even direct merging. The ultimate vision is a product that can largely build and maintain itself by learning from user interactions and data.

This talk explores advancements in AI audio processing, focusing on Google DeepMind's Gemini models. The core thesis is that Gemini's sophisticated audio understanding capabilities enable richer transcription, robust reasoning, and more nuanced speech generation, moving beyond simple speech-to-text to a more comprehensive audio comprehension and synthesis system.

This talk introduces text diffusion, a forward-looking research area in AI that offers an alternative to traditional autoregressive text generation. Unlike models that generate text one token at a time, text diffusion models start with a noisy sequence and iteratively refine it to produce clean output. This approach promises significant advantages in inference speed and enables unique capabilities such as bidirectional reasoning and in-place editing.

This talk explores the evolution of generative UI for Multi-Component Platform (MCP) applications, moving beyond static components to more dynamic and collaborative interfaces. It posits that current AI models are capable of generating sophisticated UI code, prompting a re-evaluation of how users interact with AI-powered applications. The core thesis is that the future of generative UI lies in collaborative experiences rather than just component-based outputs.

This talk details the process of training Zeta2, an edit prediction model for the Zed code editor. The core thesis revolves around using distillation from frontier models and leveraging opt-in production data to create a specialized, fast model for predicting user edits. The training pipeline emphasizes data processing, prompt engineering for teacher models, and a novel approach using "settled data" to generate high-quality training examples.

Building AI agents presents unique challenges compared to traditional software development. Instead of acting as traffic controllers with predefined rules, engineers now function more like dispatchers, defining goals for agents without dictating every step. This shift requires embracing the non-deterministic nature of AI, treating errors as inputs, and moving from rigid unit tests to broader evaluations of agent reliability and success rates.

This talk addresses the challenges of AI evaluations, which are currently scattered, quickly become outdated, and often lack transparency and verifiability. The speakers propose solutions to democratize the evaluation process, enabling a broader community to contribute to and benefit from robust AI benchmarking. Their work aims to foster more equitable AI development by allowing diverse expertise to shape evaluation standards.

This talk discusses Google DeepMind's approach to building and scaling agentic software. It highlights the Antigravity framework as an integrated IDE and agent management platform, showcasing its capabilities in code generation, task execution, and human-in-the-loop feedback. The discussion also touches upon the challenges and strategies for managing agentic systems at scale, including resource allocation, cost efficiency, and robust evaluation methods.

This talk introduces Google DeepMind's latest advancements in generative AI, focusing on the Gemini family of models and their applications. The presentation highlights the multimodal capabilities of Gemini, its cost-effectiveness, and its integration into tools like AI Studio for building applications. It also touches upon the development of specialized models for generative media, including image, video, and music creation, and the increasing accessibility of powerful AI models for on-device and local execution.

This talk introduces Gemini Nano, Google's on-device AI model for Android, and the ML Kit GenAI APIs that provide access to it. The presentation emphasizes the benefits of on-device processing, including enhanced privacy, offline capabilities, and cost savings, while also outlining hybrid approaches that leverage cloud models when necessary. The goal is to provide developers with a comprehensive offering for building intelligent experiences across a range of Android devices.

This talk introduces the concept of "any-to-any" multimodal agents, enabled by Google DeepMind's Gemini API. The core idea is to build agents that can understand and generate across various modalities including text, code, images, audio, and video. The presentation outlines the architecture for such agents, focusing on multimodal understanding and native generation capabilities, and demonstrates how to build a notebook LM clone as an example.

This talk demonstrates how to use Google DeepMind's GenMedia models to create rich multimedia content, such as images, videos, and music, by illustrating a book. The presentation walks through practical examples of generating character images, scene illustrations, video clips, and musical scores, all while emphasizing the integration and capabilities of various AI models. The core idea is to leverage AI to bring creative works to life through diverse media.

This talk argues that the common direct HTTP streaming approach for AI chat applications fundamentally limits user experience quality. The default pattern, often using Server-Sent Events (SSE), creates a single, point-to-point connection between a client and an agent, which breaks down when users switch devices, experience network interruptions, or need to interact with a working agent. The presentation proposes a shift towards a decoupled architecture using durable sessions to enable more resilient, multi-surface, and interactive AI experiences.

This talk explores the integration of coding agents, specifically the OpenClaw agent, into products. It emphasizes that coding agents are becoming a fundamental building block for software systems and highlights the Pi framework as an excellent tool for experimentation and development. The presentation demonstrates how agents can be leveraged to automate complex workflows, such as processing sales proposals, by interacting with various tools and systems.

This talk introduces FLUX, an open-source visual AI model developed by Black Forest Labs, and discusses its evolution and the future of visual AI. The presentation highlights FLUX's capabilities in text-to-image generation, image editing, and its progression towards visual intelligence. It also delves into a novel training methodology called Self Flow, designed to improve multimodal generative models by integrating representation learning directly into the generation process, eliminating the need for external encoders.

This talk focuses on accelerating AI deployment on edge devices, highlighting Google DeepMind's Gemma models and the Lite RT framework. The core thesis is that by optimizing models for edge hardware and leveraging a unified cross-platform architecture, developers can achieve significant performance gains, enhanced privacy, and reduced latency for a wide range of AI applications.

This talk focuses on the practical challenges of shipping complex AI applications, moving beyond initial prototypes to robust, production-ready systems. It emphasizes the need for operational rigor, structured workflows, and continuous evaluation, drawing on experiences from Braintrust and Trainline. The core thesis is that while AI models are increasingly sophisticated, the operational practices for deploying and managing them at scale have lagged, creating a significant hurdle for delivering real customer value.

This talk introduces the Gemini Interactions API, a new unified API designed to simplify building with large language models and agents. It emphasizes a more developer-friendly interface, akin to industry standards, and introduces server-side state management for improved agent development. The session also showcases the Gemini Live API for real-time conversational AI applications, including audio and video processing, and demonstrates how to build a coding agent with file I/O and bash command execution capabilities.

This talk explores building and deploying AI-powered applications, focusing on the capabilities and accessibility of Google DeepMind's latest models and tools. It highlights the multimodal nature of Gemini models, their integration into AI Studio for rapid prototyping, and the potential for creating complex applications with features like code execution, search grounding, and dynamic world generation. The presentation emphasizes practical demonstrations over slides, showcasing how developers can leverage these advancements for diverse projects.

This talk provides a deep dive into Gemma 4, Google DeepMind's latest family of open-source AI models. The presentation highlights significant architectural improvements and new capabilities, including enhanced multimodal support and optimized performance for both on-device and complex reasoning tasks. Gemma 4 aims to set a new standard for small, open-source models, making advanced AI more accessible to developers.

This talk provides a behind-the-scenes look at training generative image and video diffusion models at scale. It covers essential aspects from data curation and representation to modeling, architecture, training, sampling, and control mechanisms. The core thesis emphasizes that while diffusion models are powerful, their effectiveness relies heavily on careful data handling, efficient latent representations, and sophisticated sampling techniques like guidance.

Gemma is Google DeepMind's family of open models, designed to be downloadable, runnable on personal infrastructure, and fine-tunable for specific use cases. The latest release, Gemma 4, offers models ranging from 2 to 32 billion parameters, with capabilities extending to on-device agentic tasks, multimodal understanding, and multilingual support. These models are developed with a focus on developer-friendly sizes and an Apache 2.0 license for greater flexibility.

This talk explores Google DeepMind's research into the next frontiers of AI, focusing on advancements beyond traditional language models. It highlights progress in creating more robust and versatile AI systems, including multimodal embedding models, advanced weather prediction systems, and interactive 3D world models. The core thesis is that significant breakthroughs lie in developing AI that can understand and interact with the world in more comprehensive and nuanced ways, moving towards more general intelligence.

This talk addresses the critical security and design considerations for Multi-modal Communication Protocol (MCP) servers intended for production environments. It argues that insecure or poorly designed MCP servers are vulnerable to exploitation by agentic AI systems. The presentation emphasizes that robust security and effective design are intertwined, advocating for a product engineering mindset when building interfaces for AI agents.

This talk explores the challenges and potential of measuring AI capabilities, particularly focusing on developer productivity and long-term task completion. It questions the extrapolation of current AI progress trends, suggesting that physical and economic constraints, alongside potential technological breakthroughs, could alter the trajectory of AI development. The discussion also delves into the complexities of evaluating AI in real-world scenarios beyond controlled benchmarks, highlighting the gap between AI capabilities and practical application in fields like software engineering and data science.

This talk explores building durable, production-ready AI agents by integrating the OpenAI Agents SDK with Temporal. It highlights how Temporal's distributed systems capabilities can provide essential durability, visibility, and scalability to agentic applications, which are often inherently stateful and prone to failures. The presentation demonstrates how to leverage Temporal's workflow and activity abstractions to manage agent execution, tool invocation, and error handling, ensuring that agent processes can reliably resume after interruptions.

This talk introduces Spec-Driven Development as a method to enhance AI agent development, focusing on improving control, code quality, and reliability. It proposes a structured workflow that compresses the software development lifecycle, moving from prompt to requirements, design, and execution, with an emphasis on creating reproducible results. The approach aims to integrate established software engineering practices with AI capabilities to tackle more complex problems.

This talk introduces the AI Engineering Code Summit, emphasizing AI coding as the most critical problem in technology today. It highlights the event's focus on bringing together experts to advance the AI coding industry across all companies, distinguishing it as a single-track summit dedicated to this theme. The summit aims to explore the patterns, systems, and products enabling AI's transformation of software development.

This talk introduces an evaluation framework for AI product managers (PMs) focused on shipping reliable AI applications. It emphasizes the critical need for robust evaluation, analogous to software testing but adapted for the non-deterministic nature of AI models. The framework aims to provide PMs with tools and methodologies to build confidence in AI product performance, moving beyond subjective "vibe coding" to data-driven "thrive coding."

This talk introduces Google DeepMind's latest advancements, Gemini 3 Pro and Nano Banana Pro, highlighting their capabilities in building AI-powered applications. The core thesis is that these models empower anyone to build software by simplifying complex tasks through intuitive interfaces and powerful AI features, enabling a new generation of engineers to create at an unprecedented scale.

Anti-gravity is a new AI developer platform from Google DeepMind designed with an agent-first approach. It integrates an editor, a browser, and an agent manager to provide a cohesive environment for AI-powered software development. The platform aims to leverage advancements in AI models, particularly in areas like reasoning, multimodal capabilities, and tool use, to enable more ambitious and complex agentic workflows.

This talk introduces Z.ai's GLM 4.6 model series, highlighting its advancements in both language and multimodal understanding. The series has achieved over 100 million downloads across various platforms, indicating significant community adoption and contribution. GLM 4.6 demonstrates strong performance on public benchmarks, rivaling and sometimes surpassing commercial models, particularly in coding and reasoning tasks.

This talk explores the evolving landscape of AI engineering, emphasizing the need for new standard models to guide the development of AI-intensive applications. The speaker posits that the field is moving beyond simple wrappers and demos towards robust production systems, drawing parallels to foundational periods in physics and other engineering disciplines. The core thesis is that identifying and adopting these new standard models will be crucial for building valuable and intelligent AI products.

This talk addresses the current state of AI vision, arguing that computer vision models lag significantly behind language models in terms of intelligence and pre-training leverage. The core thesis is that vision models are not yet "smart" due to limitations in evaluation metrics, a lack of effective large-scale pre-training utilization, and challenges in aligning visual and linguistic features. The presentation introduces new benchmarks and models aimed at improving vision AI's capabilities.

This talk addresses the challenges of building and shipping products in the rapidly evolving AI agent space, where the capabilities of the underlying LLMs are not fully understood. It proposes a shift in product development from defining specific requirements to focusing on affordances and emergent behaviors, emphasizing the need for new tools and practices to navigate this uncertainty. The core thesis is that product management must transform to embrace discovery and adaptation in a probabilistic world.

The core thesis is that successful product development, especially in the age of AI, hinges on rapid, iterative shipping and continuous user feedback, rather than large, infrequent releases. The speaker advocates for building "skateboard" equivalents—minimally viable products that deliver immediate value and allow for quick iteration based on real user input, which is far more effective than a traditional, linear development approach.

This talk argues for the critical role of an AI product manager, emphasizing that AI expertise is essential for this position. It builds on the idea that as AI coding agents and product features reduce the cost of software development, the demand for individuals who can effectively decide *what* to build will increase. The core thesis is that a dedicated AI product manager, or at least an AI product management mindset within a team, is crucial for navigating the complexities of AI product development.

This talk argues that the current era of AI development is characterized by "ugly" or clunky products, which are merely the precursors to a more refined future. The core thesis is that building truly great AI products requires deep personal clarity, a singular focus on purpose, and a commitment to building user trust, which ultimately enables delight. The speaker emphasizes that restraint and judgment are key innovation multipliers in a landscape often dominated by overwhelming capabilities.

This talk introduces the concept of an AI co-scientist, analogous to a pair programmer but for real-world scientific experiments. The system integrates various sensors and cameras to collect empirical data in real-time, which an AI then analyzes to provide insights, generate hypotheses, and potentially accelerate scientific discovery. The presented system is built using open-source hardware and software, demonstrating a low-cost, accessible approach to AI-assisted experimentation.

This talk argues that the future of software development is agent-driven, moving beyond human-driven approaches. It posits that AI agents, when properly integrated and provided with sufficient context, can handle a majority of tasks across the software lifecycle, enabling production software to be built in minutes rather than months. The core thesis is that organizations must transition to an agent-native development paradigm to unlock the true power of AI.

This talk addresses the gap between AI-generated code prototypes and production-ready software. It introduces Sculptor, an experimental coding agent environment designed to build trust in AI-generated code by focusing on identifying and preventing defects. The core thesis is that AI should be leveraged not just for code generation but also for ensuring its quality and reliability.

This talk explores the practical application and effectiveness of software development agents, emphasizing the shift from manual coding to higher-level problem-solving. It argues that while AI excels at the iterative process of writing and running code, human engineers remain crucial for critical thinking, user empathy, and architectural decisions. The presentation details the core components of these agents, their underlying mechanisms, and best practices for their integration into the development workflow.

This talk focuses on the practical aspects of building AI products that are reliable and effective, moving beyond theoretical discussions of evaluations. It emphasizes that while AI technology is advancing, the core challenge lies in effective communication and managing the inherent complexity and undefined behaviors of AI systems. The speakers advocate for an iterative approach to product development, where continuous refinement based on real-world user data and signals is crucial for success.

This talk explores the complexities of transitioning a SaaS company to an AI-first model, emphasizing that it's an evolutionary journey rather than a binary switch. It proposes a framework focusing on strategy, ways of working, and people to navigate this transformation. The core thesis is that becoming AI-first requires reimagining product development, embracing ambiguity, and fostering a cultural shift across the entire organization, not just within AI teams.

This talk argues for building dynamic, deeply integrated AI products rather than AI "sideshows" that are bolted onto existing systems. The core thesis is that companies should move beyond using AI primarily to demonstrate technological capability and instead focus on solving customer problems by integrating AI strategically into their core product development. This approach requires aligning AI and product strategies, teams, and roadmaps, and embracing a crawl, walk, run methodology for iterative development.

This talk explores the design principles for creating effective user experiences in semi-autonomous multi-agent systems. It emphasizes that while multi-agent systems offer powerful capabilities, their complexity requires careful consideration of user interaction, observability, and control. The presentation highlights a practical approach to building such systems, starting with defining the goal and tools before agent development, and advocates for an evaluation-driven design process.

This talk argues that fundamental principles of good design remain unchanged despite advancements in AI tools. Design is defined not by aesthetics but by the entirety of a user's experience across all touchpoints, influencing how they feel about a product. In the current AI landscape, where feature parity is no longer a differentiator, design becomes the key element for creating unique and memorable user experiences.

This talk explores how to design AI products and services with language models as a primary audience, drawing lessons from Andrej Karpathy's experience building the Menuguen app. It emphasizes the shift towards LLMs consuming structured data like markdown and API schemas, rather than just human-readable web pages. The core thesis is that embracing these formats and improving developer experience is crucial for building successful AI-powered applications.

This talk details the process of developing and shipping Finn Voice, an AI-powered voice agent for customer support, within a 100-day timeframe. The core thesis is that voice AI represents a significant frontier in customer service, offering substantial benefits in cost savings, availability, and user experience compared to traditional phone support. The presentation emphasizes that successful deployment requires more than just advanced AI models; it necessitates a product-centric approach that considers use cases, conversation design, integration with existing workflows, and building user trust.

In a rapidly evolving AI landscape, building successful products requires a strategic approach beyond simply integrating AI features. The core thesis is that competitive advantage stems not from the AI technology itself, but from the unique combination of a product's proprietary data, its specific functionality, and a deep understanding of unmet customer needs. This talk emphasizes treating AI as modular components, or "Lego blocks," that can be assembled to create differentiated offerings.

This talk introduces the Google Gemini 2.5 model family, focusing on practical applications for AI engineers and builders. It highlights the multimodal capabilities of Gemini 2.5 Pro and Flash, demonstrating how to leverage these models for text generation, image and audio understanding, function calling, and integrating with external tools via MCP servers. The session emphasizes hands-on learning through a workshop format, encouraging attendees to experiment with the models and SDK.

The talk argues that as AI tools increasingly automate code development, the complexity of system design and production troubleshooting will grow, potentially leading engineers to spend more time on on-call duties. The core thesis is that current approaches to AI-assisted troubleshooting are insufficient and a new, more integrated method is needed to handle the escalating complexity of production incidents.

This talk explores the concept of "thinking" within Gemini, a new paradigm for AI models that allows for iterative computation and deeper reasoning before generating a final response. The core thesis is that by enabling models to spend more test-time compute on complex problems, we can unlock significant advancements in AI capabilities, moving beyond immediate response generation towards more profound problem-solving. This approach addresses bottlenecks in current AI by allowing for dynamic allocation of computational resources based on task difficulty.

Logan Kilpatrick of Google DeepMind discussed a year of progress with Gemini models and outlined future developments. The talk highlighted the release of a new Gemini 2.5 Pro model, emphasizing its improved performance across benchmarks and its role as a turning point for the Gemini family. Kilpatrick also touched upon the organizational shifts within Google that have integrated research and product teams, enabling faster delivery of AI capabilities to both consumers and developers.

This talk addresses the common challenge of AI product concepts failing to reach their full potential due to operational difficulties. It emphasizes that bridging the gap between product vision and AI operational reality requires a deep understanding of how to deliver quality through rigorous evaluation, human review, and strategic team building. The core thesis is that successful AI product development hinges on a robust operational foundation that supports continuous iteration and quality assurance.

This talk focuses on optimizing inference for voice models in production, emphasizing runtime performance and infrastructure considerations. It highlights how the architectural similarity between Text-to-Speech (TTS) models and Large Language Models (LLMs) allows for the application of LLM optimization techniques. The core thesis is that while runtime optimizations are crucial, non-runtime factors like infrastructure and client code implementation can significantly impact overall latency and cost-efficiency.

This talk focuses on mastering AI evaluation, moving from initial development to production. It emphasizes that even the best large language models (LLMs) require a robust testing framework due to issues like hallucinations and performance degradation with changes. Effective evaluation helps answer critical questions about model selection, cost-effectiveness, brand consistency, and ongoing improvement, ultimately reducing development time, costs, and enabling faster iteration.

This talk focuses on building production-ready AI agents, moving beyond simple prototypes to scalable, cloud-hosted solutions. It outlines the essential components of an agent, including the model, prompt, loop, history, and tools, and demonstrates how to leverage AWS services like Amazon Bedrock Agents to deploy these agents at cloud scale. The presentation emphasizes practical steps for integrating custom tools via Lambda functions and preparing agents for a production environment.

This talk, presented by Sarah Sachs of Notion AI and Carlos Esteban of Braintrust, focuses on the critical role of observability and rigorous evaluation in building world-class AI products. The core thesis is that the quality and scalability of AI products stem directly from robust evaluation processes, which allow teams to iterate effectively and ensure consistent performance beyond simple demos.

This talk introduces Contextual AI's platform for building production-ready Retrieval Augmented Generation (RAG) agents quickly, aiming to simplify the RAG pipeline. The core thesis is that RAG can be treated as a managed service, abstracting away the complexities of building and maintaining individual components like vector databases and LLM training. The platform offers an end-to-end solution for ingesting data, retrieving relevant information, and generating grounded responses, with a focus on enterprise-grade accuracy and modularity.

This talk introduces LlamaIndex as a framework for building generative AI applications, with a particular focus on agents. It emphasizes the necessity of Retrieval Augmented Generation (RAG) for agents to effectively process and utilize large, unstructured datasets. The presentation outlines several high-level design patterns that enhance agent performance in production environments.

This talk explores the complexities of pricing AI products, emphasizing that pricing is a form of friction that must be carefully managed to align with product value and audience needs. It moves beyond traditional pricing models to discuss AI-native considerations like predictability, speed of value demonstration, and rapidly changing cost structures. The core thesis is that effective AI product pricing requires a deep understanding of the target audience, value delivery mechanisms, and flexible margin structures, allowing for continuous experimentation and adaptation.

This talk introduces Google DeepMind's latest generative media models, focusing on V3 for video and audio generation, Imagine 4 for static images, and LIA 2 for music. The presentation highlights how these tools can revolutionize content creation, advertising, and user experiences by enabling the generation of novel content and offering enhanced creative control. The models are designed to improve stylistic and contextual consistency, making them powerful tools for developers and creators.

This talk discusses the challenges and solutions for implementing and scaling Message Communication Protocol (MCP) clients within a large organization. It highlights how the rapid evolution of AI models capable of tool calling led to integration chaos, with duplicated functionality and inconsistent interfaces across services. The solution proposed is to standardize on MCP for providing model context, treating it as a plumbing layer for integrations rather than a competitive differentiator.

The Model Context Protocol (MCP) is an open-source, standardized protocol designed to give AI models agency by allowing them to interact with the outside world. Originating from the need to copy context from external sources into LLM context windows, MCP aims to enable models to reach a new level of usefulness and intelligence by facilitating tool calling and broader interaction capabilities. The protocol prioritizes server simplicity and encourages community contributions to evolve its standards and utility.

This talk outlines four key patterns emerging in AI-native development, shifting the developer's role beyond traditional coding. As AI tools evolve from simple code generation to complex agent teams, developers are transitioning into roles like managers, specifiers, discoverers, and knowledge curators. This evolution signifies a move towards more strategic and less purely executional work, fundamentally changing the software development lifecycle.

This talk explores the concept of a voice-first AI overlay designed to enhance human-to-human conversations by providing real-time assistance without directly participating as a third speaker. The core idea is to leverage advancements in AI agent capabilities and voice technology to keep humans informed and on track during interactions, using voice as the most natural interface.

This talk chronicles a founder's transition from a product role at a large tech company to establishing an AI startup. It highlights the unique challenges of identifying user problems in the nascent AI space, where problems are emergent rather than clearly defined. The presentation emphasizes the need for rapid iteration, hypothesis-driven development, and the creation of user narratives to navigate this evolving landscape.

This talk proposes using AI simulation to accelerate design iteration by creating "intelligent twins" that act as invisible users. This approach aims to overcome the current AI trust gap, which stems from poorly implemented AI features, by enabling designers to identify and address user pain points more effectively. The methodology draws parallels to pilot training through simulation, suggesting a shift from traditional data collection to active AI-driven feedback loops within the design process.

This talk argues that ChatGPT's user interface is poorly designed, leading to a confusing experience despite its rapid growth. The presenter demonstrates issues with its voice and text interaction, highlighting how separate functionalities feel disconnected. The core thesis is that by integrating multimodal capabilities and intelligently routing requests to appropriate models, the user experience can be significantly improved using off-the-shelf tools.

This talk proposes a paradigm shift in AI interface design, moving from pure automation to augmentation that enhances human capabilities. Instead of AI systems attempting to automate complex tasks suboptimally, the focus should be on using AI to help humans produce higher quality work. This approach emphasizes interaction patterns that help users identify blind spots, foster creativity, and amplify thoughtful decision-making, ultimately aiming for trustworthy human-AI partnerships that grow with the user.

This talk, Frontier Feud, hosted by Barr Yaron of Amplify, uses a game show format to explore the opinions and priorities of 100 AI engineers. Teams comprised of individuals from Anthropic, Google DeepMind, Meta FAIR, and Thinking Machines competed by guessing survey answers on topics like influential researchers, model selection criteria, and AI buzzwords. The event highlighted differing perspectives on AI development and its future impact.

This talk details how Method scaled its AI agent operations to over 500 million agents with a small engineering team. Initially, they faced challenges with manual data aggregation processes and later with the high costs and limitations of using large language models like GPT-4 for parsing unstructured financial data. The solution involved fine-tuning smaller, more efficient models to meet specific production requirements for accuracy, latency, and cost.

This talk introduces the Agent Development Life Cycle (ADLC) as a structured approach to building and improving AI agents, drawing parallels to traditional software development. It emphasizes that agents are products requiring a robust platform for iterative refinement, customer feedback integration, and continuous improvement, much like a mobile app or website. The ADLC aims to leverage the strengths of large language models while incorporating traditional software practices for reliability and efficiency.

This talk focuses on the practical challenges and strategies for scaling generative AI agents in product development, particularly within a large financial data organization. It emphasizes the need for robust engineering practices, acknowledging the inherent fragility and evolving nature of LLMs and agentic systems. The core thesis is that building reliable AI products requires moving beyond basic LLM capabilities to implement structured development, resilient architectures, and clear organizational design.

This talk details the development of Deep Research, a personal research agent integrated into Gemini Advanced. The core thesis is that by removing compute and latency constraints at inference time, AI models can perform extensive web browsing to generate comprehensive answers, addressing the limitations of traditional chatbots that often provide blueprints rather than direct solutions. The presentation highlights the product and technical challenges encountered in building such an asynchronous, long-form research tool within a synchronous chatbot interface.

The AI landscape in 2025 is experiencing exponential growth, with numerous companies releasing increasingly performant and efficient models. While this presents a perfect storm for AI agents, they are not yet fully autonomous or reliable. Cumulative errors in decision-making, implementation, heuristics, and user preferences hinder their effectiveness, leading to unmet expectations despite advancements.

This talk introduces a test-driven development (TDD) approach for building reliable AI agentic workflows. It argues that while AI models are improving, success in production hinges on robust engineering practices. The presentation outlines a structured process for experimenting, evaluating, scaling, and continuously monitoring AI solutions, emphasizing the need for a systematic approach to manage complexity and ensure reliability.

This talk explores the concept of personality-driven development in AI agents, where agents are given distinct forms and personalities to enhance user interaction and understanding. The speaker argues that anthropomorphizing AI, while not new, becomes particularly relevant with modern agents, influencing user perception and expectations. This approach aims to simplify complex AI functionalities by mapping them to familiar human roles and traits, thereby improving product development and user experience.

This talk focuses on the advantages and practicalities of using open-source models for production AI applications. It highlights how custom-tuned, smaller models can outperform larger, general-purpose proprietary models in terms of speed, cost, and domain-specific accuracy. The presentation also addresses the challenges of deploying open-source models, such as setup complexity and optimization, and introduces Fireworks AI's platform designed to streamline these processes.

This talk focuses on scaling Retrieval Augmented Generation (RAG) applications for production use, specifically highlighting the capabilities of Azure AI Search. It addresses the challenges that arise when moving from prototypes to production, such as increased data volume, higher data change rates, and more complex multi-step workflows. The presentation emphasizes how Azure AI Search provides integrated solutions for these scaling dimensions, including advanced vector search, hybrid search, and efficient data ingestion.

This talk discusses the system design challenges and future directions for running and training next-generation large language models. It highlights the significant computational and memory bandwidth requirements for inference, particularly for models with trillions of parameters. The presentation also touches upon the massive scale of infrastructure being built for training these models and the associated engineering hurdles.

This talk explores the rapidly evolving landscape of AI music generation, demystifying the technology and demonstrating practical applications. It covers various categories of AI music, including voice conversion, text-to-music, and audio-to-music generation, highlighting tools like Suno and Udio. The presentation aims to equip attendees with the knowledge to transform musical ideas into finished songs and discusses the emerging opportunities for artists in this new domain.

This talk explores the relationship between the stakes of an AI application and the obstacles encountered during production. Higher stakes, defined by potential impacts on finances, reputation, health, or safety, correlate with increased challenges. The discussion contrasts vector-based retrieval-augmented generation (RAG) with knowledge graphs, suggesting that structured data and reasoning from knowledge graphs can enhance reliability for high-stakes use cases.

This talk details Zapier's approach to building AI-powered products, emphasizing an iterative development process heavily reliant on robust evaluation and observability. The speakers share their journey of integrating AI features, highlighting the challenges and successes encountered when developing and refining tools like the AI Zap Builder and Zapier Copilot. Their strategy involves close collaboration between product and engineering teams, continuous testing, and leveraging platforms like Braintrust to ensure product quality and performance.

The core thesis is that while Generative AI models are easy to demo, productionizing them presents significant challenges. The talk emphasizes that the AI development process is fundamentally experimental and non-deterministic, unlike traditional software development. This necessitates a robust approach to tracking learnings, ensuring reproducibility, and building comprehensive evaluation frameworks to move AI applications from demo to production successfully.

This talk addresses the critical importance of training data development for large language models, emphasizing that the quality and format of data directly impact model performance. It proposes a hierarchy of needs for dataset development, starting with clean data and progressing through evaluations, dataset management, and advanced techniques like synthetic data generation and quality scoring. The discussion highlights the challenges posed by the increasing scale and multimodality of AI data and introduces the Lance format as a solution for efficient data handling.

This talk explores the evolution of product design, moving from traditional screen-based interfaces to "unbounded products" that transcend typical input methods. It emphasizes the challenges of unpredictability and user confusion in these new paradigms and proposes design structures to create clarity and familiarity. The discussion covers historical precedents, current AI product patterns, and future interface directions.

This talk explores how programmers can leverage Large Language Models (LLMs) to significantly boost their productivity, aiming for a "10x" improvement. The core thesis is that LLMs should be treated not as autonomous reasoning agents, but as powerful translation engines and world simulators. By understanding how to decompose problems into language translation steps and by creatively framing prompts, developers can unlock new levels of efficiency and innovation.

This talk introduces MAX, an AI framework designed to enhance developer productivity and performance across both CPU and GPU hardware. It addresses the fragmentation and complexity in the current AI development landscape, aiming to provide a unified, high-performance solution that allows developers to own and control their AI models and data. MAX focuses on inference and aims to simplify the deployment of PyTorch models and generative AI applications.

This talk focuses on building production-ready Retrieval Augmented Generation (RAG) applications, addressing the limitations of naive RAG implementations. It emphasizes that while RAG is powerful for querying data, common issues like poor retrieval accuracy and low response quality hinder production use. The presentation outlines strategies for improving RAG performance across the entire pipeline, from data ingestion to synthesis, with a strong emphasis on evaluation and iterative optimization.

This talk outlines essential building blocks for developing effective Large Language Model (LLM) systems and products. It emphasizes the critical role of evaluations, retrieval augmented generation, guardrails, and feedback collection in the LLM development lifecycle. The core thesis is that a structured, iterative approach focusing on these components is key to building robust and reliable LLM applications.