Agents in Production: How OpenGov Built and Scaled OG Assist - Gabe De Mesa, OpenGov

This talk argues that the core challenge in software engineering is shifting from writing code to managing complexity, designing systems, and ensuring correctness. While AI models now excel at generating code syntax, the future of software development lies in leveraging AI for higher-level tasks like architectural design, complex problem decomposition, and inductive reasoning. The economics of code production are changing, making code generation nearly free and emphasizing the need for new processes and evaluation methods.

This talk addresses the critical gap between raw AI agent output and a user-friendly experience, arguing that a dedicated rendering layer is essential for LLM pipelines. It emphasizes that simply displaying raw LLM responses fails to meet user expectations for intuitive and effective interaction. The core thesis is that building a robust rendering layer is key to successfully shipping AI-powered products.

This talk outlines a repeatable framework for designing and building AI systems from initial idea to production. It emphasizes that in the current AI landscape, defining product requirements, system design, and evaluation criteria are the critical, challenging aspects, rather than just coding. The framework consists of four phases: product requirements, system design, evaluation and monitoring, and optimization.

This talk introduces design patterns for building trust in AI agents, drawing parallels to human collaboration and decision-making processes. The core thesis is that managing AI agents effectively involves applying principles similar to how teams of humans are managed, particularly in complex or ambiguous situations. The presentation emphasizes that AI agents, like people, benefit from clear guidance, access to knowledge, and structured evaluation methods to ensure reliable and trustworthy outputs.

This talk emphasizes the critical need for rigorous evaluation of AI skills before deployment, arguing that shipping skills without proper testing can lead to unpredictable behavior and degraded performance. The speaker highlights the distinction between agents developers use for personal productivity and agents built for consumers, noting that end-users lack the context to troubleshoot skill invocation issues. The core thesis is that comprehensive evaluations are essential for ensuring skill reliability, managing costs, and determining when skills can be retired as models improve.