AI Copilots for Tech Architecture: The Highest-ROI Use Case You’re Not Building — Boris B., Catio

Cursor Composer is an AI model designed for real-world software engineering, aiming to balance speed and intelligence. It performs better than top open-source models and rivals frontier models in intelligence, while being significantly more efficient in token generation. The development focused on creating a model that could serve as a daily driver for coding tasks, integrating features like parallel tool calling and semantic search to enhance its capabilities.

This talk explores the evolution of evaluating AI models for coding tasks, from simple code snippets to complex codebases. It highlights challenges like data contamination and brittle test suites, proposing dynamic evaluation sets and LLM-based judges to ensure reliable and relevant assessments as AI capabilities advance. The discussion covers various stages of coding evaluation, emphasizing the need for benchmarks that reflect real-world performance and adapt to the rapid progress in AI.

This talk explores how to optimize developer experience in the era of AI coding agents. It argues that investments in foundational aspects of software development, such as standardized environments, robust validation, and clear documentation, will benefit both human developers and AI agents. The core thesis is that practices good for human developers are also good for AI, ensuring long-term value regardless of AI advancements.

This talk introduces Command Code, a coding agent designed to learn and adapt to an individual developer's unique coding style and preferences, referred to as "taste." Unlike standard AI coding assistants that provide generic code, Command Code aims to internalize a developer's decision-making process, preferences for tools, and architectural choices. This is achieved through a meta-neuro-symbolic reinforcement learning approach, creating a more personalized and efficient coding experience.

This talk explores the architecture and development of coding agents, emphasizing the crucial role of the "harness" – the interface layer that connects models to users and tools. It highlights the rapid evolution of AI models and the challenges in adapting agents, proposing that building models and harnesses together, as exemplified by OpenAI's Codeex, offers a more robust and future-proof approach. The discussion also touches upon emerging patterns for integrating these agents into various products and workflows.