Code 2025
No Vibes Allowed: Solving Hard Problems in Complex Codebases – Dex Horthy, HumanLayer
Overview
This talk addresses the challenges of using AI for complex software engineering tasks, particularly within large, existing codebases. The core thesis is that current AI models struggle with brownfield projects, often leading to rework and "slop." The presented solution focuses on advanced context engineering and intentional compaction to maximize the effectiveness of AI coding agents, enabling them to tackle difficult problems without introducing technical debt.
Who should watch
- AI engineers and builders working with complex or legacy codebases.
- Product Managers and engineers looking to improve AI integration in software development.
- Developers struggling with AI-generated code quality and rework.
- Teams aiming to increase AI-assisted development throughput without sacrificing quality.
Key takeaways
- AI coding agents perform poorly on complex, brownfield codebases, often resulting in significant rework and "slop." Greenfield projects or simple tasks are more amenable to current AI capabilities.
- Effective AI use in software engineering requires advanced context engineering to manage the limited context window of LLMs.
- Intentional compaction of context, such as summarizing existing context into markdown files, helps agents get up to speed quickly and stay within productive "smart zones" of the context window.
- The "dumb zone" of an LLM's context window, typically beyond 40% utilization, leads to diminishing returns and poor outcomes.
- Sub-agents can be used to control context by forking new windows for specific tasks and returning succinct information to a parent agent, rather than anthropomorphizing roles.
- A workflow of Research, Plan, and Implement (RPI) is proposed, emphasizing constant context management and staying within the smart zone to maintain correctness and completeness.
- Mental alignment within a team is crucial, and detailed plans with code snippets can facilitate this, acting as a form of AI-assisted code review.
- AI amplifies existing thinking; it cannot replace human thought. Developers must remain in the loop to ensure the accuracy of research and plans, as errors here can lead to significant downstream issues.
Notable quotes
*The most naive way to use a coding agent is to ask it for something and then tell it why it's wrong and resteere it.*
*AI cannot replace thinking. It can only amplify the thinking you have done or the lack of thinking you have done.*
*A bad line of code is a bad line of code and a bad part of a plan is could be a hundred bad lines of code.*
Unofficial community note. Prefer the recording for nuance.