World's Fair 2026
ReviewDebt: a practical framework for scoring every pull request — Sachin Gupta, Ebay
Overview
This talk introduces ReviewDebt, a framework for quantifying the accumulating gap between code generated by AI agents and the human review, trust, and understanding applied to it. It argues that while AI agents increase code production speed, they also create a form of debt that compounds and impacts human attention, architectural consistency, and velocity expectations. The framework aims to provide a measurable number for this debt, enabling teams to manage it effectively.
Who should watch
- AI engineers and developers working with AI coding assistants.
- Product Managers and technical leads concerned with code quality and review processes.
- Teams experiencing increased pull request volume but decreased review attention.
- Anyone looking to measure and manage the impact of AI-generated code on their development lifecycle.
Key takeaways
- AI agents are increasing code production volume, but human review activity (proxied by comments) is decreasing, creating a gap termed "review debt."
- Review debt compounds due to AI agents learning from existing codebases, reviewers focusing on syntax over architecture, and velocity expectations resetting without proportional increases in review capacity.
- The ReviewDebt framework uses five deterministic signal families—dev size and coupling, test evidence gap, directory and ownership spread, AI authorship indicators, and evidence and rational gaps—to calculate a review debt score.
- These signals are computable without language models, ensuring scores are stable and defensible.
- AI authorship indicators are a signal, but the framework emphasizes that complexity and PR shape, not just AI authorship, drive review burden.
- The framework provides structured advice for authors and reviewers, moving conversations from subjective feelings to objective measurements.
- Teams should adopt the framework by backfilling scores, setting thresholds, surfacing scores as PR comments, aggregating data weekly, and discussing it in team meetings.
Notable quotes
*Your coding agent is creating review debt.*
*Code production volume actually went up, but the review attention went down.*
*Review debt. It's the accumulating gap between code your agent has produced and the code humans have actually reviewed, trusted, and understood.*
Unofficial community note. Prefer the recording for nuance.