Europe 2026
How We Built Zeta2: Training an Edit Prediction Model in Production — Ben Kunkle, Zed
Overview
This talk details the process of training Zeta2, an edit prediction model for the Zed code editor. The core thesis revolves around using distillation from frontier models and leveraging opt-in production data to create a specialized, fast model for predicting user edits. The training pipeline emphasizes data processing, prompt engineering for teacher models, and a novel approach using "settled data" to generate high-quality training examples.
Who should watch
- AI Engineers
- Product Managers
- Developers building AI-powered coding tools
- Those interested in model distillation and production data pipelines
- Engineers working on real-time AI features in editors
Key takeaways
- Edit prediction models require specialized, fine-tuned models due to the need for low latency on every keystroke.
- The training pipeline uses distillation, where a frontier model acts as a teacher to predict edits based on context like recent edits, cursor position, and type definitions.
- A "repair step" is employed to refine teacher model predictions by sending problematic outputs to another frontier model for correction.
- Settled data, captured after a user stops editing a region, can be used for training, but requires careful filtering due to potential noise from user indecision or agent intervention.
- Levenshtein distance and similar metrics are used to evaluate prediction quality against settled states and teacher model outputs.
- The training process involves generating multiple teacher predictions and comparing them to the settled code to identify ideal training examples that are neither too obvious nor pure noise.
- Production experiments are monitored via dashboards tracking acceptance rates and latency, with models gradually rolled out to user traffic.
- The entire data pipeline is designed around JSONL format, allowing each stage to add or rearrange fields fluidly.
Notable quotes
*Edit prediction is essentially giving the model a region of code around the cursor, asking them to predict the next edit that you're going to make.*
*The nice part about this whole process, we've designed it in such a way that it's all JSONL or a single line is a giant JSON object.*
Unofficial community note. Prefer the recording for nuance.