World's Fair 2026
Build Evals That Actually Matter - Nick Ung, Lyft
Overview
This talk addresses the common problem of AI evaluations that fail to predict real-world performance. The core thesis is that offline evaluations often use simplistic "customers" and test sets that don't reflect the complexity and adversarial nature of actual user interactions, leading to shipped models that fail in production. A solution involves building more realistic, adversarial user simulators trained on real data.
Who should watch
- AI Engineers
- Product Managers
- Builders of AI systems
- Those struggling with AI model performance discrepancies between offline tests and production
- Teams building customer-facing AI applications
Key takeaways
- Offline AI evaluations frequently fail because they use off-the-shelf LLMs as "customers" and synthetic test sets that miss the messiness of real conversations.
- Lyft developed an adversarial user simulator, a fine-tuned LLM trained on real rider and driver transcripts, to mimic frustrated, confused, and off-topic user behavior.
- This simulator identified regressions that synthetic datasets missed for months, enabling greater trust in AI agents handling a significant volume of customer issues.
- The evaluation lifecycle includes harness primitives for quick benchmark creation, calibrating LLM judges against human labels for inter-rater agreement, and routing production failures back into the offline test set.
- A continual learning loop is essential, feeding improvements into prompts, the evaluation harness, and the underlying models.
Notable quotes
*Your agent passes offline evals at 90%. You ship. Production immediately finds failure modes your eval never saw.*
*The culprit is almost always the same: the customer in your offline eval is an off-the-shelf LLM that sounds nothing like your real users.*
Unofficial community note. Prefer the recording for nuance.