Session brief
Mission-Critical Evals at Scale (Learnings from 100k medical decisions)
Overview
This talk outlines the challenges and solutions for building scalable evaluation systems for AI, particularly in mission-critical domains like healthcare where errors have significant consequences. It emphasizes the limitations of traditional human review and offline datasets, advocating for real-time, reference-free evaluation methods to ensure customer trust and enable rapid response to issues.
Who should watch
- AI Engineers
- Product Managers
- Builders working with LLMs at scale
- Those facing challenges with AI model performance and reliability
- Individuals building systems for decision-making in high-stakes environments
Key takeaways
- Scaling AI products reveals edge cases and errors that are not apparent in MVP stages.
- Relying solely on human reviews becomes prohibitively expensive and unmanageable as request volume increases.
- Offline evaluation datasets are valuable but can be too slow to identify new issues emerging in live production.
- Real-time, reference-free evaluation, where the outcome is assessed before a known true result, is crucial for immediate issue detection and response.
- An LLM can be used as a judge to provide confidence scores or predicted outcomes for AI-generated responses.
- These reference-free evaluations can dynamically prioritize cases for human review based on factors like predicted error probability and cost.
- This approach enables a virtuous cycle of continuous improvement, validating AI performance against human reviews and refining the evaluation system itself.
- Implementing such a system allows for confident deployment of AI outputs or triggers further actions like escalation to more expensive models or human review.
Notable quotes
*We've all seen that it's pretty easy to create an MVP product powered by LLMs and it's getting even easier as models get more and more powerful but what about going from MVP to serving customers at scale.*
*Relying only on offline evals is playing with fire.*
*The principles that we followed for building our system and what we would recommend is firstly make sure you build a system you know think big don't just use review data to to audit your performance use it to build audit and improve your auditing system your evaluation system.*
Unofficial community note. Prefer the recording for nuance.