Doug Guthrie

This talk introduces the concept and practical application of evals for AI systems, emphasizing their role in ensuring quality, reliability, and correctness. Evals provide a structured testing framework to move beyond the non-deterministic nature of LLMs, enabling developers to rigorously build and iterate on AI applications. The presentation highlights how evals facilitate a feedback loop between offline development and online production monitoring, ultimately leading to improved AI products.