How to run Evals at Scale: Thinking beyond Accuracy or Similarity — Muktesh Mishra, Adobe
This talk emphasizes the critical role of evaluations (evals) in developing robust AI applications, moving beyond simple accuracy or similarity metrics. It highlights that effective evals are fundamental for aligning applications with system goals, ensuring continuous improvement, and building user trust. The core thesis is that a strategic, data-driven approach to evals, tailored to specific use cases, is essential for scaling AI development.
World's Fair 2025 9 min