Zubin Koticha

This talk addresses the critical need for agent observability in AI systems, highlighting how traditional testing and evaluation methods are insufficient for non-deterministic and complex agents. It proposes a shift towards a monitoring paradigm, emphasizing the importance of collecting both explicit signals (like error rates and latency) and implicit signals (like user frustration and refusals) to understand and improve agent behavior in production. The discussion also touches upon self-diagnostics as a low-effort method for agents to report their own issues.