Alejandro Vidal

This talk proposes a shift from traditional, simplistic model evaluation methods to more sophisticated techniques borrowed from psychology and psychometrics. The core argument is that simply counting correct answers, a method akin to classical test theory, is insufficient. Instead, the presentation advocates for Item Response Theory (IRT) to provide a more nuanced understanding of model capabilities by calibrating individual questions and estimating model intelligence more accurately.