Darius Emrani

This talk argues that current AI benchmarks are fundamentally flawed and often manipulated, leading to misleading performance claims. The speaker contends that the immense financial and market value tied to benchmark scores incentivizes companies to game the system through various deceptive practices. Ultimately, the presentation advocates for building custom, use-case-specific evaluations rather than relying on public benchmarks.