Benchmarks Are Memes: How What We Measure Shapes AI—and Us - Alex Duffy, Every.to
This talk argues that AI benchmarks function as memes, spreading ideas and shaping the development of AI models. As benchmarks become popular, models are trained and tested on them until saturation occurs. This cycle presents an opportunity for individuals to create new benchmarks that can influence the future direction of AI development, emphasizing the importance of human-defined goals and values in the process.
World's Fair 2025 16 min