Philipp Krenn

This talk explores the fundamentals of information retrieval, focusing on the "R" in Retrieval Augmented Generation (RAG). It delves into both traditional keyword search and modern vector search, explaining their underlying mechanisms, strengths, and limitations. The presentation emphasizes that effective retrieval is crucial for accurate and relevant results in AI applications.

This talk addresses the common pitfalls and deceptive practices in vector search benchmarking, often referred to as "benchmarketing." The core thesis is that many published benchmarks are misleading due to biased scenario selection, outdated competitor versions, and a lack of focus on crucial factors like data freshness and query relevance. The speaker emphasizes that truly meaningful benchmarks require rigorous, reproducible, and self-conducted evaluations tailored to specific use cases.