World's Fair 2024
EyeLevel Launch: Your RAG is Tripping, Here's the Real Reason Why
Overview
This talk addresses common accuracy issues in Retrieval Augmented Generation (RAG) applications, particularly with complex enterprise documents. The core thesis is that RAG failures are typically due to content ingestion problems rather than LLM or prompt issues. The presented solution focuses on a novel ingestion pipeline that preserves crucial context lost during traditional chunking and vectorization, leading to significantly improved accuracy.
Who should watch
- AI Engineers building RAG applications
- Product Managers evaluating RAG solutions
- Builders struggling with RAG accuracy and hallucination rates
- Those working with complex or visually rich documents
Key takeaways
- RAG applications can suffer from high error rates (up to 35%) due to issues with retrieved content quality and relevance.
- Common content problems include improperly extracted text, loss of contextual information during chunking, and failure to extract visual elements.
- The presented approach bypasses traditional vector databases, instead creating "semantic objects" with extracted text and auto-generated metadata.
- This method preserves context lost during chunking, such as the document section or origin.
- Text is rewritten into optimized formats for both search and completion.
- A multi-field search is performed across the semantic object's attributes, including original text, metadata, and search-optimized text.
- Fine-tuned LLMs are used to re-rank search results, further enhancing accuracy.
- The platform claims to achieve over 95% accuracy in production and outperformed popular solutions by up to 120% in a recent study.
Notable quotes
*RAG applications can have error or hallucination rates as high as 35%*
*The source of these errors is rarely the LLMs or the prompts. Instead, it's typically rag itself*
*Most commonly, the problems with rag are content ingestion problems.*
Unofficial community note. Prefer the recording for nuance.