World's Fair 2025
When Vectors Break Down: Graph-Based RAG for Dense Enterprise Knowledge - Sam Julien, Writer
Overview
This talk explores the limitations of traditional vector-based Retrieval Augmented Generation (RAG) for complex enterprise knowledge and proposes a graph-based RAG approach. It highlights how preserving relationships within data through knowledge graphs, combined with advanced techniques like fusion-decoder, significantly improves accuracy and reduces hallucinations in AI applications, especially for dense, specialized datasets.
Who should watch
- AI Engineers
- Product Managers
- Builders working with large, specialized enterprise datasets
- Those experiencing accuracy issues or hallucinations with standard RAG systems
- Developers seeking to improve retrieval for complex knowledge domains
Key takeaways
- Vector search alone is often insufficient for sophisticated retrieval in enterprise RAG, particularly with dense or specialized data.
- Graph-based RAG preserves relationships between data points, providing richer context to LLMs and improving accuracy.
- Challenges with early graph-based RAG implementations included data structuring costs, scaling limitations, and difficulties with advanced similarity matching in graph query languages.
- Solutions involved developing specialized models for text-to-graph conversion and storing graph structures as JSON in search engines for better scalability.
- The fusion-decoder technique, originally proposed in the RAG paper, offers efficiency gains by processing passages independently in the encoder and jointly in the decoder.
- Combining knowledge graphs with fusion-decoder further enhances retrieval by understanding relationships between passages, leading to state-of-the-art performance.
- Benchmarking showed this graph-based RAG system outperformed seven different vector search systems in accuracy and response time.
- Key principles for success include focusing on customer problems over hyped solutions, staying flexible based on team expertise, and allowing research to challenge assumptions.
Notable quotes
*Vector search alone is just insufficient for sophisticated retrieval and that we're going to need multiple strategies beyond simple vector similarity.*
*The main takeaway being as you're seeing in several of these talks like the first talk about hybrid search there are many different ways that you can get the benefits of knowledge graphs in rag.*
*Our research team... they're very focused on solving practical problems for our customers. They're not just sort of like working in isolation working on theoretical things.*
Unofficial community note. Prefer the recording for nuance.