World's Fair 2025
GraphRAG methods to create optimized LLM context windows for Retrieval — Jonathan Larson, Microsoft
Overview
This talk introduces GraphRAG, a method for optimizing Large Language Model (LLM) context windows using graph structures for retrieval. The core thesis is that LLM memory with structure is a key enabler for building effective AI applications, and when paired with agents, this combination offers even greater power. The presentation demonstrates GraphRAG's application in code understanding and feature development, alongside the announcement of Benchmark QED, an open-source tool for evaluating LLM systems.
Who should watch
- AI engineers working with LLMs and codebases.
- Product Managers seeking to understand advanced AI capabilities for product development.
- Builders looking for methods to improve LLM context understanding and code translation.
- Researchers interested in LLM evaluation and structured memory techniques.
- Developers aiming to enhance enterprise productivity through AI-assisted coding.
Key takeaways
- LLM memory enhanced with structural information, like graphs, is crucial for building effective AI applications.
- GraphRAG applied to codebases enables repository-level understanding, significantly improving LLM comprehension compared to standard RAG.
- GraphRAG facilitated accurate code translation from Python to Rust for a small game and enabled feature development, such as adding a jump capability to the Doom codebase, by providing holistic understanding.
- Benchmark QED is a new open-source evaluation suite comprising AutoQ (query generation), AutoE (evaluation using LLMs as judges), and AutoD (dataset summarization and sampling).
- Lazy Graph, a new technology, demonstrates dominant performance in benchmarks against vector RAG, even with long context windows, and at a fraction of the cost.
- Lazy Graph is being integrated into Azure services and the Microsoft Discovery platform for graph-based scientific co-reasoning.
- Agents, when combined with structured LLM memory, can massively amplify AI capabilities, leading to more robust and successful outcomes in complex tasks.
Notable quotes
*LLM memory with structure is just an absolutely key enabler for building effective AI applications.*
*Agents of course paired with these structures can provide something even more powerful.*
*LLM memory with structure is just a really really powerful tool to keep in your tool belt.*
Unofficial community note. Prefer the recording for nuance.