World's Fair 2025
Stop Using RAG as Memory — Daniel Chalef, Zep
Overview
This talk argues against using Retrieval-Augmented Generation (RAG) as a generic memory solution for AI agents. Instead, it proposes modeling memory after specific business domains to create more cogent and capable memory systems. The core thesis is that semantic similarity alone is insufficient for accurate memory recall, and domain-aware memory structures are necessary.
Who should watch
- AI engineers building agents and applications.
- Product Managers seeking to improve AI product capabilities.
- Developers struggling with AI memory accuracy and relevance.
- Builders looking for alternatives to simplistic RAG implementations.
Key takeaways
- RAG often struggles with relevance, leading to inaccurate responses or hallucinations by storing arbitrary facts.
- Semantic similarity in vector databases is not equivalent to business relevance, as it lacks causal or relational understanding.
- Memory should be modeled after the specific business domain, defining custom entities and relationships.
- Zep's Graffiti framework allows developers to build domain-specific memory using custom entities and edges.
- Developers can define explicit business objects (e.g., financial goals, income sources) with defined schemas and business rules.
- Tools can be built to retrieve specific data snapshots by querying and filtering by node types within the memory graph.
- Registering custom objects with Zep builds an ontology in the graph, enabling precise data retrieval.
- A paper detailing this approach is available for further reading.
Notable quotes
*Semantic similarity is not business relevance.*
*We need domain-aware memory not better semantic search.*
Unofficial community note. Prefer the recording for nuance.