World's Fair 2025
Transforming search and discovery using LLMs — Tejaswi & Vinesh, Instacart
Overview
This talk details how Instacart is leveraging Large Language Models (LLMs) to significantly enhance its search and discovery functionalities. The core thesis is that LLMs can overcome limitations of traditional search models, particularly for tail queries and new item discovery, by better understanding user intent and context, ultimately leading to improved user experience and business metrics.
Who should watch
- AI Engineers and ML practitioners working on search and recommendation systems.
- Product Managers and builders looking to improve e-commerce search relevance and discovery.
- Teams facing challenges with cold-start problems, low-precision results, or enabling new item discovery.
- Those interested in applying LLMs to real-world e-commerce problems.
Key takeaways
- LLMs can significantly improve query understanding, especially for infrequent or broad tail queries, by providing better semantic understanding than traditional models trained solely on engagement data.
- Augmenting LLM prompts with domain-specific knowledge, such as top converting categories or user behavioral data, is crucial for aligning LLM outputs with platform-specific user intent (e.g., distinguishing cookware from food for the query "dishes").
- LLMs can generate discovery-oriented content, like substitute or complementary items, to enhance the search results page and encourage further user engagement and larger basket sizes.
- A hybrid approach, pre-computing LLM outputs for common queries and falling back to existing models or distilled LLMs for tail queries, helps manage latency and cost while improving coverage.
- Effective evaluation of LLM-generated content and predictions is critical and often more challenging than anticipated, requiring robust automated pipelines and potentially using LLMs as judges.
- Integrating LLMs can simplify complex query understanding pipelines by consolidating multiple specialized models into a more unified system, leading to more consistent results.
- For queries like "protein," LLMs can be guided to understand Instacart-specific user behavior, such as searching for protein supplements rather than protein-rich foods.
- The use of LLMs for query rewriting helped reduce the number of queries returning no results by providing more relevant alternatives.
Notable quotes
*LLMs world knowledge was super important to improve query understanding predictions especially for the tail queries.*
*While LLMs were super helpful we really found success by combining the domain knowledge of Instacart with LLMs.*
*Evaluating the content as well as the cure predictions was far more important and far more difficult than we anticipated.*
Unofficial community note. Prefer the recording for nuance.