World's Fair 2024
Build, Evaluate and Deploy a RAG-Based Retail Copilot with Azure AI: Cedric Vidal and David Smith
Overview
This talk demonstrates how to build, evaluate, and deploy a Retrieval-Augmented Generation (RAG) based copilot for a retail environment using Azure AI services. The core concept involves creating a chatbot that can answer customer questions by accessing product information from a vector database and customer data from a relational database, all orchestrated through Azure AI Studio's Prompt Flow.
Who should watch
- AI Engineers looking to build practical RAG applications.
- Product Managers interested in AI-powered customer service solutions.
- Builders seeking to understand the end-to-end process of developing and deploying LLM-based applications.
- Developers wanting to leverage Azure AI services for AI solutions.
Key takeaways
- A RAG-based copilot can be built using Azure AI Search for product data and Azure Cosmos DB for customer information.
- Prompt Flow in Azure AI Studio is a key tool for orchestrating the RAG process, managing data retrieval, prompt augmentation, and LLM response generation.
- The architecture involves embedding user queries into vectors to find relevant products, retrieving customer data, and combining this information into a prompt for the LLM.
- Evaluation of the chatbot's performance is crucial, using metrics like relevance, groundedness, and coherence, potentially with a more powerful LLM like GPT-4 to score the responses of a production LLM like GPT-3.5 Turbo.
- The development process includes setting up environments via GitHub Codespaces, connecting to Azure resources, and deploying the Prompt Flow as an endpoint.
- For production, a hybrid search combining keyword and vector search in Azure AI Search is recommended for better performance and recall.
- Multimodal capabilities are emerging, allowing LLMs to process inputs beyond text, such as images, with models like GPT-4o and Florence 2.
Notable quotes
*The whole purpose is to create one endpoint that goes through in this case a rag process but it's designed to be more flexible than just for Rag.*
*Vector surge by itself actually isn't enough... a combination of keyword search and Vector search together actually outperforms either.*
*We think of debugging in just regular apps and tests... much more difficult test to answer the question was is the answer generated by my chatbot relevant.*
Unofficial community note. Prefer the recording for nuance.