World's Fair 2024
LLM Quality Optimization Bootcamp: Thierry Moreau and Pedro Torruella
Overview
This talk focuses on optimizing Large Language Model (LLM) quality through fine-tuning, addressing common pain points like high operational costs and the inability to meet production-ready quality standards. It positions fine-tuning within a broader "crawl, walk, run" strategy for LLM quality, emphasizing that it should be considered after prompt engineering and Retrieval Augmented Generation (RAG) have been explored. The presentation outlines a continuous deployment cycle for fine-tuned LLMs, including data collection, model fine-tuning, deployment, and evaluation, aiming to demystify the process for AI engineers.
Who should watch
- AI Engineers facing high generative AI operational costs.
- Engineers struggling to meet the quality bar for LLM applications in production.
- Developers looking to improve LLM performance for specific, focused tasks like classification or information extraction.
- Individuals interested in leveraging open-source LLMs and fine-tuning techniques.
- Builders seeking to understand the trade-offs between different fine-tuning methods and deployment strategies.
Key takeaways
- Fine-tuning is most effective for highly specific tasks after prompt engineering and RAG have been exhausted.
- A continuous deployment cycle for fine-tuned LLMs involves data set collection, model fine-tuning, deployment, and iterative evaluation.
- High-quality, real-world data is crucial for successful fine-tuning; options range from human-generated responses to synthetic data.
- Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA produce smaller, more efficient models suitable for specialized tasks and easier deployment.
- Open-source models offer ownership of fine-tuned weights, providing flexibility in deployment and avoiding vendor lock-in.
- Smaller open-source models (e.g., Llama 3 8B) can achieve significant improvements when fine-tuned for specific tasks, offering a cost-effective alternative to larger proprietary models.
- Deploying fine-tuned models on optimized platforms can lead to substantial cost reductions, with potential savings of up to 200x compared to using general-purpose models like GPT-4.
- Quality evaluation can be automated for precise tasks but often requires human-in-the-loop assessment for more subjective outputs.
Notable quotes
*Fine-tuning is a method that we're going to use to improve the LLM quality but as a bonus we're going to be also showing how to improve quality significantly.*
*The key here is to be able to build on a solution that is designed to serve models at production scale volumes.*
*Fine-tuning is a journey but a very rewarding Journey there's truly no Finish Line here.*
Unofficial community note. Prefer the recording for nuance.