World's Fair 2025
Effective AI Agents Need Data Flywheels, Not The Next Biggest LLM – Sylendran Arunagiri, NVIDIA
Overview
This talk argues that building effective AI agents relies on data flywheels rather than simply using the largest available language models. A data flywheel is a continuous cycle of data processing, model customization, evaluation, and safety guardrailing. This process allows agents to refine their performance over time by learning from user feedback and production data, ultimately enabling the use of smaller, more cost-effective models without sacrificing accuracy.
Who should watch
- AI engineers and builders looking to improve the long-term relevance and helpfulness of their AI agents.
- Product Managers and engineers focused on shipping and scaling AI-powered products.
- Those facing challenges with rapidly changing data, evolving user preferences, and high inference costs associated with large models.
- Developers interested in practical frameworks and tools for building and maintaining agentic systems.
Key takeaways
- AI agents are systems that perceive, reason, and act, often utilizing tools and external systems to complete tasks. Their effectiveness is enhanced when they can capture and learn from user feedback.
- Data flywheels create a continuous loop for data curation, model customization, evaluation, and guardrailing, leading to more accurate and relevant agent responses.
- This cycle helps identify and promote smaller, efficient models that offer comparable accuracy to larger models but with lower latency and inference costs.
- NVIDIA's Nemo microservices offer components for each stage of the data flywheel: Curator for data, Customizer for fine-tuning, Evaluator for benchmarking, Guardrails for safety, and Retriever for RAG pipelines.
- A case study on NVIDIA's internal employee support agent demonstrated how a data flywheel, using user feedback and production logs, enabled a smaller 8B model to match the accuracy of a 70B model for routing queries, while significantly reducing cost and latency.
- The process involves monitoring user feedback, analyzing and attributing errors, creating ground truth datasets, experimenting with and fine-tuning models, and establishing a cadence for continuous monitoring and improvement.
- *Data flywheels can help surface efficient smaller models that provide at par accuracy expectations as that of larger language models but which offer lower latency, faster inference and effectively lower cost.*
- A framework for building data flywheels includes monitoring user feedback, analyzing errors, planning model experiments and fine-tuning, and executing by triggering cycles and tracking performance.
Notable quotes
Silendrin Arunagiri stated that effective AI agents need data flywheels, not just the next biggest LLM.
*Data flywheels create a continuous loop of data processing and curation, model customization, evaluation, and guardrailing for safer interactions.*
*By deploying a smaller model, you're looking at a 98% savings in terms of lower inference cost and also 10x 70x model size reduction with 70x lower latency.*
Unofficial community note. Prefer the recording for nuance.