Session brief
WTF do people use Open Models for??
Overview
This talk explores the actual usage patterns of open-source AI models, moving beyond benchmarks to understand how individuals and businesses are employing them. It highlights that while new models emerge rapidly, established, reliable models often persist in production due to the significant effort required to change them. The presentation emphasizes that user preference, or "vibes," plays a crucial role in model selection, especially for creative and companionship use cases.
Who should watch
- AI Engineers
- Product Managers
- Developers building AI applications
- Anyone interested in the practical applications of open-source AI models
- Those evaluating AI model performance beyond standard benchmarks
Key takeaways
- Over 50,000 AI models have been uploaded to Hugging Face monthly, indicating rapid growth in the open-source AI space.
- Models like Llama 2 and Mistral Nemo continue to be widely used in production despite newer alternatives, due to their established reliability and the cost of switching.
- Individual users often choose models based on preference and perceived "vibes" rather than solely on benchmarks or price.
- Creative writing, role-playing, and companionship are significant use cases for open models, with a surprising majority of users in these segments being women.
- Coding assistance, particularly through agentic workflows and auto-completion tools, represents a substantial and growing segment of AI model usage.
- For enterprise applications, a phased approach to AI adoption, starting with human-in-the-loop systems and gradually automating reliable use cases, is more successful than aiming for full automation from the outset.
- The concept of "fully automated agents" is currently mythical in production environments; focusing on solving the 80% with escape hatches is a more pragmatic approach.
- New architectural approaches, like linear Transformers, offer potential for more cost-effective and reliable AI models.
Notable quotes
*The stain power of models in production and tutorials is a lot more sticky as most people think things that enters production in in Enterprises tends to not be changed for quarters or or even years at a time.*
*Women tend to use these apps to hold long conversations and distress talking about their day-to-day lives each day in part because their real life partners are either non-existent or emotionally unavailable for them.*
*The mindset that we should be thinking when we try to build AIS into production is that we should approach things of like trying to solve the 80% with Escape hatches along the way.*
Unofficial community note. Prefer the recording for nuance.