World's Fair 2024
Making Open Models 10x faster and better for Modern Application Innovation: Dmytro (Dima) Dzhulgakov
Overview
This talk focuses on the advantages of using open-source AI models for modern applications, emphasizing how they can achieve significantly better performance and speed compared to proprietary models. The presentation highlights the challenges in productionizing open models, such as complex setup, optimization, and achieving enterprise-scale reliability, and introduces Fireworks AI's solutions for these issues. The core argument is that by optimizing serving stacks and enabling fine-tuning, open models can be made 10x faster and more cost-effective for a wide range of applications.
Who should watch
- AI Engineers looking to optimize inference speed and cost.
- Product Managers evaluating model choices for new applications.
- Builders seeking to leverage open-source models for production.
- Developers facing challenges with the setup and maintenance of open models.
- Teams needing to integrate AI models with external tools and data sources.
Key takeaways
- Open-source models offer significant advantages in speed and cost-effectiveness, often outperforming larger proprietary models for specific use cases.
- Productionizing open models presents challenges in setup, optimization, and scalability, which Fireworks AI aims to address with a custom serving stack.
- Fine-tuning open models for specific domains or tasks, such as function calling, can lead to substantial performance gains and enable complex agentic systems.
- Fireworks AI provides a platform for serverless inference, on-demand dedicated hardware, and enterprise-scale solutions, supporting a wide array of open-source models.
- The trend is shifting from single models to compound AI systems that integrate models with tools, RAG, and function calling for robust applications.
- Fireworks AI has developed specialized models for function calling, like FireFunction V2, designed to translate user queries into complex reasoning and tool execution.
- The platform supports various modalities, including image generation with models like SDXL and SD3, and offers fine-tuning capabilities for custom model development.
- Fireworks AI is API-compatible with OpenAI, allowing for easy integration with existing tools and frameworks like LangChain and LlamaIndex.
Notable quotes
*Open source models are great so what are the challenges? The challenges really come from three areas: complicated setup and maintenance, optimization, and getting it production ready.*
*We really believe that open source models are the future also for like for for Gen application and our focus at fireworks is precisely on that.*
Unofficial community note. Prefer the recording for nuance.