Europe 2026
What Lies Beneath the API — Benjamin Cowen, Modal
Overview
As AI applications mature, companies increasingly turn to fine-tuning models for improved performance and cost efficiency. This talk explores the transition from using frontier APIs to custom model development, highlighting the emerging middle ground that simplifies fine-tuning and deployment. It suggests that differentiated products naturally evolve towards domain specificity, prompting a reevaluation of when custom models become necessary and feasible.
Who should watch
- AI Engineers considering custom model development.
- Product Managers evaluating model performance and cost.
- Builders looking to optimize AI applications beyond prompt engineering.
- Teams facing latency or throughput requirements not met by frontier APIs.
- Developers interested in leveraging open-source libraries for fine-tuning.
Key takeaways
- Frontier APIs offer rapid development but lack customization beyond prompt engineering, which doesn't scale for rapidly growing businesses or specific enterprise needs.
- Fine-tuning provides model differentiation, better costs, and tailored performance, but traditionally required significant infrastructure investment and expertise.
- A new wave of serverless compute platforms and open-source libraries makes fine-tuning more accessible, enabling algorithm control without managing complex infrastructure.
- Signals indicating readiness for fine-tuning include economic scaling issues with current APIs, plateauing performance metrics, and the availability of curated data and evaluation frameworks.
- Building an agent harness or evaluating product performance generates the necessary data and insights for effective model training.
- Supervised fine-tuning and reinforcement learning can now be implemented with significantly less code, often within a few hundred lines, leveraging scalable serverless environments.
- Serverless platforms are beneficial not only for inference but also for training tasks like hyperparameter tuning and reinforcement learning rollouts, allowing for massive, on-demand scaling.
- Post-training, serving models can be achieved efficiently using tools like vLLM or custom Python workflows on serverless platforms, with auto-scaling capabilities.
Notable quotes
*The frontier labs probably don't have the exact same goal as you. They want their models to win on everything possible. And we want our models to win at our business logic.*
*It's just a matter of time until your product steps into a being domain specific.*
*If you have built a product, you probably have at least touched all the things you need to train if you haven't already done it.*
Unofficial community note. Prefer the recording for nuance.