World's Fair 2024
Fine tune 20 Llama Models in 5 Minutes: Santosh Radha
Overview
This talk demonstrates how to fine-tune and deploy multiple AI models directly from Python code without requiring complex infrastructure like Kubernetes or Docker. The core idea is to use a Pythonic interface to abstract away the underlying compute resources, allowing users to specify hardware requirements and execution parameters with simple decorators. This approach simplifies the process of running computationally intensive tasks, such as model training and inference, on various backends, including local machines, on-premise systems, or cloud-based GPU clusters.
Who should watch
- AI Engineers looking to simplify model training and deployment workflows.
- Product Managers aiming to understand how AI models can be built and shipped efficiently.
- Builders who want to prototype and deploy AI applications using pure Python.
- Developers seeking to run complex AI tasks on specialized hardware without managing infrastructure.
Key takeaways
- Covalent is an open-source tool that enables running Python code on any compute backend, locally or in the cloud.
- Users can define compute requirements (e.g., specific GPUs, memory, time limits) using single decorators in their Python functions.
- The system automatically ships Python code to the specified backend, executes it, and returns the results.
- Inference endpoints can be deployed with automatic scaling, authentication, and custom auto-scaling mechanisms defined in Python.
- A workflow can be constructed in Python to orchestrate multiple tasks, such as fine-tuning, evaluating, and deploying models sequentially.
- The process of fine-tuning and deploying models can be achieved entirely through Pythonic code, abstracting away Docker and Kubernetes.
- Users pay only for the compute time consumed by their tasks, with costs detailed for specific hardware and durations.
- The talk showcases an example of fine-tuning and evaluating multiple models, then deploying the best-performing one.
Notable quotes
*The code gets shipped to a back end in a GPU and you get back the result on your side.*
*You can define your Custom Auto scaling mechanism like hey I want to Auto scale it to 10 gpus exactly at 9:00 every day.*
*This is completely pythonic and once you dispatch this to our server which is essentially calling single line over here.*
Unofficial community note. Prefer the recording for nuance.