Speaker
Charles Frye
2 sessions in this library.
- What every AI engineer needs to know about GPUs — Charles Frye, Modal
This talk explains why AI engineers need to understand GPUs, shifting focus from API-based development to leveraging hardware capabilities. It draws an analogy to database usage, where developers don't build databases but must understand how to query them effectively. Similarly, AI engineers will increasingly need to understand GPU architecture, particularly tensor cores, to optimize performance for tasks like language model inference.
- How fast are LLM inference engines anyway? — Charles Frye, Modal
This talk explores the performance of open-source LLM inference engines, highlighting how recent advancements in model quality and inference software have made self-hosting viable. It presents benchmarking data to help engineers understand and optimize LLM performance for various use cases, emphasizing the trade-offs between different configurations and workloads.