Maxime Labonne

This talk explores the unique challenges and opportunities in training small, frontier AI models, emphasizing that they are not simply scaled-down versions of larger models. The core thesis is that small models, designed for edge deployment, possess distinct characteristics like being memory-bound, having lower knowledge capacity, and being latency-sensitive. Addressing these specific traits requires tailored architectural choices, training methodologies, and problem-solving approaches, particularly for issues like doom-looping.