Cormac Brick

This talk explores the challenges and opportunities of running AI models on edge devices, where memory (RAM) is the primary constraint, not compute power. It highlights the development of smaller, more efficient models, including quantized versions of Gemma and even sub-billion parameter models, to enable AI capabilities on resource-limited hardware like Raspberry Pis and mobile NPUs. The focus is on practical applications and the trade-offs involved in deploying AI at the edge.

This talk explores the development and application of small and tiny Large Language Models (LLMs) for on-device AI agents. It highlights the benefits of on-device processing, such as improved latency, privacy, and offline functionality. The presentation introduces the Google AI Edge stack, including MediaPipe and the lighter TLM runtime, and discusses two primary approaches: leveraging system-level GenAI like Gemini Nano and developing App GenAI with customizable tiny LLMs.

This talk explores the advancements and applications of tiny Large Language Models (LLMs) and agent skills on edge devices. It highlights how these technologies enable powerful on-device AI experiences, focusing on reduced latency, enhanced privacy, and offline functionality. The presentation delves into the capabilities of Google's Gemma models and the LiteRT-LM runtime, showcasing their potential for building sophisticated AI-powered applications directly on user devices.