Europe 2026
Accelerating AI on Edge — Chintan Parikh and Weiyi Wang, Google DeepMind
Overview
This talk focuses on accelerating AI deployment on edge devices, highlighting Google DeepMind's Gemma models and the Lite RT framework. The core thesis is that by optimizing models for edge hardware and leveraging a unified cross-platform architecture, developers can achieve significant performance gains, enhanced privacy, and reduced latency for a wide range of AI applications.
Who should watch
- AI engineers interested in on-device inference.
- Product Managers looking to integrate AI into edge products.
- Developers building applications requiring real-time AI capabilities.
- Builders seeking to deploy AI models across diverse platforms (mobile, IoT, desktop).
- Those concerned with data privacy and offline AI functionality.
Key takeaways
- Google DeepMind's Gemma 4 models, specifically the 2B and 4B variants, are optimized for edge deployment, offering capabilities like function calling, structured JSON output, and chain-of-thought reasoning.
- The Lite RT framework, built on TensorFlow Lite, provides a unified, cross-platform solution for deploying AI models from various frameworks (TensorFlow, PyTorch, JAX) on devices including Android, iOS, Linux, and IoT.
- Running AI on edge devices offers benefits such as reduced latency for real-time use cases, enhanced privacy for sensitive data, offline functionality, and potential cost savings compared to cloud-based processing.
- The gallery app serves as a playground for developers to explore Gemma model capabilities, with sample code and the ability to fork the app for custom experiences.
- NPU acceleration can provide significant performance improvements (3-10x) for AI tasks on edge devices, particularly for AR/VR and real-time applications.
- Models are available on Hugging Face under an Apache 2.0 license, encouraging community development and experimentation.
- A CLI tool and the AI Edge portal (a cloud-based benchmarking service) are available to aid developers in deployment and performance optimization.
- Performance benchmarks show competitive or superior speeds compared to other runtimes on mobile and IoT devices.
Notable quotes
*Running on edge has many benefits. Certainly like latency is important for those who are keen.*
*The big evolution with Gemini 4 is going to be really moving from like chatbot type capabilities to more autonomous agents that also support reasoning capabilities.*
Unofficial community note. Prefer the recording for nuance.