Europe 2026
Gemma 4 Deep Dive — Cassidy Hardin, Researcher, Google DeepMind
Overview
This talk provides a deep dive into Gemma 4, Google DeepMind's latest family of open-source AI models. The presentation highlights significant architectural improvements and new capabilities, including enhanced multimodal support and optimized performance for both on-device and complex reasoning tasks. Gemma 4 aims to set a new standard for small, open-source models, making advanced AI more accessible to developers.
Who should watch
- AI Engineers
- Product Managers
- Developers building AI applications
- Researchers exploring model architectures
- Those interested in on-device AI capabilities
- Builders looking for open-source models with advanced reasoning and multimodal features
Key takeaways
- Gemma 4 introduces four model sizes: two smaller models optimized for on-device applications (text, vision, audio input) and two larger models (26B MoE and 31B dense) for complex reasoning.
- The 31B dense model achieved third place on the AI leaderboard and boasts a 256K context length with native support for function calling and structured JSON outputs.
- The 26B Mixture of Experts (MoE) model is the first in the Gemma family, requiring only 3.8 billion active parameters during inference.
- Architectural improvements include interleaved local and global attention layers with grouped query attention to enhance efficiency, especially in global layers.
- On-device models utilize Per-Layer Embeddings (PLE) stored in flash memory instead of VRAM, significantly reducing memory constraints for mobile and laptop applications.
- Gemma 4 models are natively multimodal, with vision encoders supporting variable aspect ratios and resolutions, allowing developers to control token budget allocation for images.
- Audio capabilities are integrated into the E2B and E4B models, supporting translation and speech recognition through an audio tokenizer and conformer.
- All Gemma 4 models are released under an Apache 2.0 license, promoting broader accessibility and integration into development lifecycles.
Notable quotes
Cassidy Hardin, Researcher, Google DeepMind, stated that Gemma 4 brings *incredible improvements at a scale that has not been seen before*.
The models are designed to *provide incredible performance at a small scale which are able to run locally on phones, iPads, and laptops*.
The move to an Apache 2.0 license was *deliberately done in order to make our models more accessible for the everyday developer*.
Unofficial community note. Prefer the recording for nuance.