Europe 2026
You Might Not Need 50 Diffusion Steps — Ziv Ilan, Nvidia
Overview
This talk explores techniques to accelerate diffusion models for image and video generation, addressing the challenge of high latency in practical applications. The core thesis is that by employing methods like quantization, caching, and distillation, developers can significantly reduce the number of diffusion steps required, thereby improving performance and enabling real-time generation without sacrificing quality.
Who should watch
- AI engineers working with diffusion models for image and video generation.
- Developers aiming to improve the speed and scalability of generative AI applications.
- Product managers and builders seeking to understand how to make generative AI more practical and efficient.
- Anyone interested in reducing the computational cost and latency of diffusion models.
Key takeaways
- Diffusion models typically require 20-50 steps for generation, leading to high latency.
- Quantization, a technique similar to its application in LLMs, can reduce memory usage and improve performance, making models runnable on lower-end GPUs.
- Caching techniques, adapted for diffusion models, can avoid recomputing unchanged parts of the image or video during denoising steps, offering a performance boost.
- Distillation, specifically step distillation, trains a student model to achieve high quality with fewer steps (e.g., 4-8 steps instead of 50), potentially yielding 10x-200x performance improvements.
- Real-time generation is achievable through techniques like step distillation, opening up new use cases such as streaming content.
- These optimization techniques are incremental and can be combined, allowing developers to start with simpler methods like quantization and progressively add more complex ones.
- Open-source resources and pre-quantized checkpoints are available to help the community adopt these optimization strategies.
- Distillation, while powerful, is often a post-training technique that may require specific data and more compute for fine-tuning.
Notable quotes
*The main challenge once we have some interesting use cases is how to make it actually usable, right?*
*If you are able to reduce this significant number of steps, but maintaining the quality, it's something that can give you 10x, 200x improvement in performance.*
*All of it is incremental. You can use this plus this plus this. You don't necessarily need to decide, okay, I'm doing only distillation or only quantization.*
Unofficial community note. Prefer the recording for nuance.