World's Fair 2026
Voice In, Visuals Out: The Agony and the Ecstasy - Allen Pike, Forestwalk Labs
Overview
This talk explores the challenges and opportunities in building AI experiences that accept voice input and produce visual output. It argues that while voice is a natural human communication method, current AI implementations are often slow and awkward. Conversely, visual outputs, including rich HTML, interactive controls, and illustrations, are highly effective for conveying information and engaging users. The core thesis is that by optimizing for voice-in, visuals-out interactions, developers can create more delightful and seamless AI experiences, overcoming the latency barriers inherent in voice-to-voice conversations.
Who should watch
- AI engineers and builders experimenting with conversational AI.
- Product Managers and UX designers seeking to improve user interaction with AI.
- Anyone interested in the practical challenges of real-time AI responsiveness.
- Developers looking to leverage AI for richer, more intuitive user interfaces.
Key takeaways
- Voice is a high-bandwidth human communication method, conveying more information per word than typing.
- Visual outputs from AI, such as generated HTML, interactive elements, and images, are highly effective for user comprehension and engagement.
- The primary barrier to seamless voice-in, voice-out AI is latency, with human conversation requiring sub-200ms response times.
- Voice-in, visuals-out interactions benefit from more forgiving latency envelopes, where responses within a second can feel seamless.
- Achieving responsive AI requires using fast, low-latency models (e.g., Haiku class) and potentially offloading heavier tasks asynchronously.
- Eager inference, sending prompts every 1-2 seconds as a user speaks, is crucial for real-time responsiveness, rather than waiting for silence.
- Stable caching, particularly prefix caching where the initial context remains consistent, is essential for reducing inference costs and improving speed.
- Minimizing output tokens and optimizing context windows are key to achieving fast and affordable inference turns.
Notable quotes
*Voice is the human preferred input for AIs. But that we prefer visuals as the output.*
*The models that we have so far, the experiences that people most people have seen are both slow and dumb.*
*The tyranny of latency. It's really difficult to get a response through that whole chain fast enough.*
Unofficial community note. Prefer the recording for nuance.