Session brief
The Intelligent Interface: Sam Whitmore & Jason Yuan of New Computer
Overview
This talk explores the future of human-computer interaction, moving beyond traditional text and voice inputs to embrace a more intuitive, context-aware, and multimodal interface. The core thesis is that computers should adapt to human circumstances and context, rather than forcing users to adapt to rigid, deterministic interfaces. This involves leveraging a wider range of sensory inputs and expressive outputs to create a more natural and intelligent interaction loop.
Who should watch
- AI Engineers
- Product Managers
- Builders and developers interested in next-generation interfaces
- Those exploring multimodal AI and context-aware computing
- Anyone curious about the evolution of human-computer interaction
Key takeaways
- The future of interaction lies in adapting to user context, using inputs like pose detection, ambient sound, and even social cues to inform AI responses.
- New hardware, such as advanced headsets with hand and eye tracking, will enable more expressive social gestures as a form of explicit user intent.
- Generative intelligence can be treated as a fluid, probabilistic material, suggesting that metaphors from physics and nature might be more suitable for designing interfaces than rigid ones like metal or wood.
- Interfaces can blend modalities, allowing for simultaneous voice, gesture, and visual inputs, with outputs that dynamically adapt to the user's current context.
- *Explicit social gestures can be a great way to determine user user intent.*
- *We can mix these modalities in real time for whatever makes sense in whatever context you're in.*
- Probabilistic interfaces require grounding through familiar metaphors from nature, physics, or human-made tools to manage their diverse output possibilities.
- Social and cultural norms are also critical materials to consider when designing AI interactions, ensuring appropriateness for different contexts.
Notable quotes
Sam Whitmore and Jason Yuan suggest that computers should adapt to people's circumstances and context.
The presenters propose that generative intelligence is a probabilistic material, perhaps like fog or mercury, suggesting new design metaphors.
Unofficial community note. Prefer the recording for nuance.