World's Fair 2026
OpenClaw in Your Hand: Building a Physical AI Terminal - Lech Kalinowski, Callstack
Overview
This talk details the creation of OpenClaw in Your Hand, a physical AI terminal designed for interacting with AI agents like OpenClaw. The project explores building an AI-native device using microcontrollers and a dual-display system (OLED and e-paper) for energy efficiency and a unique user experience. It highlights the challenges and potential of creating dedicated hardware for LLM interaction, moving beyond traditional screen-based interfaces.
Who should watch
- AI engineers and builders interested in custom hardware interfaces for AI.
- Product managers exploring novel user experiences for AI applications.
- Developers working with agentic AI systems and LLMs.
- Hobbyists interested in embedded systems and AI integration.
- Anyone curious about creating a physical terminal for AI interaction.
Key takeaways
- A physical AI terminal was built using an ESP32 microcontroller, an OLED display for dynamic content, and an e-paper display for static rendering.
- The system architecture includes a firmware on the terminal and a backend that handles agentic work and LLM interactions, demonstrated with a 120 billion parameter open-source model.
- Challenges encountered included software I2C issues, GPIO port selection, power supply regulation to protect displays, and the need for high-quality components like encoders.
- The project explored the potential of text-based RPGs, using generative AI to create worlds, characters, and narratives, showcasing a niche for quiet, distraction-free AI interaction.
- A key design principle was creating an AI-native operational system for microcontrollers, focusing on text-based interaction rather than audio or video.
- The backend exposed an OpenAI-style LLM proxy to handle compatibility issues with various open-source models.
- The device is designed to be robust, with fallback options for input and display if certain components fail.
- The project resulted in a provisional patent, suggesting potential commercial applications for dedicated AI terminals.
Notable quotes
The speaker noted, *I just wanted to build a device which is physical and AI native like the device which comes from the future.*
Regarding the RPG mode, the speaker mentioned, *this part really surprised me.*
On the market niche, the speaker stated, *There's a niche on the market.*
Unofficial community note. Prefer the recording for nuance.