Europe 2026
Agents on the Canvas in tldraw — Steve Ruiz, tldraw
Overview
This talk explores the evolution of AI agents within the tldraw collaborative canvas environment. It begins with early experiments like Make Real, which translated drawings into functional prototypes, and progresses to integrating AI as a direct collaborator on the canvas. The core thesis is that by bringing agents directly onto the canvas, rather than confining them to sidebars, we can unlock more intuitive and powerful forms of human-AI collaboration, especially for creative and development tasks.
Who should watch
- AI Engineers exploring agentic workflows
- Product Managers and builders interested in novel AI applications
- Developers looking to integrate AI into creative tools
- Anyone curious about the future of human-AI collaboration in design and prototyping
Key takeaways
- Early AI tools like Make Real demonstrated the potential of translating visual input into functional prototypes, bridging the gap for non-technical users.
- Integrating AI directly into a canvas environment allows for more natural collaboration, where agents can manipulate and generate content alongside human users.
- The tldraw SDK enables building applications where AI agents can act as collaborators, performing tasks like drawing, completing diagrams, or filling out forms.
- Agentic loops, where an AI produces output, reviews it, and iterates, can be visualized and managed directly on the canvas, offering transparency into the AI's process.
- The concept of "fairies" in tldraw showcases multiple agents working together, with a leader agent orchestrating tasks and delegating to others, demonstrating agent orchestration.
- Giving agents direct access to a runtime API, particularly in a local-first desktop application, allows for powerful, albeit potentially risky, capabilities like modifying application code or generating interactive UIs.
- The hackable nature of the tldraw canvas, especially within its desktop application, enables agents to perform complex scripting and code manipulation, blurring the lines between design and development.
- The development of these agentic canvas tools highlights the ongoing challenges and innovations in managing shared state, agent communication, and safety within collaborative AI systems.
Notable quotes
*The core idea with the SDK is that you can build cool things with the SDK, which means it's part of my job to build cool things with the SDK to prove it.*
*The fun of this is as a way of exploring the model and what the model knows and how it can comport all that stuff.*
*The next step for this one is to kind of give more access to the canvas to the agents. And there's really we started to kind of run into the barriers of of safety.*
Unofficial community note. Prefer the recording for nuance.