Europe 2026
Skill Issue: How We Used AI to Make Agents Actually Good at Supabase — Pedro Rodrigues, Supabase
Overview
This talk explores how to improve AI agent performance through the use of skills, which are essentially structured folders containing instructions and files for running workflows or providing custom information. The speaker, Pedro Rodrigues from Supabase, details the process of creating, testing, and automating the evaluation of these skills, emphasizing their role in enhancing agent capabilities within products like Supabase. The core idea is to leverage skills for progressive disclosure, allowing agents to access information only when needed, thereby optimizing context window usage.
Who should watch
- AI engineers and builders looking to enhance agent performance.
- Product Managers and engineers aiming to integrate AI capabilities into their products.
- Developers interested in advanced techniques for agent testing and evaluation.
- Those working with or considering Supabase for their backend needs.
Key takeaways
- Skills are modular units comprising a `skill.md` file (with name and description) and optional reference files or scripts, enabling agents to perform specific tasks or access curated information.
- Progressive disclosure, facilitated by skills, allows agents to load information contextually rather than all at once, managing the context window more efficiently.
- Skills differ from tools like MCP by providing richer context and enabling workflows, while MCP tools are better suited for integrations and do not require a local environment to run.
- Testing agent skills can be done manually or automated using evaluations (evals), which provide a non-deterministic way to assess agent behavior, reasoning, and tool usage.
- An eval-driven development approach involves defining metrics, creating the skill, running evaluations, and iterating based on the results.
- Skills can be packaged and installed using tools like Vercel's NPM package, allowing for local or remote installation and integration with various agent frameworks.
- When creating database views, especially with row-level security (RLS) enabled, using the `security_invoker` flag in PostgreSQL is crucial to ensure RLS policies are applied correctly to the view.
- Automated testing pipelines for skills can be built by comparing agent performance with and without a specific skill, using evaluation frameworks and potentially LLMs as judges.
Notable quotes
*The secret sauce has been basically skills.*
*Progressive disclosure is basically when the agent or all the information about a subject is not loaded straight to context.*
*Skills actually just provide more context to to agent, right?*
Unofficial community note. Prefer the recording for nuance.