What to do when 1 in 8 Skills have critical vulns and tries to steal your keys — Ezra Tanzer, Snyk

This talk addresses the growing problem of "skill hell" in AI development, where developers struggle to effectively use and integrate available skills. The core thesis is that a lack of clear criteria for what constitutes a "great skill" hinders progress. The presentation offers a structured checklist and practical techniques to help developers build, evaluate, and improve AI skills, moving beyond the current state of confusion and inefficiency.

This talk explores how to effectively guide AI agents when they interact with complex products like Superbase. It argues that while agents are capable of many tasks, they require specific guidance to handle novel situations, security protocols, and optimized workflows. The presentation introduces the concept of "skills" as a method to provide this guidance, demonstrating how they improve agent performance and safety, particularly when integrated with tool-calling capabilities.

This talk proposes a shift from building standalone AI agents to developing modular "skills" that agents can utilize. The core argument is that while agents possess intelligence, they often lack the specific domain expertise required for real-world tasks. Skills, defined as organized collections of files containing procedural knowledge and tools, offer a more composable and expertise-driven approach to enhancing agent capabilities. This paradigm aims to make AI more accessible and adaptable by packaging expertise into shareable units.

This talk emphasizes the critical need for rigorous evaluation of AI skills before deployment, arguing that shipping skills without proper testing can lead to unpredictable behavior and degraded performance. The speaker highlights the distinction between agents developers use for personal productivity and agents built for consumers, noting that end-users lack the context to troubleshoot skill invocation issues. The core thesis is that comprehensive evaluations are essential for ensuring skill reliability, managing costs, and determining when skills can be retired as models improve.

This workshop explores the concept and practical application of "skills" in AI agents, presenting them as discrete, composable units of work designed to enhance agentic workflows. The core thesis is that skills provide a structured way to encode essential context, instructions, and behaviors, preventing repetitive information input and ensuring consistent, predictable agent performance across various tasks and projects.