Session brief
Lets Build An Agent from Scratch — Kam Lasater
Overview
This talk demonstrates how to build an AI agent from scratch, breaking down the core components and their interactions. The presenter aims to demystify agent functionality by starting with a simple LLM call and progressively adding elements like conditional logic, tool usage, and planning mechanisms. The goal is to provide viewers with an intuitive understanding of how agents operate, enabling them to experiment with and adapt the provided code.
Who should watch
- AI engineers interested in the fundamental building blocks of agents.
- Developers looking to understand agent frameworks by seeing their underlying principles.
- Builders who want to experiment with creating agents without relying on existing frameworks.
- Product Managers seeking to grasp the capabilities and limitations of agent technology.
Key takeaways
- An agent can be conceptually defined as an LLM combined with memory, planning capabilities, tools, and a loop.
- Tool usage involves defining functions (like web search) that the LLM can call, with the LLM specifying which tool to use and with what parameters.
- The process of handling tool calls requires local code to interpret the LLM's request, execute the tool, and return the results to the LLM.
- Parallel tool calling allows an agent to execute multiple tool requests simultaneously, improving efficiency.
- Introducing a to-do list mechanism enables planning, allowing the agent to break down tasks, track progress, and manage its workflow.
- The LLM can manage the execution loop by making tool calls and iterating based on the results, but guardrails may be needed to prevent infinite loops.
- Tools can be viewed as text transformations, often taking string inputs and producing string outputs.
- The talk progresses from basic LLM calls to incorporating search APIs, parallel tool execution, and a planning system using a to-do list.
Notable quotes
*An agent LLM memory planning tools and a while loop.*
*The running and breaking is very very important part to the learning.*
*The tools are are best handled by thinking of them as text Transformations.*
Unofficial community note. Prefer the recording for nuance.