Harnesses in AI: A Deep Dive — Tejas Kumar, IBM

This talk explores the integration of coding agents, specifically the OpenClaw agent, into products. It emphasizes that coding agents are becoming a fundamental building block for software systems and highlights the Pi framework as an excellent tool for experimentation and development. The presentation demonstrates how agents can be leveraged to automate complex workflows, such as processing sales proposals, by interacting with various tools and systems.

This talk introduces the concept of agentic engineering, shifting the paradigm from merely using AI tools to actively collaborating with them. It highlights the evolution of AI in software development from simple line completion to sophisticated agents capable of executing complex tasks. The core thesis is that understanding and managing AI agents as collaborators, akin to junior developers, is crucial for maximizing their potential and navigating the complexities of modern software development.

This talk addresses the challenges of AI evaluations, which are currently scattered, quickly become outdated, and often lack transparency and verifiability. The speakers propose solutions to democratize the evaluation process, enabling a broader community to contribute to and benefit from robust AI benchmarking. Their work aims to foster more equitable AI development by allowing diverse expertise to shape evaluation standards.

This talk explores agentic search as a critical component of context engineering for Large Language Models (LLMs). The core thesis is that effective context curation for LLMs relies heavily on sophisticated search mechanisms, moving beyond simple retrieval to agent-driven exploration of diverse information sources. The presentation highlights the evolution from basic RAG to agentic RAG and emphasizes the challenges and strategies for building robust search tools that agents can effectively utilize.

This talk explores the effectiveness of semantic code retrieval for AI coding assistants, specifically benchmarking it against traditional grep-based methods on Claude Code. The core thesis is that while grep is simpler and often sufficient, semantic search, powered by vector embeddings, can significantly improve precision and the agent's ability to locate relevant code, especially for complex tasks. This approach offers a form of cached compute, reducing redundant processing and potentially leading to better accuracy and user satisfaction.