World's Fair 2026
What if the harness mattered more than the model? - Aditya Bhargava, Etsy
Overview
This talk argues that the "harness" surrounding an AI model, which includes its tools, safety mechanisms, and reasoning capabilities, is often more critical to performance than the model itself. The speaker proposes shifting focus from solely improving large, proprietary models to developing sophisticated harnesses that can enhance the performance of smaller, open-source, locally runnable models. This approach aims to reduce reliance on closed-source AI and empower developers to build powerful agents with greater control and flexibility.
Who should watch
- AI engineers and builders interested in agent development.
- Product Managers evaluating AI capabilities and infrastructure.
- Developers seeking to improve the performance of open-source models.
- Those concerned with agent safety and autonomous operation.
- Anyone exploring advanced agent architectures like sub-agents and self-optimization.
Key takeaways
- A significant performance difference (over 20 points) can be achieved by changing only the harness, even with the same model and tasks.
- A well-designed harness can make weaker models perform comparably to cutting-edge ones, enabling the use of local, open-source models.
- Building effective harnesses may require language-level support, leading to the development of the Agency language for agent creation.
- Key harness improvements include adding tools, implementing safety features like interrupts and handlers, enabling autonomy through partial function application, incorporating feedback loops for reasoning, utilizing sub-agents for modularity, and employing self-optimization for systematic performance gains.
- Agency provides language-level features for tool definition, safety mechanisms (interrupts, handlers, partial function application), and built-in optimizers, simplifying complex agent development.
- *A good harness can compensate and make a weaker model perform better.*
- *Better harness equals better performance, especially for weaker models.*
Notable quotes
Aditya Bhargava suggests that the industry is moving in the wrong direction by relying on fancy proprietary models that can't be run locally.
He posits that a good harness can make a weaker model perform better, allowing developers to build their own harnesses without depending on paid models.
Bhargava states, *Simple things should be simple, complex things should be possible*, as a guiding principle for the Agency language.
Unofficial community note. Prefer the recording for nuance.