Session brief
Trust, but Verify: Knowledge Agents for Finance Workflows - Mike Conover
Overview
This talk explores the development and application of knowledge agents designed to process vast amounts of financial data, drawing parallels to the transformative impact of spreadsheets on accounting. The core thesis is that these AI agents can significantly accelerate financial research and due diligence, moving beyond human limitations in handling complex, large-scale information. The presentation emphasizes the need for systems that can reveal their thought processes and allow for human oversight and intervention to ensure accuracy and relevance.
Who should watch
- AI engineers and builders working on agentic systems.
- Product Managers and UX designers exploring new interfaces for AI.
- Finance professionals seeking to understand how AI can augment their workflows.
- Anyone interested in the practical challenges and solutions for building reliable AI agents.
- Individuals dealing with large-scale data analysis and research in any domain.
Key takeaways
- Knowledge agents can digest and analyze massive corpuses of financial documents, accelerating tasks like due diligence and competitive analysis by orders of magnitude.
- Effective agents require systems that can perform end-to-end reinforcement learning over tool use, enabling locally suboptimal decisions for globally optimal outputs.
- The user interface for these agents is a critical design problem, moving beyond simple chat interfaces to more interactive and transparent displays of the agent's reasoning.
- Synthesizing information from disparate documents into coherent narratives is a key capability, though current models have limitations in handling complex real-world factors like temporality and combinative reasoning.
- Human oversight and the ability to "nudge" the model with directives remain crucial, especially for incorporating qualitative insights and "taste" not present in digitized data.
- The Unix philosophy of simple tools working well together can be applied to agentic systems, where different models or systems are selected for specific nodes in a compute graph.
- Reducing the "latency trap" by optimizing feedback loops between user and agent is essential for developing user faculty and improving the product experience.
- Interactive outputs that allow users to drill down into findings, see audit trails, and request more details on demand are vital for building trust and utility.
Notable quotes
*frankly not a human level intelligence task and the reality as we've stepped into this space um is that these professionals uh just get put in a meat grinder*
*the sophistication of the thought that you can bring to bear on the problem has increased so substantially because there are tools that allow us to think more effectively more efficiently*
*the winning systems will perform end to end RL over tool use calls where the results of the API call are in fact part of the RL um sequence of decisions*
Unofficial community note. Prefer the recording for nuance.