Europe 2026
Self Driving Products: Product Signals to Pull Requests — Joshua Snyder, PostHog
Overview
This talk introduces a pipeline designed to automate the process of turning product observability data into actionable code changes. Instead of engineers manually reviewing dashboards and creating pull requests (PRs) for issues, the system aims to automatically detect product signals, diagnose problems, and generate PRs for review or even direct merging. The ultimate vision is a product that can largely build and maintain itself by learning from user interactions and data.
Who should watch
- Product engineers and managers looking to automate bug fixing and feature iteration.
- AI engineers interested in building agentic systems that interact with codebases.
- Developers seeking to reduce time spent on manual debugging and dashboard analysis.
- Teams exploring how to leverage observability data for automated development workflows.
Key takeaways
- The current process of identifying product issues from dashboards to PRs is slow, often taking days.
- The proposed pipeline ingests various product signals (errors, logs, replays), groups them into meaningful reports, and uses an agent to research and propose fixes.
- A critical step involves grouping diverse signal types by generating queries from signals rather than directly matching embeddings, which improves accuracy.
- The research agent utilizes tools like an MCP server for additional data, codebase context, and external knowledge bases (e.g., Linear, Notion) to understand problems.
- The pipeline assesses actionability, either flagging issues for human review, gathering more data, or directly generating a PR.
- Automated PR generation involves cloning the repo, building fixes with an agent, and handling CI failures or comments by re-running the sandbox until the PR is green.
- Key lessons learned include the importance of robust evals on production data, careful consideration of embedding strategies for diverse data types, ensuring problem specificity for agents, and not prematurely optimizing for token costs during experimentation.
- The system is currently in alpha, with plans to roll out automated shipping of experiments and continuous learning from PR outcomes and deployment results.
Notable quotes
*Instead of ever looking at your analytics dashboard, or your errors, or your logs, we just want you to look at PRs that are ready for you in GitHub.*
*If you don't think about the structural similarity of your different sources when you're grouping them, then the grouping works really badly.*
*Tokens are free. Uh obviously, that's not true. They're not free. Um but when you're experimenting, uh we were at first uh focused a lot on the costs of the pipeline.*
Unofficial community note. Prefer the recording for nuance.