World's Fair 2025
Fuzzing in the GenAI Era — Leonard Tang, Haize Labs
Overview
This talk introduces hazing, a form of fuzz testing adapted for Generative AI systems. It addresses the critical challenge of validating and verifying AI outputs, which are inherently subjective and unstructured. Traditional evaluation methods relying on static datasets are insufficient due to the brittle and non-deterministic nature of GenAI, where minor input variations can lead to drastically different outputs. Hazing aims to pressure-test AI systems through large-scale simulation and optimization before deployment to ensure robustness and reliability.
Who should watch
- AI Engineers
- Product Managers
- Builders of AI applications
- Those concerned with AI reliability and security
- Teams struggling with the "last mile problem" in AI product development
Key takeaways
- GenAI systems are brittle, meaning small changes in input can cause significant output variations, a core challenge beyond simple non-determinism.
- Traditional evaluation methods using static, human-curated datasets lack sufficient coverage and struggle to quantify subjective quality or output similarity.
- Hazing involves simulating large-scale stimuli, analyzing responses, and using this feedback to guide iterative search for bugs and corner cases.
- Scaling judge time compute is crucial for effective evaluation, achieved through methods like building agents as judges (e.g., the Verdict library) or using reinforcement learning (RL) for fine-tuned criteria and critiques.
- Input generation for hazing is an optimization problem over the natural language space, requiring guided search rather than brute force due to the vastness of possible inputs.
- Optimization algorithms like gradient-based methods, tree search, and latent space manipulation can be applied to discover inputs that break AI applications.
- Hazing is particularly valuable for regulated industries like finance and healthcare, where it has uncovered prompt injections and unexpected corner cases in critical applications.
- Adversarial testing, a more aggressive form of hazing, can emulate malicious user attempts to prompt inject or jailbreak AI systems, even across different modalities like audio.
Notable quotes
*We know that AI systems are extremely unreliable. They're hard to trust in practice, and you sort of need to pressure test them before you put them out into the wild.*
*GenAI apps are incredibly brittle. And I think this is the actual core property that makes building with AI so difficult.*
*Standard evals don't have sufficient coverage.*
Unofficial community note. Prefer the recording for nuance.