World's Fair 2024
Building security around ML: Dr. Andrew Davis
Overview
This talk addresses the critical need for robust security measures in machine learning systems. It highlights the inherent fragility of ML models, making them susceptible to various attacks. The discussion covers data poisoning, model theft, adversarial examples, supply chain vulnerabilities, and software exploits, emphasizing proactive strategies and continuous vigilance to protect ML deployments.
Who should watch
- AI Engineers
- Product Managers
- Builders working with ML models
- Anyone concerned about the security of ML systems
- Those looking to understand adversarial attacks and defenses
Key takeaways
- Data poisoning can compromise model integrity by introducing malicious or incorrect data, necessitating skeptical treatment of public and user-generated data sources.
- Model theft, where attackers train surrogate models by querying an API, can be mitigated through model observability, logging, and limiting returned information.
- Adversarial examples, particularly crafted noise added to inputs, remain a significant threat, with current defenses offering limited robustness, especially in multimodal LLMs.
- The ML model supply chain is vulnerable; verifying model provenance, scanning for malware, and isolating untrusted models are crucial steps.
- Software vulnerabilities in ML frameworks and libraries, similar to traditional software, require consistent patching and vigilance, with tools like CVE scanners being essential.
- Prompt injections are a form of adversarial attack on LLMs, and while defenses exist, multimodal prompt injections are particularly challenging to detect.
- A proactive security mindset, assuming worst-case usage scenarios, is vital for building secure ML applications.
- Maintaining up-to-date ML frameworks and libraries, along with vigilant monitoring, is essential for mitigating software vulnerabilities.
Notable quotes
*Machine learning models are very fragile very easy to attack very easy to get them to do things that you don't necessarily intend for them to do.*
*If you're not doing any sort of observability or logging in your platform like you're not going to know if anybody is doing anything bad.*
*It's just a constant back and forth game* (referring to defense and attack evolution).
Unofficial community note. Prefer the recording for nuance.