Session brief
AI Frontiers in Trust and Safety Combatting Multifaceted Harm on Tinder at Scale: Vibhor Kumar
Overview
This talk explores the application of AI, particularly Large Language Models (LLMs), to enhance trust and safety measures on platforms like Tinder. It addresses the challenges posed by generative AI, such as content pollution and sophisticated scams, while highlighting the opportunities LLMs present for detecting and mitigating multifaceted harm at scale. The presentation details the end-to-end process of using LLMs for violation detection, from data preparation and fine-tuning to production deployment.
Who should watch
- AI Engineers
- Product Managers
- Builders working on trust and safety systems
- Those interested in applying LLMs to detect harmful content
- Individuals concerned with scaling moderation efforts
Key takeaways
- Generative AI poses significant trust and safety challenges, including rapid content generation, deepfake technology, and scaled spam operations, which can overwhelm traditional detection methods.
- Open-source LLMs offer powerful capabilities for trust and safety tasks, with fine-tuning enabling state-of-the-art performance and accelerating model development from months to weeks.
- Creating high-quality, smaller datasets (hundreds to thousands of examples) is crucial for fine-tuning LLMs, and hybrid approaches using LLMs for data mining followed by manual verification are effective.
- Fine-tuning LLMs in-house offers advantages in cost, latency, throughput, and maintainability compared to relying solely on external APIs, especially for high-volume platforms.
- Parameter-efficient fine-tuning techniques like LoRA and QLoRA are critical for enabling rapid experimentation and efficient deployment on limited hardware.
- Frameworks like LoRAX allow for the efficient serving of thousands of fine-tuned adapters on a single GPU, making it feasible to address a wide range of specific trust and safety violations with near-zero marginal cost per adapter.
- LLMs offer superior generalization performance compared to traditional NLP models, making them more resilient to adversarial attacks like intentional typos and innuendos, thus slowing model degradation.
- Future directions include leveraging non-textual modalities (e.g., image detection with visual language models) and building next-generation defensive systems by rapidly training adapters for long-tail violations.
Notable quotes
*Trust and safety is about preventing risk, reducing risk, detecting harm, and mitigating harm. The ultimate goal is to protect users and also the companies creating the products that they use.*
*GenAI enables rapid generation of content, which makes it particularly easy to spread misinformation, propaganda, and low-quality spam by drowning out genuine content.*
*By fine-tuning our own models, we have full control over the model weights and can fine-tune when production performance inevitably degrades, without needing to worry about changes in the underlying base model.*
Unofficial community note. Prefer the recording for nuance.