Session brief
120k players in a week: Lessons from the first viral CLIP app: Joseph Nelson
Overview
This talk details the creation and viral success of paint.wtf, an AI Pictionary game where users draw prompts generated by GPT-3, and CLIP judges the submissions based on image-text similarity. The game attracted 120,000 players in its first week, handling seven requests per second at its peak. The presentation covers the technical implementation, lessons learned from user interactions, and the potential of multimodal AI in building new types of applications.
Who should watch
- AI Engineers interested in multimodal applications.
- Builders experimenting with generative AI and image-text models.
- Product Managers exploring novel user engagement mechanics.
- Developers looking for practical examples of using CLIP and GPT-3.
- Anyone curious about the challenges and successes of launching a viral AI-powered game.
Key takeaways
- The game paint.wtf used GPT-3 for prompt generation and CLIP for judging user drawings against those prompts, achieving significant viral growth.
- Building a functional MVP of the game can be done with minimal code, leveraging open-source tools like the inference server.
- CLIP can interpret and rank drawings based on text prompts, demonstrating a new primitive for AI application development: open-form, open-set understanding.
- Users submitted a wide range of creative and abstract drawings, highlighting the internet's engagement with AI-driven creative tools.
- Lessons learned include CLIP's ability to read text within images and its potential for content moderation by comparing image similarity to NSFW concepts.
- CLIP's similarity scores are conservative, often ranging from 8% to 48%, necessitating scaling for clearer user feedback.
- The inference server, used in the demo, supports numerous pre-trained models and can handle high throughput on target hardware.
Notable quotes
*The internet loved this thing and in fact like we reloaded it with new prompts just because of demand folks wanting to do this.*
*This kind of represents a new primitive in building nii and that's like open form open set understanding as opposed to just very specific lists of classes and models.*
*We used clip to moderate clip.*
Unofficial community note. Prefer the recording for nuance.