World's Fair 2025
360Brew: LLM-based Personalized Ranking and Recommendation - Hamed and Maziar, LinkedIn AI
Hamed , Maziar , LinkedIn AI
Overview
This talk details LinkedIn's journey in developing and deploying 360Brew, a large language model-based system for personalized ranking and recommendations. The core thesis is that a single, holistic LLM can replace disjointed, task-specific models, offering zero-shot capabilities, in-context learning, and instruction following to understand user journeys and serve diverse personalization needs on the LinkedIn platform.
Who should watch
- AI Engineers and ML practitioners interested in LLM applications for personalization and recommendation systems.
- Product Managers and Builders looking to leverage LLMs for unified user experience across various platform features.
- Engineers focused on optimizing LLM performance for high-throughput, low-latency production environments.
- Those exploring methods for model distillation, pruning, quantization, and efficient serving of large models.
Key takeaways
- A single foundation LLM, 360Brew, was developed to handle all of LinkedIn's personalization and ranking tasks, moving away from siloed, task-specific models.
- The model is designed for zero-shot capabilities, allowing it to address new problems without extensive retraining, and leverages in-context learning for cold-start users.
- The development pipeline involves starting with large open-source models, upcycling them through continuous pre-training, fine-tuning, and alignment to create a powerful base model (e.g., 150 billion parameters).
- To achieve production-level efficiency, a distillation process is employed, gradually reducing model size (e.g., to 3 billion parameters) while retaining performance.
- Key levers for improving model quality include data scaling, increasing model size, and extending context length, though generalization to very long contexts can be challenging.
- Serving these large models efficiently requires techniques like gradual distillation, pruning, quantization (using mixed precision like FP8, with FP32 for the LM head), and attention-based specification.
- The team achieved a 7x reduction in latency and a 30x increase in throughput by optimizing the serving infrastructure and model efficiency.
- Comprehensive benchmarking, including out-of-domain tasks, is crucial for validating generalization and ensuring the model performs well across diverse use cases.
Notable quotes
The mission was to build a large foundation model based on large language models that understand the user journey on LinkedIn platform and can solve all the personalization tasks with just one model.
*A key lesson is that you build a lot of automation into these pipelines in order to make the fact that making these models is actually very complicated into a much easier and more manageable situation.*
*The recipe here is that we need to go big and then go small; if you go with a smaller model initially, it doesn't have enough capacity.*
Unofficial community note. Prefer the recording for nuance.