Authors: Ying Li, Arjun Rao, Shradha Sehgal
Introduction
Suggestions sit on the coronary heart of the Netflix expertise. Our present manufacturing fashions depend on 1000’s of hand‑crafted options over customers, gadgets, and interactions, together with specialised architectures for sequence modeling, function interactions, and multi‑job aims. This stack has developed over a few years to assist numerous content material varieties (films, collection, video games, dwell, podcasts) and product surfaces, however its complexity makes it pricey to onboard new use instances: including a content material sort or floor can require vital function engineering, structure change, infrastructure work, and experimentation.
On the identical time, massive language fashions (LLMs) are altering how we take into consideration advice, as proven by current work comparable to PLUM, GLIDE, and OneRec-Suppose. Their broad world information and powerful language understanding make it doable to characterize consumer histories and merchandise metadata immediately as textual content, seize wealthy relationships in a shared semantic area, and steer suggestions by way of pure‑language prompts. Nonetheless, off‑the‑shelf LLMs are nonetheless removed from manufacturing‑prepared recommenders: they usually over‑suggest globally standard content material, hallucinate out‑of‑catalog gadgets, ignore enterprise constraints, and supply solely restricted personalization.
To handle this, we constructed GenRec, an LLM‑backed advice ranker that put up‑trains an inner basis LLM on Netflix‑particular information and aims. GenRec reveals that an LLM‑primarily based ranker can match or exceed a mature manufacturing system whereas counting on far fewer labeled examples and enter alerts.
At a excessive degree, GenRec:
- Verbalizes consumer histories, merchandise metadata, and context as textual content.
- Submit‑trains a Netflix‑tailored basis LLM for rating.
- Provides a catalog‑conscious scoring head over Netflix titles.
- Makes use of reward alerts to align with lengthy‑time period member worth and enterprise objectives.
- Runs in prefill‑solely mode on Netflix’s LLM serving stack for value effectivity.
In a big‑scale A/B check in opposition to a nicely‑tuned manufacturing ranker, GenRec achieves statistically vital enhancements in each brief‑time period and lengthy‑time period on-line metrics, whereas utilizing solely a small fraction of the Section‑2 labeled information and enter alerts. It reduces our reliance available‑engineered options and shifts the main target from function engineering to context engineering. On this weblog put up, we are going to describe how GenRec works, the way it performs, and why we imagine it factors towards a extra LLM‑centric future for advice at Netflix.
Downside Setting
We give attention to a full‑catalog rating job (or prime‑Okay rating when a candidate set is supplied).
Given a consumer 𝑢, their interplay historical past 𝐻, and the present context 𝜏 (gadget, floor, locale, time, and so forth.), GenRec scores every merchandise and produces a personalised rating that may immediately energy suggestions or function enter for downstream personalization methods.
Formally, we map a request (u,τ,t,H) — consumer, context, time, and historical past — to a rating 𝜋 over the catalog C, the place π(i) is the place assigned to merchandise i. We optimize π for anticipated lengthy‑time period member utility (a proxy for satisfaction and retention), not simply brief‑time period engagements.
From Basis LLM to Suggestion Ranker
GenRec follows a two‑section coaching framework (Determine 2):
Section 1 — Netflix-Tailored Basis LLM.
We begin from an open‑supply LLM and adapt it on proprietary Netflix corpora, so it learns foundational capabilities comparable to
- Netflix content material understanding
- Member conduct and choice patterns
- Basic language understanding and era.
Section 1 is up to date comparatively sometimes and serves as a shared, Netflix‑conscious spine for a lot of functions.
Section 2 — GenRec.
We then flip this basis mannequin right into a excessive‑high quality rating mannequin by put up‑coaching on rating‑particular information and aims. Section 2:
- Focuses on rating high quality and steering
- Incorporates a number of reward alerts by way of reward‑weighted losses
- Is refreshed extra ceaselessly to trace new content material and evolving tastes
- Is explicitly optimized underneath serving value constraints.
Coaching Information as Conversations
Netflix members generate a whole bunch of billions of interplay occasions spanning many surfaces (views, performs, durations, thumbs up/down, add to checklist, abandons, and so forth.). We convert this log information into single‑flip or multi‑flip “conversations” between a consumer and a recommender. Every flip incorporates:
- Consumer message: verbalized context, profile, historical past, merchandise metadata, and job (e.g., suggest what the consumer will watch or thumb subsequent).
- Assistant message: the member’s precise engagement (e.g., which titles had been performed, for the way lengthy, what suggestions they supplied).
Throughout Section‑2 coaching, the LLM learns how assistant messages depend upon consumer messages. This enables us to precise wealthy advice alerts as textual content, collectively supporting each the language-modeling (LM) and rating aims.
At inference time, we feed within the verbalized context and apply a catalog‑conscious scoring head to rank gadgets; we don’t decode assistant messages. The conversational format is primarily used throughout coaching to assist the LM goal and protect sturdy language understanding over the verbalized textual content.
Verbalization and Context Engineering
Conventional recommenders function on dense options and embeddings. GenRec takes a unique strategy: it verbalizes wealthy consumer histories and context as pure language, encoding uncooked interplay alerts immediately within the LLM’s semantic area. In doing so, it depends on the mannequin to find increased‑degree patterns — comparable to merchandise relationships and evolving consumer pursuits — somewhat than on guide function engineering.
Naively verbalizing each interplay in a consumer’s historical past can shortly exceed the token price range and be too costly at Netflix scale. The context window turns into our new “function price range”, so we apply context engineering:
- Retain in full: excessive‑sign engagements (e.g., lengthy performs, thumbs‑up) with richer particulars
- Omit: low‑sign occasions (e.g., very brief performs or fast hovers)
- Summarize or compress: repetitive behaviors (e.g., binge‑watching )
- Elaborate selectively: vital or chilly‑begin gadgets (e.g., new releases)
Inside a hard and fast token price range, we prioritize current, excessive‑sign historical past and compress or drop older historical past. We additionally construction the immediate to maximise shared prefixes for higher prefix caching. The objective is a compact, excessive‑data immediate that preserves rating high quality with out prohibitive prices.
Goals: Rating, Language, and Rewards
The general GenRec mannequin is skilled with a multi‑goal loss that mixes a advice rating goal, language modeling aims, and alignment by way of reward‑weighted coaching.
1. Catalog‑Conscious Rating Goal
The first job is a rating goal that teaches the mannequin to attain gadgets by engagement high quality. We label positives utilizing excessive‑worth engagements (e.g., sufficiently lengthy performs, sturdy specific suggestions), with thresholds and denoising logic, and prepare the mannequin — by way of a cross‑entropy loss over the catalog or candidate set — to assign increased scores to those positives given a verbalized context.
2. Language Modeling Goal
We additionally retain a language modeling (LM) goal over the verbalized inputs and outputs. This helps protect the mannequin’s basic language understanding, improves its potential to interpret wealthy pure‑language histories and merchandise metadata, and retains the door open for textual content‑era use instances comparable to advice explanations.
3. Reward‑Weighted Loss for Alignment
Past uncooked rating accuracy, GenRec should (1) respect enterprise necessities — for instance, balancing films, collection, video games, dwell, and podcasts — and (2) optimize lengthy‑time period member satisfaction somewhat than simply rapid clicks or performs.
Get Netflix Expertise Weblog’s tales in your inbox
Be part of Medium free of charge to get updates from this author.
Coaching solely on uncooked interplay sequences can result in undesirable behaviors, comparable to over‑favoring binge‑watching or over‑specializing in a single content material sort. To handle this, we weight the rating loss utilizing alerts from separate reward fashions. Every coaching instance receives a scalar weight derived from two varieties of alerts:
- Lengthy‑time period satisfaction proxies: estimate how a lot a brief‑time period engagement contributes to lengthy‑time period outcomes, comparable to return conduct, catalog exploration, or sustained engagement.
- Habits rebalancing: modify behaviors throughout content material varieties and launch levels (for instance, video games vs. films, new releases vs. evergreen titles) to higher align with enterprise objectives.
The instance’s rating loss is then scaled by this weight: excessive‑worth engagements obtain bigger weights, and low‑worth ones are down‑weighted. This reward‑weighted strategy is easier and extra cost-efficient than full reinforcement studying, but gives efficient alignment in observe. We now have seen extra positive aspects from RL‑fashion strategies (e.g., GRPO), however depart them to future work attributable to their increased value.
Mannequin Structure and Serving
Spine and Scoring Head
GenRec’s structure carefully follows our foundational LLM: a decoder‑solely Transformer skilled with subsequent‑token‑prediction fashion aims, augmented with a catalog‑conscious rating head that scores solely Netflix in-catalog gadgets. The scoring pipeline works as follows:
- Verbalization: A verbalizer V serializes consumer historical past H, context 𝜏 , and related merchandise metadata right into a single textual content sequence x.
- Pooled illustration: The LLM processes x, and we extract a pooled hidden state h that summarizes the consumer’s present preferences and context.
- Catalog‑conscious scoring: Every catalog merchandise i has a discovered embedding eᵢ. A scoring head ϕ combines h and eᵢ (e.g., by way of dot product or small MLP) to provide a rating sᵢ. Making use of a softmax over scores yields a likelihood distribution which we convert right into a rating π.
All parameters — the spine, scoring head, and merchandise embeddings — are skilled collectively. For very massive catalogs, we will use sampled softmax or candidate units for environment friendly coaching and inference. This structure constrains suggestions to the Netflix catalog whereas supporting environment friendly scoring over massive candidate units.
Serving and Value Optimization
GenRec is served on Netflix’s inner LLM stack utilizing vLLM. At Netflix scale, serving value is pushed primarily by 1) Mannequin dimension; 2) Context size; 3) Inference mode (prefill vs. autoregressive decoding). We management value by way of three methods:
- Smaller / distilled fashions: We prepare GenRec on smaller or distilled basis fashions, usually with bigger or extra focused datasets, to seize many of the high quality of bigger fashions at decrease serving value.
- Aggressive context compaction: Utilizing the context engineering described earlier, we decrease tokens whereas preserving rating high quality.
- Prefill‑solely inference: Autoregressive decoding over massive candidate units could be prohibitively costly. As a substitute, we run in prefill‑solely mode: the mannequin consumes the immediate as soon as and scores the complete candidate set in a single ahead go, with no token‑by‑token decoding.
Collectively, these decisions make it possible to serve GenRec on excessive‑quantity workloads inside compute budgets.
Offline and On-line Experiments
We evaluated GenRec in opposition to a mature manufacturing ranker that has been tuned over a few years. The baseline mannequin depends on 1000’s of engineered dense and embedding options, in addition to customized architectures for modeling function interactions and sequences. We assessed efficiency utilizing each offline analysis metrics and a big‑scale on-line A/B check.
GenRec vs Manufacturing Baseline
Offline, GenRec outperformed the manufacturing ranker on rating metrics regardless of utilizing far fewer enter alerts and labeled examples. With roughly 40× fewer Section‑2 labeled coaching examples, GenRec achieved about +1.6% enchancment in Imply Reciprocal Rank (MRR). As we elevated Section‑2 coaching information and enriched the enter alerts, GenRec’s offline metrics continued to enhance.
On-line, we ran a big A/B check on batch‑compute advice surfaces, protecting ~10% of Netflix site visitors over ~4 weeks. On this low‑information, low‑sign configuration, GenRec delivered statistically vital positive aspects over the manufacturing baseline on each brief‑time period and lengthy‑time period on-line metrics (Determine 3).
These outcomes point out {that a} correctly put up‑skilled and aligned LLM‑backed ranker could be a sturdy various to conventional advice fashions, with substantial headroom as we additional scale information and enter alerts.
Information, Mannequin, and Section Contributions
We ran ablations to grasp the place GenRec’s positive aspects come from.
Information and Mannequin Scaling
- Information scaling: For each ~1B and ~10B parameter backbones, offline MRR improves as we improve Section‑2 put up‑coaching information. Bigger fashions attain increased absolute MRR however comply with an identical scaling curve (see Determine 4).
- Mannequin scaling: Underneath a hard and fast coaching price range, we put up‑skilled GenRec variants from ~1B to ~10B parameters. Inside this price range, bigger backbones persistently achieved increased offline MRR than smaller ones.
Section-1 vs. OSS, Section-2 vs. Section-1
- Section-1 vs. OSS: Utilizing the Section‑1 Netflix‑tailored basis LLM as the bottom mannequin improves offline rating metrics by roughly 10–20% in comparison with beginning immediately from an off‑the‑shelf LLM.
- Section-2 vs. Section-1: Section‑2 put up‑coaching provides one other 35–50% achieve in offline rating metrics when evaluated close to the Section‑1 coaching cutoff (i.e. when Section‑1 mannequin is the freshest). As time passes and Section‑1 turns into stale with new content material and shifting tastes, the relative advantage of Section 2 grows to about 80% after 2 weeks.
Information effectivity vs. manufacturing ranker
- Ranging from a robust Section‑1 mannequin, GenRec matches or exceeds the manufacturing ranker utilizing 10–40× fewer Section‑2 labeled examples, relying on configuration. This marginal information effectivity is very useful as a result of Section 2 is refreshed way more ceaselessly than Section 1.
Context Size Optimization
Context size drives each high quality and value: longer verbalizations expose extra conduct and context however improve coaching and serving value. To review this commerce‑off, we diversified context size and verbosity and optimized them in three steps:
- Clear and compress occasions: drop low‑sign engagements and compress repetitive conduct to type a cleaned sequence of occasions.
- Discover the “elbow level”: differ what number of historic occasions we embody and plot MRR vs. variety of occasions to establish an elbow past which extra context yields diminishing returns (see Determine 5).
- Optimize verbosity: for the retained occasions, check completely different ranges of particulars and simplified wordings, measuring MRR every time.
In our experiments, we will scale back the context tokens to roughly one-third of the unique price range with negligible degradation in offline rating metrics. Since serving value is roughly proportional to context size, we noticed an identical discount in serving value.
In direction of LLM‑Native Suggestion
GenRec is greater than “swapping in a Transformer” for an current ranker. It hints at a broader shift towards LLM‑native advice at Netflix. A number of notable adjustments:
From Function Engineering to Context Engineering
Conventional RecSys stacks revolve round massive function units and heavy function infrastructure. LLM‑centric methods as an alternative revolve round developing wealthy textual contexts from uncooked logs, metadata, and instruments. The “immediate” turns into the brand new function vector.
Modeling effort shifts from designing options to deciding which alerts to incorporate, how far again in time to go, easy methods to compress or summarize historical past inside a token price range. Our experiments on verbalization compaction illustrate this shift: cautious context design can protect high quality whereas dramatically decreasing serving value.
From Custom-made Architectures to Basis Backbones
Traditionally, every advice job usually had its personal customized structure (two‑tower fashions, DLRM‑fashion networks, bespoke consideration blocks). In an LLM‑centric world, many duties share a widespread basis spine, with differentiation coming from information and verbalization methods, put up‑coaching aims and rewards, and inference optimization.
GenRec leverages the identical spine as our basis LLM somewhat than introducing a brand new structure constructed from scratch. This makes it simpler to share learnings throughout functions, and opens the door to pure‑language steering for future experiences.
Scaling Legal guidelines as Design Guides
Conventional RecSys can hit diminishing returns attributable to sparse IDs, heavy engineering aims, and job‑particular architectures. With an LLM‑backed spine, advice inherits clearer information and mannequin scaling conduct: inside value limits, extra information and bigger fashions persistently enhance high quality. This brings RecSys design nearer to the broader LLM paradigm, the place scaling legal guidelines assist information mannequin and information funding.
From RecSys Infra to LLM Infra
LLM‑backed recommenders push us towards LLM‑fashion infrastructure: GPU‑accelerated, vLLM/Triton‑primarily based, with cautious batching and caching. Over time, advice serving infra begins to look extra like basic LLM infra than traditional RecSys stacks constructed round MLPs or factorization fashions.
Conclusions
We now have introduced GenRec, an LLM‑backed advice ranker at Netflix that adapts an inner basis LLM for giant‑scale personalization. By verbalizing consumer histories, context, and merchandise metadata, including a catalog‑conscious rating head, utilizing reward‑weighted aims aligned to lengthy‑time period satisfaction and enterprise objectives, and serving effectively on our LLM infrastructure, we receive a mannequin that improves on a robust manufacturing ranker whereas utilizing far fewer Section‑2 labels and enter alerts.
GenRec is an early however promising step towards a extra LLM‑centric advice stack at Netflix. Our outcomes recommend that, with cautious consideration to value, infrastructure, and alignment, LLM‑backed recommenders can play a central position in massive‑scale personalization.
Acknowledgments
GenRec is the results of shut collaboration amongst a number of groups and organizations throughout Netflix. The contributors to this work (in alphabetical order):
AI for members: Arjun Rao, Ashish Rastogi, Baolin Li, Fernando Amat Gil, Grace Huang, Justin Basilico, Kamelia Aryafar, Linas Baltrunas, Moumita Bhattacharya, Ogheneovo Dibie, Rein Houthooft, Shradha Sehgal, Sejoon Oh, Sergi Perez, Sourabh Medapati, Thea Wang, Yaochen Zhu, Yesu Feng, Ying Li, Yun Li, Yucheng Shi, Yunan Hu
AI platform and serving: Abhishek Agrawal, Adam Singer, Binh Tang, Daneo Zhang, Derek Olejnik, Ed Maddox, Erik Osheim, Lingyi Liu, Liping Peng, Meghana Chilukuri, Nicolas Hortiguera, Shaojing Li, ZQ Zhang
Product: Ilke Kaya, Michelle Kislak, Scarlet Chen, Si Cheng
