How to Build a Recommendation Engine: Retrieval, Ranking and What LLMs Changed

A product team puts its catalogue into an LLM prompt and asks the model to recommend products. The demo works on 200 items; at 200,000 the catalogue no longer fits the context and every request takes seconds.

How to build a recommendation engine in 2026 still starts from the retrieval and ranking funnel YouTube described a decade ago, and Meta, YouTube and Netflix run their generative models as stages inside it.

Types of recommender systems and the data each one needs

A recommendation engine ranks items for a user in a context: the products on a home page, the next video, the articles under a story. If you are still deciding whether you need one, our guide to what a recommendation engine is and how businesses use it covers that ground. The two classic families differ in the data they learn from.

Content based filtering

Content based filtering recommends items whose attributes resemble what the user already liked: same category, overlapping tags, similar description. It needs a catalogue with good item data and very little interaction history, so it works for a new item on its first day. Its weakness is narrowness. It keeps recommending more of the same, and it only knows what the attributes say, which for many catalogues is not much.

LLMs changed this family more than any other, because a text embedding turns a product description into a usable signal without hand-built tags. We come back to that below.

Collaborative filtering

Collaborative filtering ignores item attributes and learns from behaviour: users who bought this also bought that. Matrix factorisation and its descendants learn a vector per user and per item from the interaction matrix. Amazon's item-to-item version, described in Two Decades of Recommender Systems at Amazon.com, is the best known; our post on the Amazon recommendation system walks through it.

Old methods hold up better than papers suggest. A RecSys 2024 reproducibility study found that "with proper tuning of its hyperparameters, the BPR model can achieve performance levels close to state-of-the-art methods". Collaborative filtering fails on cold start: a new item has no interactions, so it never gets recommended, so it never gets interactions.

Sequence models, from SASRec to transformers

Both families above treat a user's history as a set. A sequence model reads it in order, so the item a user looked at a minute ago weighs more than one from last year, and "phone, then case, then screen protector" becomes a pattern it can learn. Kang and McAuley's SASRec (2018) applied self-attention to this and reported that it "outperforms various state-of-the-art sequential models (including MC/CNN/RNN-based approaches)" while being "an order of magnitude more efficient than comparable CNN/RNN-based models".

Transformer sequence models over user history are now a standard ranking component. They are also the direct ancestor of the generative recommenders described further down, which scale the same idea to billions of parameters.

Context-aware and knowledge-based recommenders

A context-aware recommender adds the situation to the model: time of day, device, location, the page the user is on. Food delivery at 8 am and at 8 pm are different problems even for the same person. In practice context arrives as features in the ranker rather than as a separate system.

A knowledge-based recommender works from explicit rules and item constraints instead of behaviour: the cartridge that fits the printer the customer owns, the insurance cover a customer qualifies for, the room that sleeps four. It needs complete item attributes and works from the first visit, which makes it the right tool for configurable, regulated or rarely bought products where interaction data will never be plentiful.

A hybrid recommendation system combines both, and almost every production system is one. The funnel below is how the combination is organised in practice.

Recommendation system architecture: the retrieval and ranking funnel

Covington, Adams and Sargin's 2016 paper on YouTube recommendations set out the design most systems still follow: "one for candidate generation and one for ranking". Candidate generation pulls hundreds of videos from a corpus of millions, and the ranker orders those few hundred, all "under a strict serving latency of tens of milliseconds".

The split exists because of cost. A model that scores every item in the catalogue for every request has to be cheap, so retrieval uses fast approximate methods and accepts some misses. The ranker sees only the candidates, so it can use hundreds of features and a much heavier model. Re-ranking applies the rules a model should not learn on its own, and serving keeps the whole chain inside the page's latency budget.

Offline, nearline and online computation

The second architectural decision is when each piece is computed. Netflix's engineers described the split in a 2013 post on its personalisation architecture that still describes most systems: "computation can be done offline, nearline, or online", and "all approaches can and should be combined".

• Offline jobs train the models and compute item embeddings, co-occurrence tables and popular lists in batch. They allow "complex algorithms and less limitations on the amount of data", at the price of staleness.
• Nearline processing reacts to events as they arrive but stores the result instead of serving it: the user's recently viewed list or session embedding, updated seconds after a click.
• Online work happens inside the request: the user tower's forward pass, the nearest neighbour lookup, ranking with the request's context, the business rules.

Two practical rules follow. First, every online step needs a fallback; the Netflix post puts it as "reverting to a precomputed result" when the online path misses its deadline. Second, a feature must be computed by the same code for training and for serving, or the model learns from values it never sees in production. Most teams discover that bug, known as training-serving skew, through a model whose offline gains never show up online.

How to build a recommendation engine, step by step

1. Start with a baseline and an A/B test

The first version is popularity per category plus co-occurrence ("bought together" counts from orders). It takes days to build, and it gives every later model something to be measured against.

Raw co-occurrence counts favour bestsellers: everything is "bought together" with the most popular item. Normalise the counts by each item's own popularity (cosine similarity or lift both work), count within a time window such as a session or 30 days, and set a minimum count so that two coincidental orders do not create a recommendation. Popularity lists need the same care: compute them per category and per recent window, or the home page shows last year's bestsellers to everyone.

Build the A/B test at the same time, because offline metrics will mislead you. Booking.com compared 23 model pairs and found a Pearson correlation of -0.1 between offline performance gain and business gain, and concluded that "offline model performance metrics are only a health check". Recall@k and NDCG tell you a model works. Only the online test tells you whether it sells more.

2. Log impressions as well as clicks

A click log without impressions cannot tell an ignored item from an unseen one. Log what was shown, in which position, and what the user did with it. Position matters because, as Joachims and colleagues showed, "position bias in search rankings strongly influences how many clicks a result receives", so a model trained on raw clicks learns to rank whatever was already on top. XGBoost's ranking objective supports position debiasing for click data; the impression log is what makes it possible.

One event record per action is enough, as long as every record can be joined back to the request that produced it:

{
  "event_type": "impression",
  "event_id": "e-7f3c2a1d",
  "timestamp": "2026-10-05T09:14:22.318Z",
  "user_id": "u-48213",
  "session_id": "s-9f1c",
  "request_id": "r-77a0",
  "surface": "product_page.similar_items",
  "item_id": "sku-10442",
  "position": 3,
  "retriever": "two_tower",
  "model_version": "ranker-2026-09-30",
  "context": { "device": "mobile", "country": "DE" }
}

Clicks, add-to-cart and purchase events carry the same request_id, so each outcome can be traced to the list that was shown; purchases add price and quantity. The retriever and model_version fields let you attribute results to a source and a model later, which you will want the first time a metric moves for no obvious reason.

The managed services publish minimums that work as a sanity check on volume. Amazon Personalize will not train on fewer than 1,000 interactions from at least 25 users with two or more each, and Algolia's frequently-bought-together model needs 1,000 conversion events that contain two or more items. A store below those numbers needs popularity, content similarity and rules, not machine learning.

3. Retrieve candidates from several sources

Production systems blend retrievers: co-occurrence for "goes with", recent popularity for freshness, the user's own history, and an embedding retriever for everything that rule-based sources miss. Each returns a few hundred items, and the union goes to the ranker.

The two tower model

The standard embedding retriever is the two tower model. One tower encodes the user and context into a vector, the other encodes each item into a vector of the same size, and training pulls together the pairs that led to an interaction. Item vectors are computed offline; at request time only the user tower runs. TensorFlow Recommenders' retrieval tutorial is a compact starting point.

Yi and colleagues at Google showed in their RecSys 2019 paper that training on in-batch negatives over-penalises popular items, and corrected for it with streaming frequency estimates. The corrected model was deployed at YouTube "to retrieve personalized suggestions from a corpus with tens of millions of videos". If you train a two tower model on your own data, read that paper first.

The paper also explains a bias every two tower model inherits. With in-batch negatives, the other items in a training batch serve as examples of what the user did not choose, and popular items appear in batches in proportion to their popularity. Without correction the model learns to push popular items down; with too much correction it recommends only them. Logging the retriever's outputs per request is how you see which way your model leans.

Approximate nearest neighbour search

Finding the closest item vectors to a user vector across a large catalogue needs an approximate nearest neighbour (ANN) index. The options are mature: FAISS (release 1.15.1 in September 2026), pgvector for Postgres with HNSW indexes since version 0.5.0, Elasticsearch and OpenSearch if you already run a search cluster, and dedicated vector databases such as Qdrant. If your catalogue already lives in Postgres, start with pgvector and move only when it measurably limits you.

The retrievers' results are then merged before ranking. Give each source a quota, remove duplicates, and drop anything the user cannot buy (out of stock, region-locked, already owned) before the ranker spends time on it. Keep the source label on each candidate: it becomes a ranking feature, and it lets you measure what a new retriever adds, which is exactly how YouTube evaluated PLUM.

4. Rank with gradient-boosted trees first

The ranker scores each candidate with every feature you have: user history, item statistics, context, the retrievers' own scores. Gradient-boosted trees with a learning-to-rank objective are the right first ranker: XGBoost's default ranking objective is LambdaMART-based rank:ndcg, and LightGBM offers lambdarank. They train in minutes, handle mixed features without much preparation, and are easy to inspect.

A first feature set that covers most of the signal:

• User features: recent categories and brands, price band, purchase count, days since the last visit, the user tower's embedding.
• Item features: popularity over several windows (a day, a week, a month), conversion rate, price, margin, stock level, age, the item embedding.
• Context features: surface, device, time of day, and the item already on the page.
• Cross features: whether the user bought this brand before, the similarity between user and item embeddings, the candidate's retriever and its rank there.

Deep rankers such as Meta's DLRM pay off once you have very large sparse features (millions of user and item IDs) and the traffic to learn their embeddings. Starting there means building that infrastructure before you know a model is worth it.

5. Re-rank for diversity and business rules

A ranker optimised for clicks will happily show ten near-identical black T-shirts. Re-ranking fixes the list: maximal marginal relevance, from Carbonell and Goldstein, picks each next item for being "both relevant to the query and contains minimal similarity to previously selected documents". This is also where out-of-stock items, items the user already bought, and margin or contractual priorities are handled, as explicit rules rather than something a model has to infer.

Re-ranking is also where new items get a chance. A model trained on past interactions will rarely rank an item nobody has seen, so reserve a slot or two for exploration: new or under-exposed items shown to a small share of traffic, so the system learns about them. Amazon Personalize builds this in, describing its exploration setting as making "recommendations include new items or items with less interactions data", and it updates its models every two hours to consider new items. A custom engine needs the same policy written down, with the share of traffic it may use.

6. Serve inside a latency budget

Recommendations sit on pages people are waiting for, and slowness costs money: the same Booking.com paper reports that "an increase of about 30% in latency costs more than 0.5% in conversion rate". Precompute what you can (item vectors, popular lists, co-occurrence tables), cache per user where the context allows, and give each stage of the funnel its own share of the budget.

When we built Recostream, our own recommendation engine for ecommerce stores, a new recommendation came back in 20 to 30 milliseconds under very high event volumes, and stores installed it with one line of JavaScript.

Where the engine runs relative to your application is the other serving decision. There are three common patterns, and they trade latency against ownership differently:

• A recommendation service behind a REST or gRPC API, called server-side while the page is built. You own the data and the rendering, and the call adds to page time, so it needs its own latency budget and a fallback list. This is the usual choice for platforms with an engineering team.
• A model embedded in the application itself. A tree ranker can run inside a Java service through XGBoost4J or ONNX Runtime, with no network hop. It ties the model's release cycle to the application's, so it suits small rankers and re-ranking rules; the retrieval index normally stays a separate service.
• A client-side script that calls the engine from the browser after the page loads, which is how Recostream installed with one line. The store needs no backend work, and the script can collect the impression and click events itself. Results arrive after the main content, which is fine for carousels below the fold and wrong for the first thing a visitor sees.

7. Measure online, in a tool the business already trusts

The second design decision in Recostream mattered as much as the latency. Its dashboard was integrated with Google Analytics and Google Optimize, so each store could run its own A/B test and see the effect in its own numbers. Most stores measured a sales uplift of 5 to 10%. GetResponse tested the product from January 2022 and completed the acquisition in December 2022.

Measure each layer with the metric that fits it. Offline, use recall@k for retrieval (did the item the user later chose appear among the candidates?), NDCG for ranking, and catalogue coverage and novelty to catch a model that recommends the same hundred items to everyone. Online, choose one primary metric (orders or revenue per visitor) and a set of guardrails. Microsoft's experimentation team defines guardrail metrics as aspects "that we don't want to degrade but won't necessarily improve", such as page load time, error rate and abandonment.

Run tests for whole weeks. The same Microsoft article gives seven days as the typical duration of an A/B test there and warns that "day-of-the-week effects can affect the applicability of the change". New recommendations also attract clicks because they are new: a treatment effect that shrinks day by day is, in Microsoft's words, "an indicator of novelty effect", and the result to trust is the one after it settles.

What LLMs changed in recommendation systems

Three things changed between 2023 and 2026, and they land in different stages of the funnel.

Text embeddings fixed much of the cold-start problem

An LLM embedding of an item's title and description gives content based filtering a signal that used to need manual tagging, and it can feed the item tower of a two tower model directly. This is the LLM-era piece most teams should adopt first.

LinkedIn replaced its feed retrieval with a Llama 3 dual encoder and measured a 0.8% revenue increase overall and 3.29% among members with fewer connections, where interaction data is thinnest. DoorDash embedded LLM-written store profiles and reported a 2.4% relative increase in homepage order rate. Netflix's foundation model lets "new titles with limited interaction data to rely more on metadata".

Where text is thin, LLMs can also generate the missing signal. Alibaba's FilterLLM, a 2025 preprint, uses an LLM to predict which users will interact with a new item; in a two-month test on a platform with 300 million daily users and a million new items a day it reported a 26.58% gain in GMV from cold items, at 236 milliseconds of LLM time per item. That cost is paid once per item, at upload, which is why it is affordable.

Generative recommenders and semantic IDs

The second change is the generative recommender: a sequence model, trained like a language model on users' action histories, that predicts the next item. Google's TIGER introduced semantic IDs: instead of an arbitrary ID, each item gets a short tuple of codes learned from its content (four 8-bit codes, 32 bits per item), and the model generates the codes of the next item token by token. Similar items share code prefixes, which helps the long tail and new items.

At the largest scale these models now beat the deep rankers they replace. Meta's HSTU paper reports a 1.5 trillion parameter generative recommender that improved one online task by 12.4% in A/B tests, with quality that "scales as a power-law of training compute", while DLRM-style models saturated at about 200 billion parameters. YouTube's PLUM, warm-started from Gemini, is "in production for YouTube recommendations, serving the retrieval of both long-form and short-form videos", where it raised panel click-through on Shorts by 4.96%.

Look at where these sit. PLUM is a retrieval source added to the candidate pool. Netflix uses its foundation model's embeddings as features and candidate generators, and its 2026 LLM-backed ranker moved the core online metric by 0.006%, which Netflix calls "statistically meaningful at Netflix scale". These are results from platforms with hundreds of millions of users and GPU budgets to match; the Netflix algorithm post covers how the pieces fit at that scale.

The one published exception to the funnel is Kuaishou's OneRec, a technical report describing a single generative model that replaces the retrieval and ranking cascade. Kuaishou says it handles 25% of the queries in its apps at an operating expense "only 10.6% of traditional recommendation pipelines". It is a strong result, from a single company with a short-video feed and its own GPU fleet.

Cost is the constraint everywhere else. Netflix notes that the inference cost of its LLM ranker "is roughly proportional to the product of model size and context length", serves it in a single forward pass rather than generating text, and cut the serving cost to about a third by trimming the context. Meta's HSTU paper reports that its model, 285 times more computationally complex than the one it replaced, still achieved 1.5 to 3 times higher throughput through a custom attention kernel and candidate batching. Generative recommenders are affordable only with that kind of serving engineering.

LLMs as rankers, explainers and conversational front ends

Prompting a general LLM to rank candidates directly works less well than it looks in a demo. Hou and colleagues found that LLMs "struggle to perceive the order of historical interactions" and "can be biased by popularity or item positions in the prompts"; on MovieLens-1M their best ChatGPT prompt scored an NDCG@10 of 44.23 against 78.89 for a trained SASRec model. LinkedIn reported in 2026 that its LLM ranker "never achieved superior online performance over the existing production model", and shipped a transformer ranker without an LLM instead.

What LinkedIn shipped is instructive. Its Feed SR model is a transformer over the member's history, trained on interactions rather than text, and it raised time spent by 2.10% and likes, comments and reshares by 3.52% in online tests. The paper's explanation for the LLM ranker's failure is that "it was difficult to encode the strength of network relationships in a text prompt". Signals that are naturally numbers or graphs lose information when they are written out as text.

The conversation layer is where LLMs have paid off. Amazon says its Rufus shopping assistant was used by more than 300 million customers in 2025 and helped deliver "nearly $12 billion in incremental annualized sales", a company-reported figure. Spotify's DJ has reached 94 million Premium users. Both put a language interface on top of retrieval and ranking systems that still do the selecting.

Which approach pays off where

Where recommendation systems pay off: results by industry

Published results come mostly from very large platforms, where a fraction of a percent is a large sum. They show where recommendations carry weight; for a mid-sized business the gain that matters is the step from no recommendations, or hand-picked ones, to a measured baseline.

Ecommerce and marketplaces

Amazon's own engineers, in their 2017 retrospective, cite a Microsoft Research estimate that 30 percent of Amazon.com's page views were from recommendations. The more widely quoted "35 percent of what consumers purchase on Amazon" comes from a 2013 McKinsey article that gives no method. Rufus's $12 billion of incremental annualised sales is Amazon's own 2025 figure. For personalisation in general, McKinsey's 2021 research puts the typical revenue lift at 10 to 15 percent, a consultancy estimate across sectors. The stores running Recostream saw 5 to 10 percent, measured in their own analytics.

Streaming video and music

Netflix's foundation model learns from "hundreds of billions of interactions" across more than 300 million members, and YouTube now retrieves both long-form videos and Shorts with PLUM. Spotify reported in 2023 that on days when users tune in to its AI DJ, "fans spend 25% of their listening time with DJ". Our posts on the Netflix algorithm and on movie recommendation systems go further into this sector.

Professional networks and delivery

LinkedIn's results show where the gain concentrates: its LLM-based feed retrieval lifted revenue by 0.8% overall and by 3.29% among members with fewer connections, where the old system had the least to go on. DoorDash's LLM-generated homepage themes raised the homepage order rate by 2.4%. Both are cases of new representations helping the users and items that classic collaborative filtering served worst.

Machine learning recommendation system: build, buy or start from a library

Managed services

For a store with an ordinary catalogue, a managed service is usually the right answer. Prices and limits as published in October 2026:

• Amazon Personalize: $0.15 per 1,000 recommendation requests on its v2 recipes, with a minimum charge of one request per second for every active campaign. Training needs at least 1,000 interactions, and v2 models consider up to 5 million items.
• Google's AI Commerce Search, renamed in June 2026 from Vertex AI Search for commerce: $0.27 per 1,000 predictions for the first 20 million a month, with catalogue and event import free.
• Algolia Recommend: 10,000 requests a month included on its Grow plans, then $0.60 per 1,000. Frequently-bought-together needs 1,000 conversion events, and models retrain once a day.
• Shopify stores get related-product recommendations generated automatically, configured through the free Search & Discovery app.

Services also end. Microsoft stopped new Azure Personalizer resources in 2023 and retired the service on 25 August 2026. Keep your event log in your own storage whichever service you choose, so moving to another service means retraining on data you already hold.

What the request volume costs at list prices, for recommendation requests alone:

Assumptions: traffic spread evenly over a 30-day month, so Personalize's per-second minimum is not exceeded by peaks; training, data ingestion, search and storage excluded; Recombee's extra units also cover the same number of interactions. Prices are from each vendor's pricing page in October 2026, without negotiated discounts. At a hundred million requests a month, requests alone cost $15,000 to $60,000 a month, which is where a custom engine's running cost becomes a fair comparison.

Open-source starting points

For a recommendation system project of your own, the libraries differ mainly in how actively they are maintained. TorchRec from Meta ships roughly monthly (1.9.1 in September 2026). Microsoft Recommenders, now a Linux Foundation AI & Data project, has worked examples of most classic algorithms, though its last release was in December 2024. TensorFlow Recommenders and implicit still get releases but move slowly, and NVIDIA Merlin has had no release since June 2024.

Their uses differ. Microsoft Recommenders is the place to compare classic algorithms on your own data before choosing one. Implicit trains matrix factorisation on implicit feedback fast enough to be a strong second baseline after co-occurrence. TorchRec is for teams that will train large embedding tables across GPUs, which most will not need in the first year. Whichever you pick, the event log, the evaluation set and the A/B setup carry over when you change models, so build those first.

When to build your own

Build when the recommendations are part of what you sell, when the catalogue or traffic outgrows a managed service's limits or prices, when latency has to fit a budget a hosted API cannot meet, or when the data cannot leave your systems. Recostream met the first and the third: the recommendations were the product, and their latency was one of the reasons stores bought it.

Choosing a partner for a custom build

If you build with an outside team, ask about the system after launch. Who runs it in year two, and who retrains the models? How will uplift be measured, and in whose analytics? Is the latency budget written into the scope, with the fallback behaviour for when it is missed? Do the code, the models and the event data live in your repositories and your cloud account from the first commit? A partner who has run a recommendation engine in production will answer these with numbers from a system they ran.

If you already run a Java platform and want recommendations, retrieval or an LLM component inside it, with an evaluation plan and an A/B test from the first week, that is the work our AI integration team does. A scoping call gets you a written ballpark with its assumptions.