recommendation system

A recommender system ranks a catalog of millions of items down to the handful a specific user will actually see, and the entire engineering problem is doing that ranking in milliseconds without ever running a model over the whole catalog for every request. No production system scores every item for every user — instead it narrows the field in stages, each stage cheaper and more selective than the last, and the architecture of that narrowing is what separates a recommender system from an ordinary classifier.

Collaborative filtering learns from the interaction pattern alone, with no need to understand what an item actually is. Matrix factorization represents the sparse matrix of users and items they've interacted with as the product of two much smaller matrices — a latent vector for every user and a latent vector for every item — such that a user's predicted affinity for an item is just the dot product of their two vectors. The approach was popularized at scale by the Netflix Prize (2006-2009), where latent-factor models proved to be the single most effective technique for predicting ratings from past behavior alone, with no knowledge of a movie's genre, cast, or plot. Rendle and colleagues' Bayesian Personalized Ranking (2009) adapted the idea to implicit feedback — clicks and purchases rather than explicit star ratings — by training directly on the signal that actually matters for ranking: that an item a user interacted with should score higher than one they didn't, not that the predicted rating should match some specific number.

Content-based methods exist specifically to cover the gap collaborative filtering structurally cannot. A new item has no interaction history, so a latent factor model has nothing to learn its vector from — the cold-start problem is not a bug in a particular implementation, it's a direct consequence of learning purely from interaction patterns. Content-based recommendation instead represents an item by its actual attributes — text, image, or audio embeddings, category metadata — so a brand-new item can be recommended the moment it's catalogued, matched against a user's profile built from the embeddings of what they've previously consumed. Almost every production system today is hybrid specifically because of this tradeoff: collaborative signal gives the strongest personalization once enough interaction data exists, and content signal covers the users and items that don't have that data yet.

At scale, recommendation runs as a funnel, not a single model. Scoring tens of millions of candidate items against a single user with a heavy ranking model is too slow to run per request, so production systems split the problem into stages of increasing cost and decreasing candidate count.

Two-tower retrieval is what makes the first, highest-volume stage of that funnel fast. Covington and colleagues' description of YouTube's recommendation system (2016) popularized the pattern that's now standard for candidate generation: one neural network tower encodes the user into a vector, a separate tower encodes each item into a vector, and the two are trained so that relevant user-item pairs land close together in the same embedding space. Because item embeddings can be precomputed and indexed offline, retrieval at request time becomes an approximate nearest-neighbor search against that index rather than a forward pass through a model for every candidate — the same ANN-search pattern vector databases use for any similarity search, applied here specifically to narrowing millions of items down to a few hundred.

The ranking stage then affords a far more expensive model, because it only has to score what survived the funnel. Google's Wide & Deep architecture (Cheng and colleagues, 2016) combined a wide linear component — which memorizes specific, previously-seen feature combinations well — with a deep neural network component — which generalizes to feature combinations it hasn't seen before — on the premise that a recommender needs both memorization and generalization, and neither alone is sufficient. This is the stage where rich context — time of day, device, session behavior, the full cross of user and item features — becomes affordable to use, precisely because the two-tower retrieval stage already did the expensive work of eliminating the overwhelming majority of irrelevant candidates.

Offline metrics measure whether the ranking is good; only a live experiment measures whether it's good for the business. Precision@K, Recall@K, and NDCG score how well a ranked list matches known relevant items, but a model can win on every offline metric and still underperform in production if it over-optimizes for short-term clicks at the expense of long-term satisfaction, or if it reinforces a feedback loop where already-popular items get shown more, get more clicks as a result, and crowd out everything else regardless of underlying quality. This is why recommendation teams treat offline evaluation as a necessary filter before an A/B test, never as a substitute for one.

Read a recommender system as an exercise in budget allocation rather than pure modeling: at every stage — retrieval, ranking, re-ranking — the real design question is how much compute and latency that stage can afford, and the entire two-tower-then-heavy-ranker funnel exists because no single model affordably does the whole job at the scale a real catalog and a real request-latency budget demand.