Design a Recommendation System — ML System Design Interview (Staff-Level)

How does your feed pick the perfect video from a hundred million options in under 200 milliseconds? The answer trips up most engineers: a recommendation system is a FUNNEL, not a model. Here's the full staff-level ML system design breakdown. In this ML system design deep-dive: • Why you can't just train a model to score the catalog — the latency wall (off by 500×) • The three-stage funnel: retrieval → ranking → re-ranking (cheap to expensive, wide to narrow) • Retrieval: two-tower embeddings + approximate nearest neighbor (ANN/HNSW) to search 100M items in milliseconds • Blending multiple retrieval sources for recall (embeddings, popularity, follows, fresh) • Ranking: a heavy multi-task model, features (user/item/context/cross), and the feature store • Re-ranking: diversity (MMR), freshness, business rules — and the filter-bubble feedback loop • Offline training vs online serving, and the train-serve skew that silently kills quality • Cold start, offline metrics vs online A/B, position bias Practice this exact interview — with an AI interviewer that pushes back — free at https://craqit.io ⏱ Chapters 0:00 The trap: reaching for the model 1:28 Requirements 3:03 The latency wall (the math) 4:30 The funnel 5:59 Retrieval: two-tower + ANN 7:54 Retrieval: many sources 9:25 Ranking: expensive & precise 11:02 Re-ranking & the objective trap 12:39 Offline training vs online serving 14:12 Senior follow-ups 15:40 Red flags 16:58 Recap ML System Design series — more in this playlist #systemdesign #machinelearning #recommendationsystem #mlsystemdesign #ai #faang