Scientify
AI Explained

How does a video app know what you want to watch next?

From a most-popular list to deep learning that predicts how long you'll watch: how video recommendations are built, with papers from YouTube's own team.

Lots of new videos are uploaded to video apps every minute. Yet as soon as one video ends, the next one is ready, and it is often just right. The video above (in Indonesian) builds that recommendation system step by step, from the simplest approach to the one YouTube uses.

Seven steps of a recommendation system

  1. Most popular videos. Just show the most-watched videos. Safe enough for new users, but everyone gets the same videos.
  2. Content-based filtering. Every video gets a label: cooking, football, cats. Watch a cooking video and you get more cooking videos. More relevant, but it soon gets repetitive.
  3. Collaborative filtering. Find people whose taste is like yours, then offer the videos they liked that you haven't watched. You discover new things, but comparing hundreds of millions of people one by one is too heavy.
  4. Item-based with co-visitation. Compare videos instead of people: which videos are often watched in the same session within 24 hours. The catch: the table is mostly empty, because one person watches only a tiny fraction of millions of videos.
  5. Matrix factorization. The machine looks for hidden tastes (latent factors). Every person and every video is summarised as a few numbers, and the empty cells can be guessed. But the only clue used is watch history.
  6. Two-stage deep learning. Every clue goes in: videos watched, words searched, location. The first stage narrows millions of videos to a few hundred, the second ranks them down to a few dozen. It predicts how long a video will be watched (watch time), not whether it gets clicked, so clickbait doesn't win.
  7. Multitask learning and position bias. The machine chases several goals at once: clicked, watched, liked or dismissed. It also gets a dedicated part to correct position bias: the video at the top gets clicked more even when it isn't the best.

Sources

The YouTube part of the video is based on three papers by YouTube and Google's own teams:

  • James Davidson et al., "The YouTube Video Recommendation System", RecSys 2010. It explains counting videos co-watched within sessions (co-visitation), usually over 24 hours.
  • Paul Covington, Jay Adams and Emre Sargin, "Deep Neural Networks for YouTube Recommendations", RecSys 2016. It explains the two stages (candidate generation and ranking), why watch time is used instead of clicks, and that the previous system used matrix factorization.
  • Zhe Zhao et al., "Recommending What Video to Watch Next: A Multitask Ranking System", RecSys 2019. It explains multitask learning (clicks, watches, likes, dismissals) and a dedicated part to remove position bias.

Most-popular lists, content-based filtering and collaborative filtering are general recommender techniques, not claims about how YouTube works.

Want to build a system like this?

Recommender systems, including collaborative filtering, are covered hands-on in the Recommender System part of Machine Learning Part 2.

More explainers

Want to build AI like this yourself?

Learn from zero in Indonesian, with a project in every topic. From Rp 79.000 for a month. 7-day money-back guarantee.