When I Do Header

Inside Recommendation Algorithms and How to Steer Them

stuartmiles/depositphoto

Much of what people watch, read and buy online is chosen by recommendation algorithms. Netflix engineers Carlos Gomez-Uribe and Neil Hunt wrote in a 2015 paper that about 80 percent of the hours streamed on Netflix came from recommendations rather than search. YouTube's chief product officer said in 2018 that more than 70 percent of time spent watching YouTube was driven by its recommendations. Knowing how these systems work makes it easier to shape what they show you.

The basic approaches

Collaborative filtering recommends items that people with similar tastes liked. If many people who watched the same three shows as you also watched a fourth, you're likely to see that fourth show suggested. It doesn't need to understand the content at all, only patterns of behavior.

Content-based filtering recommends items similar to ones you've engaged with, based on their features: genre, actors, topics, keywords or audio characteristics. Spotify, for example, analyzes the sound of songs as well as listening patterns.

Modern systems combine both and add many other signals, using machine learning models trained on huge amounts of user activity.

Signals that shape recommendations

  • What you click, watch, like, share, save or buy.
  • How long you watch or read, and whether you finish.
  • What you skip, scroll past quickly or mark "not interested."
  • Searches, follows and subscriptions.
  • Time of day, device and general location.
  • What's popular or trending overall.

Watch time and engagement are especially influential on social video platforms, which is why a few seconds of lingering on a video can noticeably change a feed.

A famous competition

In 2006 Netflix offered a $1 million prize to anyone who could improve its rating predictions by 10 percent. A team called BellKor's Pragmatic Chaos won in 2009 by combining many different models. The contest helped popularize recommendation research, though Netflix later said it never fully implemented the winning solution, partly because its business shifted from DVD rentals to streaming.

Recommendations in shopping and music

Online stores recommend products based on what you've viewed and bought, what others bought together, and what's trending. These suggestions can be genuinely useful for finding accessories or alternatives, but they can also favor products with higher margins or sponsored placements, which marketplaces are required to label as ads. Music services build personalized playlists, such as Spotify's Discover Weekly, by combining what you listen to with what similar listeners play and the sound characteristics of songs. Because the systems learn from skips as well as plays, skipping songs you don't like improves future playlists.

The "cold start" problem

Recommendation systems struggle when they know little about a user or an item, a challenge known as the cold start problem. That's why new apps ask you to pick favorite genres or artists when you sign up, and why new products or videos are often shown to small test audiences first to see how they perform before being recommended widely. It's also why a brand-new account can seem to recommend random or very popular content at first.

Concerns about recommendation systems

  • Filter bubbles: systems may keep showing more of the same, narrowing the range of views and content people see. Research findings on how strong this effect is are mixed.
  • Escalation: optimizing for engagement can push sensational or extreme content, a concern raised by former employees and researchers studying social platforms.
  • Teen wellbeing: regulators and researchers have examined whether recommendation feeds encourage excessive use or harmful content for young users.
  • Transparency: users usually can't see why something was recommended.

What regulation requires

The European Union's Digital Services Act requires very large online platforms to explain the main parameters of their recommendation systems and to offer at least one feed option that isn't based on profiling of personal data. Since 2023, several platforms have added chronological or non-personalized feeds in the EU as a result. Rules elsewhere are less developed, though several US states have passed laws targeting algorithmic feeds for minors.

How to take control of your recommendations

  • Use "not interested" or "don't recommend channel" on YouTube and similar options on other apps. These are stronger signals than simply scrolling past.
  • Clear or pause watch and search history. YouTube, Netflix and Google let you delete or pause history, which resets recommendations.
  • Use separate profiles for family members so children's viewing doesn't shape your suggestions, and vice versa.
  • Switch to a following or chronological feed where available, such as the "Following" tabs on several social apps.
  • Turn off ad personalization in your Google, Meta and Amazon account settings.
  • Search deliberately for new topics you want to see more of. The system learns from active choices quickly.

Why recommendations aren't going away

With millions of songs, videos and products available, recommendation systems are what make large catalogs usable. The practical question for most people is less whether to use them than how to keep them working for their interests rather than simply maximizing their time on an app.