Recommendation engines produce a lot of revenue for Amazon, Netflix and Facebook, but challenges include data dependency, trust, and lack of innovation.
Almost everything you read, see, or buy on the internet these days has been selected by an algorithm. That includes news articles on Google, status updates on Facebook, products shown on Amazon, movies on Netflix or Hulu, or Google map directions.
Algorithms are big business. The Netflix algorithm reportedly makes the company $1 billion a year by keeping viewers watching so they don’t cancel their accounts. Amazon, meanwhile, which has woven product recommendations throughout its site, saw net sales rise about $20 billion last year to $107 billion. And Facebook ads, which use an algorithm to match ads to user activity, were responsible for 80 percent of the company’s ad revenue in the fourth quarter of 2015.
Recommendation system approaches
According to IBM, most recommendation systems are built on two basic approaches:
Collaborative filtering uses data from the customer and other users who have similar traits to arrive at a recommendation. Amazon, for instance, might recommend certain products based on what you’ve previously bought or viewed as well as what other users bought or viewed in a certain product category.
Content-based filtering makes a recommendation on the basis of a user’s behavior. If you’ve ever seen a Google ad for a wine store after typing in “best French wines” into Google, that’s content-based filtering at work.
Most large recommendation engines use both approaches. Netflix, for instance, recommends movies based on what movies are popular among all customers, as well as what an individual user has previously watched in certain genres.
Recommendation systems also employ machine learning. As more user activity is recorded, algorithms take note of differences in your preferences and those of other users to continually refine the recommendations shown.
But algorithms are far from perfect — here are three drawbacks:
1. Data issues
Generally algorithms are more accurate with fewer data points. But as the amounts of data grow, “traditional algorithms can have difficulty keeping up,” notes IBM. “Although this might not be a problem for offline processing, more-specialized approaches are needed for real-time scenarios.”
The paradox is that, while being better with small amounts of data, recommendation systems may not be worth the investment unless there’s enough items in a database. A mom-and-pop ecommerce outfit that carries three styles of tennis shoes probably doesn’t need a recommendation engine.
One approach to solving the problem for big data involves ever-increasing classification. Musimap’s recommendation system, for instance, uses a Neo4j graph database to plots 55 weighted variables for each piece of music. An artist such as James Brown, may have variables that include gospel, soul, and funk.
Recommendation systems also tend to be limited by the product database. As of this March, the Netflix catalog contained just 4,335 movies – a tiny fraction of films produced globally — and with many popular U.S. titles unavailable.
2. Trust
One issue with recommendation engines is they think they know you. Because of data and product dependency, they don’t. Their knowledge of a person is based on clickstream activity within an interface, the efficacy of the algorithm itself, and the wisdom of the crowds. Facebook algorithms, for example, have been shown to poorly reflect a user’s actual preferences.
IBM notes that “although some users’ behavior can be modeled, other users do not exhibit typical behavior.” These users can skew the results of a recommender system and decrease its efficiency.
Another danger is that the recommendation system can be exploited to favor one product over another. Some companies, in fact, will hire an “online reputation management” firm to leave positive reviews about the product. Do you really trust that four-star rating on Yelp?
3. Lack of innovation
In trying to select a winner product, movie, or article, recommendation systems tend to reproduce stereotypes and reinforce existing practices.
A related danger is bias: While algorithms can learn from weighted variables in a product database, the listing itself often has a human touch, including the product description, image, and overall web design. A recommendation system may thus favor one product over another simply due to better imagery and not product quality.
An article by BloomReach further notes that algorithms tend to do poorly with nuances. For example, Disney’s release of the movie “Frozen” in 2013 redefined the term, but an algorithm that hadn’t been updated may retrieve results for frozen food.
