Matt explores the rise of the recommender system over the last 10 years
Over the past 10 years, companies have invested heavily in recommender systems. In 2006, Netflix began the Netflix Prize – a competition offering a grand prize of $1,000,000 with the aim of bettering their existing recommender system. Teams were given a dataset of over 100 million movie ratings and return recommendations that were 10% more accurate than those offered by Netflix’s existing recommender system.
The advantages of accurately recommending products or items to users justifies the investment that companies make into building such platforms. Nowadays, we are all familiar with the “Frequently Bought Together” recommendations on Amazon. In 2011 Amazon integrated recommendations into nearly every part of the purchasing process – from product discovery right through to checkout. The following year, Amazon’s sales increased by 29% in comparison to the same quarter in 2011; an increase of nearly $3 billion. Of course the increase in sales was not purely because of the recommender systems put in place, but they certainly played a big part of it.
Online retail is one of the most obvious industries that can reap the rewards of a powerful recommender system, and ASOS’s platform is a perfect example of this. However during a recent chat, Jedidiah Francis, the Principal Data Scientist at ASOS, talked us through the difficulties of building such a system. Their current platform has been a key focus for a large team for the past couple of years, and they still have many years ahead of them and has even branched into the formation of a new, personalisation team. Systems like theirs require a lot of support and buy-in from senior stakeholders across the business. However they have built an incredible hybrid system that he is very proud of.
Recommender systems produce a list of recommendations in one of three ways:
1. Collaborative based filtering recommends products or items based on an individual user’s past behaviour and similar decisions made by other users
2. Content based filtering recommends products or items based on the products themselves – applying properties or characteristics to each product and recommending products with similar properties or characteristics
3. Hybrid Recommender Systems combine collaborative based filtering and content based filtering to recommend products or items
Before making the investment in to recommender systems, companies need to be very careful about how and why they are doing it. As Justin Grace (Lead Data Scientist at Sony Computer Entertainment Network) mentioned in his recent blog:
“Your recommendation engine is only as smart as your customers’ purchases are”
Recommender systems will have varying impacts depending on the industry and cost of products or items. For example, a consumer’s decision to buy a car will be a lot less influenced by a recommendation than their decision of what film to watch on Netflix. Before building these systems it is important to understand who your users or consumers are and why they behave the way they do, a point that a lot of companies often miss out on.
Recommender systems are not necessarily a great asset for every company. They require a lot of data to effectively make recommendations. A good recommender system firstly needs item data (from a catalogue or other form), then it must capture and analyse user data (behavioural events), and then the magic algorithm does its work. The more item and user data a recommender system has to work with, the stronger the chances of getting good recommendations.
There is no doubt that provided companies have enough data, are able to truly understand their customers, and approach the build from a strategic as well as technical perspective, then recommender systems can be a fantastic asset for any company. There is evidence of this success today in many industries – Last.fm (music), Amazon (retail), Netflix (media), WhatShouldIReadNext.com (books), StumbleUpon.com (websites), Twitter’s recommendations for people to follow (social media) are just some examples.
The future of recommender systems is really interesting. Which companies will adopt them? What further advancements can be made? What new companies will emerge with their business focussed on recommender systems? RecSys 2015 took place last week in Austria and is widely regarded as the most important conference in the recommender system world. I for one am looking forward to hearing some of the highlights!
Matthew Jones | Data Science Consultant
0203 301 9931 | email@example.com
For all the latest Data Science jobs please click here.
Join our LinkedIn group Data Science Collective.