Building Better Recommendation Engines
Ask about Knowledge at Wharton
Episode Description
n today’s data-rich environment, everyone from retailers to content providers is constantly suggesting options they think consumers will like based on past behavior. Research by Wharton’s Kartik Hosanagar takes a closer look at the real impact of those recommendations. More: http://knlg.net/1PKVfEJ 📂 Marketing 🏷️ Research
Read the full transcriptHide the full transcript
- an important team of my research is how personalized recommendations and similar
- algorithms affect consumer choice we are all flooded by product choices today and
- these kinds of personalized recommendations play an important role in helping us discover new products are
- sorting through large choice sets and we see a personalized recommendations in a
- number of industries whether it's in retail for example Amazon's of people who bought this also bought this or in
- media such as Netflix or YouTube we see it with news as well for example Google News will recommend personalized news
- stories and we know they have a pretty significant impact on consumer choice for example at Amazon they drive
- anywhere from a quarter to a third of the choices that consumers make online so although we know that they have a big
- impact on consumer choice we don't fully understand what kinds of products are more likely to be accepted by consumers
- when recommended and when do recommenders work well and when they don't so in my research with Professor
- document Carnegie Mellon we look at two main questions the first is what kinds of products are more likely to benefit
- from recommendations specifically our mainstream products are niche products more likely to benefit from
- recommendations and the other question we look at is what is it about a product that that makes it more likely to elicit
- a response from a consumer when it's recommended for example the ratings of the products or the price of the product
- or the type of the product do they influence whether recommendations are effective for that product
- so in one of our research studies we look at whether recommendation systems help us discover novel and nisha items
- that we might not otherwise discover but are a great fit for us personally what we find in our study is that because
- common recommendation systems are based on sales and ratings for example you know people who bought this also about
- this they are unable to surface truly novel items that have not been discovered by many other people and this
- tends to create a rich gets richer effect for popular items and it might also prevent consumers for finding a
- better product matches because of this bias for items that have been purchased by others or that have been rated well
- by others now this is a finding that a lot of people find surprising because many friends tell me that they do find
- very new items that they previously did not know about through recommendations and in line with that we find that these
- recommend recommendation systems can push us as individuals to new items but they push all of us towards the same new
- items and does at the aggregate level we don't see this great increase in diversity of purchases from consumers so
- in another research study we looked at what is it about a product that makes it more likely to elicit a response from a
- consumer when it's recommended for example we looked at interactions between a products rating and the
- recommendation response and we find that as one would expect recommendations help
- all kinds of products whether they rated high or whether they rated low but interestingly we find that it's the
- products that have a low average ratings that elicit a greater response from the
- consumer that is the purchase probability of a product goes up a lot more when for lower rated products than
- high rate higher rated products and this tells us that recommendations and ratings are in some way substitutes so
- if a product has high ratings to begin with then the recommendation has an impact but it's not as great but when it
- has low ratings you know in the absence of the recommendation we might not even respond to that product
- but when that product is recommended then we are willing to give the product the benefit of doubt maybe the product
- isn't right for everyone out there but perhaps it's right for me and so we find that recommendations and ratings can be
- substitutes as well another aspect we looked at is whether the type of the product matters so we classified all the
- products in our data set into what we call utilitarian products and hedonic products so utilit ate it utilitarian
- products are products that serve some functional purpose for example appliances or groceries and hedonic
- products are products that don't serve a functional purpose and really appeal to
- some sensory perception for example jewelry and we found that recommendations have a you know low to
- moderate impact for utilitarian products but it is for the hedonic products that
- they have a very significant impact and these hedonic products you know things like jewelry where we don't really mean
- it and when a recommendation kind of suggests that this is a great fit for us or people with similar tastes like this
- product that really moves the needle in terms of making us respond to that recommendation we look at other things
- like denote a description of the product matters the price does it matter and ute
- and we find what you would expect you know people respond to recommendations for lower price products and higher
- price products and where there's better description for the product and if there
- is very limited description so there are two conclusions that surprised us that we didn't expect a
- priori one was that recommendations don't necessarily help us discover niche products and that is interesting because
- there has been a lot of discussion for at least a decade now about how online systems whether it's search engines or
- personalized recommendations they will help us find niche items and they will help benefit you know what we call the
- long tail the products that are not super popular that almost don't get produced that may get produced but don't
- really sell much and the promise of recommendation systems is they really give a fair opportunity for these kinds
- of products and we find that for the common designs it doesn't happen and that's pretty surprising there are some
- designs where you you make design modifications and you favor nisha items you can make it work but most common
- designs that are used at most retailers they don't do that and we found that you
- know it might have the opposite effect now another result that surprised us was that recommendations and ratings or
- substitutes again our priori we expected that you know people will respond to recommendations when they are highly
- rated and we did find that when they're highly rated people respond to recommendations but what surprised us
- was that their responses even greater when the products have a lower rating and so that suggests that there's this
- effect of recommendations as substitutes for ratings and we hadn't predicted that
- beforehand so our research has implications for retailers for producers and even
- consumers for retailers to the extent that their strategy is to offer a wide product assortment is our research
- suggests that the choices of technology they make may not always be consistent with that strategy and they need to
- think harder about the technology choices for example Amazon strategy is that you can find any product on earth
- at Amazon and really wide product assortment is its strategy similarly many online retailers also offered white
- product assortment so our research suggests that if you offer great product assortment you also need to think about
- how will consumers discover that white product assortment and recommendations are an important part of the solution
- but they don't often work in practice because they have this bias towards products that have been bought before
- and that have been rated before and so our research suggests that they need to think about technology choice and think
- about how to modify this common design so it is consistent with their product choices in general for producers you
- know research shows that recommendations and similar algorithms they drive consumer choice in a big way so
- producers need to think hard about how the game discovered by these algorithms today producers are used to thinking
- about how do I how does our product get discovered by consumers they need to also ask how do how does our product get
- discovered by algorithms and for consumers we find that these systems are great at helping us as individuals
- discover new products but at the aggregate level we are not seeing that diversity which is not necessarily
- troublesome for consumers but it does suggest that there are products out there that could be this needle in the
- haystack perfect product for you which may not be surfaced by recommendations so one has to be open to other sources
- of discovery as well so there's a lot of talk these days about big data and analytics and how
- there's so much data and companies are building intelligent algorithms that can help them be smart that can help
- consumers find products they like and so on and our research shows that these efforts do work but at the same time we
- have to be cautious about unintended consequences so for example the idea of recommendations is that they help us
- find novel items but we don't want them to have biases built into them where they favor certain kinds of items versus
- others and to the extent they favor certain kinds of items that might have an unintended consequence and we can
- think about that say in the context of news if we all consume news through personalized recommendations we may not
- always get that breadth of perspective we want and so algorithms might be driving a lot of our choice with media
- and we need to think hard about how big data and algorithms can be be biased and
- and largely they work but they do have some biases we need to be cautious about so one of the things that's really novel
- about our work is that our research is informed by really large-scale data and analysis of that data there have been a
- lot of theories about how recommendations impact consumer choice what kinds of products they favor when
- consumers respond to them and so on in practice there had been very little empirical evidence and that's partly
- because in order to answer these questions for example do they favor niche items or mainstream items we need
- a contrast between people exposed to recommendations and unexposed to recommendations and for most retailers
- they observe consumers only after they come to their website and are exposed to recommendations and so our study is
- based on and an experiment that was done with a large retailer in North America with whom we ran an a/b experiment where
- some people got recommendations some did not and we ran this for hundreds of different product categories and so we
- were able to not only get that contrast needed to answer the question but we are
- also able to generalize beyond a single product category because we had so many different products and that I think is
- one of the things that sets this research apart which is that it's based on data it's based on concrete evidence
- and it's based on a large enough sample point and a large very representative sample of consumers
- so in terms of next steps we're trying to generalize some of our results beyond recommendations and think about all
- kinds of search tools we have online and so they include social tools and social
- news feeds for example on Facebook we discover products it includes search engines we're also trying to understand
- this world from the producers perspective as I mentioned you know there are implications for producers and
- producers need to think hard about how consumers will discover their products through algorithms and so they need to
- think about discovery by the algorithms as well and there's very limited understanding of what is it about a
- product that makes a recommendation pick that product among thousands of potential candidates and so we're trying
- to study that and hopefully we'll provide some insights to producers so that they can be active participants in
- helping their products get discover rather than passive observers you
Brief Summary
Personalized recommendations shape consumer choices but often favor mainstream products over niche options.
Key points
- Impact of Recommendations on Consumer Choice. Research shows that personalized recommendations significantly influence consumer choices online.
- Surprising Findings on Product Ratings. Lower-rated products elicit a greater response when recommended than higher-rated ones.
- Niche Products and Recommendations. Contrary to popular belief, recommendations may not help discover niche products.
Episode highlights
- Impact of Recommendations on Consumer ChoiceResearch shows that personalized recommendations significantly influence consumer choices online.“Recommendations drive anywhere from a quarter to a third of choices consumers make online.”0:46Dec 4, 2015
- Surprising Findings on Product RatingsLower-rated products elicit a greater response when recommended than higher-rated ones.“Recommendations and ratings are in some way substitutes.”3:51Dec 4, 2015
- Niche Products and RecommendationsContrary to popular belief, recommendations may not help discover niche products.“Recommendations don't necessarily help us discover niche products.”5:43Dec 4, 2015
Episode quotes
Tension
30/ 100Low tension
