Building Better Recommendation Engines

Knowledge at Wharton

Ask about Knowledge at Wharton

Episode Description

n today’s data-rich environment, everyone from retailers to content providers is constantly suggesting options they think consumers will like based on past behavior. Research by Wharton’s Kartik Hosanagar takes a closer look at the real impact of those recommendations. More: http://knlg.net/1PKVfEJ 📂 Marketing 🏷️ Research

Read the full transcriptHide the full transcript
  1. an important team of my research is how personalized recommendations and similar
  2. algorithms affect consumer choice we are all flooded by product choices today and
  3. these kinds of personalized recommendations play an important role in helping us discover new products are
  4. sorting through large choice sets and we see a personalized recommendations in a
  5. number of industries whether it's in retail for example Amazon's of people who bought this also bought this or in
  6. media such as Netflix or YouTube we see it with news as well for example Google News will recommend personalized news
  7. stories and we know they have a pretty significant impact on consumer choice for example at Amazon they drive
  8. anywhere from a quarter to a third of the choices that consumers make online so although we know that they have a big
  9. impact on consumer choice we don't fully understand what kinds of products are more likely to be accepted by consumers
  10. when recommended and when do recommenders work well and when they don't so in my research with Professor
  11. document Carnegie Mellon we look at two main questions the first is what kinds of products are more likely to benefit
  12. from recommendations specifically our mainstream products are niche products more likely to benefit from
  13. recommendations and the other question we look at is what is it about a product that that makes it more likely to elicit
  14. a response from a consumer when it's recommended for example the ratings of the products or the price of the product
  15. or the type of the product do they influence whether recommendations are effective for that product
  16. so in one of our research studies we look at whether recommendation systems help us discover novel and nisha items
  17. that we might not otherwise discover but are a great fit for us personally what we find in our study is that because
  18. common recommendation systems are based on sales and ratings for example you know people who bought this also about
  19. this they are unable to surface truly novel items that have not been discovered by many other people and this
  20. tends to create a rich gets richer effect for popular items and it might also prevent consumers for finding a
  21. better product matches because of this bias for items that have been purchased by others or that have been rated well
  22. by others now this is a finding that a lot of people find surprising because many friends tell me that they do find
  23. very new items that they previously did not know about through recommendations and in line with that we find that these
  24. recommend recommendation systems can push us as individuals to new items but they push all of us towards the same new
  25. items and does at the aggregate level we don't see this great increase in diversity of purchases from consumers so
  26. in another research study we looked at what is it about a product that makes it more likely to elicit a response from a
  27. consumer when it's recommended for example we looked at interactions between a products rating and the
  28. recommendation response and we find that as one would expect recommendations help
  29. all kinds of products whether they rated high or whether they rated low but interestingly we find that it's the
  30. products that have a low average ratings that elicit a greater response from the
  31. consumer that is the purchase probability of a product goes up a lot more when for lower rated products than
  32. high rate higher rated products and this tells us that recommendations and ratings are in some way substitutes so
  33. if a product has high ratings to begin with then the recommendation has an impact but it's not as great but when it
  34. has low ratings you know in the absence of the recommendation we might not even respond to that product
  35. but when that product is recommended then we are willing to give the product the benefit of doubt maybe the product
  36. isn't right for everyone out there but perhaps it's right for me and so we find that recommendations and ratings can be
  37. substitutes as well another aspect we looked at is whether the type of the product matters so we classified all the
  38. products in our data set into what we call utilitarian products and hedonic products so utilit ate it utilitarian
  39. products are products that serve some functional purpose for example appliances or groceries and hedonic
  40. products are products that don't serve a functional purpose and really appeal to
  41. some sensory perception for example jewelry and we found that recommendations have a you know low to
  42. moderate impact for utilitarian products but it is for the hedonic products that
  43. they have a very significant impact and these hedonic products you know things like jewelry where we don't really mean
  44. it and when a recommendation kind of suggests that this is a great fit for us or people with similar tastes like this
  45. product that really moves the needle in terms of making us respond to that recommendation we look at other things
  46. like denote a description of the product matters the price does it matter and ute
  47. and we find what you would expect you know people respond to recommendations for lower price products and higher
  48. price products and where there's better description for the product and if there
  49. is very limited description so there are two conclusions that surprised us that we didn't expect a
  50. priori one was that recommendations don't necessarily help us discover niche products and that is interesting because
  51. there has been a lot of discussion for at least a decade now about how online systems whether it's search engines or
  52. personalized recommendations they will help us find niche items and they will help benefit you know what we call the
  53. long tail the products that are not super popular that almost don't get produced that may get produced but don't
  54. really sell much and the promise of recommendation systems is they really give a fair opportunity for these kinds
  55. of products and we find that for the common designs it doesn't happen and that's pretty surprising there are some
  56. designs where you you make design modifications and you favor nisha items you can make it work but most common
  57. designs that are used at most retailers they don't do that and we found that you
  58. know it might have the opposite effect now another result that surprised us was that recommendations and ratings or
  59. substitutes again our priori we expected that you know people will respond to recommendations when they are highly
  60. rated and we did find that when they're highly rated people respond to recommendations but what surprised us
  61. was that their responses even greater when the products have a lower rating and so that suggests that there's this
  62. effect of recommendations as substitutes for ratings and we hadn't predicted that
  63. beforehand so our research has implications for retailers for producers and even
  64. consumers for retailers to the extent that their strategy is to offer a wide product assortment is our research
  65. suggests that the choices of technology they make may not always be consistent with that strategy and they need to
  66. think harder about the technology choices for example Amazon strategy is that you can find any product on earth
  67. at Amazon and really wide product assortment is its strategy similarly many online retailers also offered white
  68. product assortment so our research suggests that if you offer great product assortment you also need to think about
  69. how will consumers discover that white product assortment and recommendations are an important part of the solution
  70. but they don't often work in practice because they have this bias towards products that have been bought before
  71. and that have been rated before and so our research suggests that they need to think about technology choice and think
  72. about how to modify this common design so it is consistent with their product choices in general for producers you
  73. know research shows that recommendations and similar algorithms they drive consumer choice in a big way so
  74. producers need to think hard about how the game discovered by these algorithms today producers are used to thinking
  75. about how do I how does our product get discovered by consumers they need to also ask how do how does our product get
  76. discovered by algorithms and for consumers we find that these systems are great at helping us as individuals
  77. discover new products but at the aggregate level we are not seeing that diversity which is not necessarily
  78. troublesome for consumers but it does suggest that there are products out there that could be this needle in the
  79. haystack perfect product for you which may not be surfaced by recommendations so one has to be open to other sources
  80. of discovery as well so there's a lot of talk these days about big data and analytics and how
  81. there's so much data and companies are building intelligent algorithms that can help them be smart that can help
  82. consumers find products they like and so on and our research shows that these efforts do work but at the same time we
  83. have to be cautious about unintended consequences so for example the idea of recommendations is that they help us
  84. find novel items but we don't want them to have biases built into them where they favor certain kinds of items versus
  85. others and to the extent they favor certain kinds of items that might have an unintended consequence and we can
  86. think about that say in the context of news if we all consume news through personalized recommendations we may not
  87. always get that breadth of perspective we want and so algorithms might be driving a lot of our choice with media
  88. and we need to think hard about how big data and algorithms can be be biased and
  89. and largely they work but they do have some biases we need to be cautious about so one of the things that's really novel
  90. about our work is that our research is informed by really large-scale data and analysis of that data there have been a
  91. lot of theories about how recommendations impact consumer choice what kinds of products they favor when
  92. consumers respond to them and so on in practice there had been very little empirical evidence and that's partly
  93. because in order to answer these questions for example do they favor niche items or mainstream items we need
  94. a contrast between people exposed to recommendations and unexposed to recommendations and for most retailers
  95. they observe consumers only after they come to their website and are exposed to recommendations and so our study is
  96. based on and an experiment that was done with a large retailer in North America with whom we ran an a/b experiment where
  97. some people got recommendations some did not and we ran this for hundreds of different product categories and so we
  98. were able to not only get that contrast needed to answer the question but we are
  99. also able to generalize beyond a single product category because we had so many different products and that I think is
  100. one of the things that sets this research apart which is that it's based on data it's based on concrete evidence
  101. and it's based on a large enough sample point and a large very representative sample of consumers
  102. so in terms of next steps we're trying to generalize some of our results beyond recommendations and think about all
  103. kinds of search tools we have online and so they include social tools and social
  104. news feeds for example on Facebook we discover products it includes search engines we're also trying to understand
  105. this world from the producers perspective as I mentioned you know there are implications for producers and
  106. producers need to think hard about how consumers will discover their products through algorithms and so they need to
  107. think about discovery by the algorithms as well and there's very limited understanding of what is it about a
  108. product that makes a recommendation pick that product among thousands of potential candidates and so we're trying
  109. to study that and hopefully we'll provide some insights to producers so that they can be active participants in
  110. helping their products get discover rather than passive observers you

Brief Summary

Personalized recommendations shape consumer choices but often favor mainstream products over niche options.

Key points

  • Impact of Recommendations on Consumer Choice. Research shows that personalized recommendations significantly influence consumer choices online.
  • Surprising Findings on Product Ratings. Lower-rated products elicit a greater response when recommended than higher-rated ones.
  • Niche Products and Recommendations. Contrary to popular belief, recommendations may not help discover niche products.

Tension

30/ 100Low tension

Related episodes