Overcoming "Algorithm Aversion"

Knowledge at Wharton

Ask about Knowledge at Wharton

Episode Description

Many people are averse to using algorithms when making decisions, preferring to rely on their instincts. New Wharton research says a simple adjustment can help them feel differently. For more: http://knlg.net/2kvYtDR 📂 Technology 🏷️ Research

Read the full transcriptHide the full transcript
  1. we're here today with Wharton professors Joe Simmons and Kate Massey to talk about some of their new research which
  2. focuses on algorithm aversion and how to stop it Joe and Katie thanks for being with us today
  3. thank you so first of all could you give us a brief summary of this research and
  4. also I know this is actually the papers actually a follow-up to something that you had done previously yeah so so we're
  5. studying a phenomenon called an algorithm aversion which is basically the tendency for people to not want to
  6. follow specific evidence-based rules when they make decisions even though a lot of the research that we do in
  7. judgment decision-making shows that that's exactly the way that you should be making judgments and forecasts so a
  8. lot of people just want to you know rely on their gut or go to the seat of their
  9. pants they don't want to rely on consistent evidence-based rules and they should and so we've been studying for a
  10. couple years now you know first of all why don't they or under what circumstances don't they want to rely on
  11. these algorithms and then our second paper the one you you asked about is about how to get people to be more
  12. likely to rely on algorithms and so the answer to our first question we basically found that people are if you
  13. tell them you're going to make a forecast an algorithm is going to give you some advice or you can go with your
  14. own opinion what do you want to do and you just ask them that they're actually okay with saying I'll use the algorithm
  15. however once you give them some practice and let them see how their algorithm performs now all of a sudden they don't
  16. want to use it anymore and that's because they see the algorithm make mistakes and once they see algorithms or
  17. computers make mistakes they don't want to do it anymore even though the algorithm or computer is going to make a
  18. smaller mistake or more infrequent mistakes than they themselves are going to make the algorithm supposed to be
  19. perfect right so people want algorithms to be perfect and they expect them to be
  20. perfect even though of course what we really want is for them to simply be a little more a little better than than
  21. the humans and so our second paper our first paper is kind of pessimistic and shows that once people see the
  22. algorithms do its thing they don't want to use it our second paper shows that actually you can get people to use
  23. algorithms as long as you give them a little bit of control over ETSU say the algorithm
  24. tells you that this person is going to have a GPA of 3.2 what do you think their GPA is going to be and they don't
  25. want to just go with a 3.2 the algorithm says but if you say you can adjust it a
  26. little bit you can adjust it by 0.1 then they're like okay I'm fine to use the algorithm and so we basically find as
  27. long as you give people a little bit of control over these things they're more likely to use them and that's that's
  28. pretty good news so this is we operationalize this in experimental context but we're motivated by a
  29. real-world context so some of the early ideas for this research came from working with companies where we would go
  30. in with models for decision-making and in particular this is about hiring and recruiting new employees and based on
  31. many years worth of data and some pretty good analytics we'd have advice and we were sure that we had the best advice
  32. going and yet those organizations would be reluctant to use it because they want
  33. to rely on just their intuition they're reluctant to use those models so it's very common and hiring it's very common
  34. in performance evaluation it's even common increasingly in some fields where they automate decision-making like how
  35. to manage a hedge fund or what should the sales forecast be for some product so those are all places where
  36. increasingly automatically generated forecasts or advice is available we call it an algorithm and the final
  37. decision-maker has discretion over whether they listen to that advice or whether they use their own or they use
  38. some blend so your key takeaway was basically that people are a little less averse to using algorithms that they
  39. have some control but there is a conclusion that surprised you in how much control you had to give them or you
  40. could give them to make them feel better so tell us a little bit about that well
  41. we were agnostic on how much control would be necessary to kind of get them to buy end the downside of given and
  42. controls they start degrading the algorithms they make they're not as in most domains about as good as the model
  43. and so the more of their opinion is in there the worse it performs and so in some sense you'd like to give them as
  44. little control as possible and yet still have them Buy in we didn't know what the
  45. answer that would be and we got early evidence that it wasn't gonna be very much as we started testing the limits of
  46. it and we found that we could give them just a little bit of control move something around 5% or so and they
  47. would be much more interested in using the algorithm and then if you give them more it doesn't increase their list at
  48. all it's just give them a little bit it's about the same as given the moderate influence yeah and cave was
  49. mentioning what's nice about that is that every when they adjust algorithms they make them worse but if they can
  50. only adjust it this much they can only make it is that much worse and they're more like entrance they're just more
  51. likely to use it in that case their final judgments will end up being correlated with the algorithm close to
  52. perfectly and so we can't get people to use algorithms 100% we can get them to use algorithms 99% and that bat
  53. massively improved their judgments so tell me a little bit about so with this paper like applying it in real life like
  54. how could if I'm a business owner or even someone who's going to be charged with using one of these algorithms how
  55. might I apply this research in real life so the the overarching lesson would be that you don't simply impose a
  56. monolithic model or blackbox model and say this is how you use judgment people will fight that this is how you this is
  57. how you should cause by your decision making people will fight that you want to let them have discretion and that's
  58. going to look different in different places so consider graduate school making admissions they they rank their
  59. applicants and at some point they cut a line and they make exceptions they move people around you can automate some of
  60. that process even if you use their judgment to provide inputs to the model you can use them a model automatic model
  61. to say these are the folks that you should take on one hand you could say here's a model this is what it says take
  62. it or leave it we're going to automate the process you'll basically have a revolt on your
  63. hand but if you say here's a model its advisory we suggest that you consider it if you want to move things around move
  64. them around we've actually worked with schools on in exactly this way and what you find is that they're a little
  65. skeptical early on they lean on the model some and over time they lean on the model more and eventually they're
  66. practically using the entire model as it is even though they have discretion to change as much as they want so sort of a
  67. get-to-know-you process there's very much a good to know you process with it and now how important is it with these
  68. models I mean just I would also make the presentation if it will be important that you make sure that people know that
  69. they have this control but you're sort of presenting it in such a way that like here's what you can
  70. control and here's how much control or can it be a little more available Matt yeah I basically think the important
  71. thing is to avoid an all-or-nothing framing like you are all you you have to stick with the algorithm 100% of the
  72. time if people think that based on how you've described it they are going to push back but instead you can frame it
  73. as you know even like ninety nine percent of time we're gonna go with the algorithm but you have the option to
  74. change the algorithm or to not go with the algorithm at a given moment I think that that's going to go and make people
  75. a lot more amenable to using it I mean in other context of this might matter is like self-driving cars I mean you can
  76. imagine people saying like I'm not feeling comfortable being in a self-driving car if they have no control
  77. whatsoever but if they say well there's this sort of thing you can do it's a little bit difficult and unusual but
  78. there's this thing you can do to gain control over the car in circumstances where you might need to do it now we
  79. found that people never need to use this but it does exist we would predict that
  80. in that circumstance you will be much more amenable to getting in getting in a self-driving car because there's some
  81. control it's like autopilot is usually safer than real pilots but people want a pilot there even though lots of plane
  82. crashes are due to pilot error they feel better about that and so I think our research sort of speaks to that a bit
  83. now are there other stories in the news that might apply to this research Grenoble self-driving cars have
  84. obviously been in the news quite a bit how about election forecasts yeah so I think you know back in November of
  85. course we had a presidential election that surprised the world and there are a bunch of people out there predicting
  86. based on past polling information what the election was going to to look like like probably the most famous cases made
  87. silver who's who writes for fivethirtyeight.com and he basically said that there is a
  88. 70% chance that Hillary Clinton would win the election and a 30% chance that Donald Trump would win and of course you
  89. know Donald Trump won and there was a lot of pushback against Nate Silver at the end of it being like you know you
  90. were you know you were wrong and and we think in part of your model with your model your model was wrong
  91. and the thing is it wasn't necessarily wrong 30% happens 30% of the time but I think that it's it's the kind of thing
  92. where when individual I think when individual pundants go out there and say one thing is going
  93. to happen I don't think they get as much blowback as when the person who uses statistics and the model and an
  94. algorithm that people expect to be right 100% of the time when they actually wind
  95. up getting it wrong in that particular case so I think the blowback that we've seen in in the direction of nate silver
  96. has been in line with what we found before so it circles back to their first paper where people are just much harder
  97. on models and algorithms from there than they are on people they're just more forgiving they we've explored it a
  98. little bit but the bottom line is that they're held to a higher standard yeah now does that I mean does that make
  99. sense or is that I mean I would think I can see what people would feel that way although no one's perfect or anything is
  100. perfect well you want there a variety of reasons we think people do this one of them is that they believe that people
  101. can improve over time whereas a model is relatively fixed and that's the way they
  102. feel in a way both of those things are not necessarily true models can improve over time and people don't necessarily
  103. improve over time so the psychology of that I think is compelling but it's not necessarily correct the certainly would
  104. be some settings where people can improve more than a model but we think that people have that intuition more
  105. than actually should now is anything - kind of sets this research apart from other work that's been done in this area
  106. that we're not the first to talk about the difference between model judgment and human judgment it's been established
  107. for decades now that models are quite good we're relatively early into trying to understand why that is and then how
  108. you fix it yeah so right so not many people have documented previously the reasons why people are averse to using
  109. algorithms there's been some anecdotal research there's been some writings about how people don't like these things
  110. but no one's really looked at it systematically before and so that's that's what we sort of started and again
  111. back to the motivation motivation was we work with organizations we want them to
  112. use more models we need to know how to break down that is you don't you can't prescribe
  113. anything into you better understand of why and exist because you give someone a model and I don't use it you might as
  114. well know delirious frustrated we did never wrote a few times and now is there anything what's next for this research
  115. we took a couple of ideas are you know we continue to play with some factors that might contribute to the people's
  116. reluctance to use acronyms but we also want more real-world tests of it so if we go work with professionals with real
  117. money at stake do they fall in the same biases and is there other ways for us to
  118. help them so we have a couple of organizations we've talked to over time and they're interested in running
  119. experiments on their employees or their own their customers to see if what we see in the lab takes place in the field
  120. as well great thank you both so much for being with us today yeah you [Music]

Brief Summary

Wharton professors discuss algorithm aversion and how control increases reliance on algorithms in decision-making.

Key points

  • Algorithm Aversion Explained. Exploring why people hesitate to trust algorithms despite their effectiveness.
  • The Power of Control. Giving users a little control increases their willingness to use algorithms.

Tension

40/ 100Low tension

Related episodes