Overcoming "Algorithm Aversion"
Ask about Knowledge at Wharton
Episode Description
Many people are averse to using algorithms when making decisions, preferring to rely on their instincts. New Wharton research says a simple adjustment can help them feel differently. For more: http://knlg.net/2kvYtDR 📂 Technology 🏷️ Research
Read the full transcriptHide the full transcript
- we're here today with Wharton professors Joe Simmons and Kate Massey to talk about some of their new research which
- focuses on algorithm aversion and how to stop it Joe and Katie thanks for being with us today
- thank you so first of all could you give us a brief summary of this research and
- also I know this is actually the papers actually a follow-up to something that you had done previously yeah so so we're
- studying a phenomenon called an algorithm aversion which is basically the tendency for people to not want to
- follow specific evidence-based rules when they make decisions even though a lot of the research that we do in
- judgment decision-making shows that that's exactly the way that you should be making judgments and forecasts so a
- lot of people just want to you know rely on their gut or go to the seat of their
- pants they don't want to rely on consistent evidence-based rules and they should and so we've been studying for a
- couple years now you know first of all why don't they or under what circumstances don't they want to rely on
- these algorithms and then our second paper the one you you asked about is about how to get people to be more
- likely to rely on algorithms and so the answer to our first question we basically found that people are if you
- tell them you're going to make a forecast an algorithm is going to give you some advice or you can go with your
- own opinion what do you want to do and you just ask them that they're actually okay with saying I'll use the algorithm
- however once you give them some practice and let them see how their algorithm performs now all of a sudden they don't
- want to use it anymore and that's because they see the algorithm make mistakes and once they see algorithms or
- computers make mistakes they don't want to do it anymore even though the algorithm or computer is going to make a
- smaller mistake or more infrequent mistakes than they themselves are going to make the algorithm supposed to be
- perfect right so people want algorithms to be perfect and they expect them to be
- perfect even though of course what we really want is for them to simply be a little more a little better than than
- the humans and so our second paper our first paper is kind of pessimistic and shows that once people see the
- algorithms do its thing they don't want to use it our second paper shows that actually you can get people to use
- algorithms as long as you give them a little bit of control over ETSU say the algorithm
- tells you that this person is going to have a GPA of 3.2 what do you think their GPA is going to be and they don't
- want to just go with a 3.2 the algorithm says but if you say you can adjust it a
- little bit you can adjust it by 0.1 then they're like okay I'm fine to use the algorithm and so we basically find as
- long as you give people a little bit of control over these things they're more likely to use them and that's that's
- pretty good news so this is we operationalize this in experimental context but we're motivated by a
- real-world context so some of the early ideas for this research came from working with companies where we would go
- in with models for decision-making and in particular this is about hiring and recruiting new employees and based on
- many years worth of data and some pretty good analytics we'd have advice and we were sure that we had the best advice
- going and yet those organizations would be reluctant to use it because they want
- to rely on just their intuition they're reluctant to use those models so it's very common and hiring it's very common
- in performance evaluation it's even common increasingly in some fields where they automate decision-making like how
- to manage a hedge fund or what should the sales forecast be for some product so those are all places where
- increasingly automatically generated forecasts or advice is available we call it an algorithm and the final
- decision-maker has discretion over whether they listen to that advice or whether they use their own or they use
- some blend so your key takeaway was basically that people are a little less averse to using algorithms that they
- have some control but there is a conclusion that surprised you in how much control you had to give them or you
- could give them to make them feel better so tell us a little bit about that well
- we were agnostic on how much control would be necessary to kind of get them to buy end the downside of given and
- controls they start degrading the algorithms they make they're not as in most domains about as good as the model
- and so the more of their opinion is in there the worse it performs and so in some sense you'd like to give them as
- little control as possible and yet still have them Buy in we didn't know what the
- answer that would be and we got early evidence that it wasn't gonna be very much as we started testing the limits of
- it and we found that we could give them just a little bit of control move something around 5% or so and they
- would be much more interested in using the algorithm and then if you give them more it doesn't increase their list at
- all it's just give them a little bit it's about the same as given the moderate influence yeah and cave was
- mentioning what's nice about that is that every when they adjust algorithms they make them worse but if they can
- only adjust it this much they can only make it is that much worse and they're more like entrance they're just more
- likely to use it in that case their final judgments will end up being correlated with the algorithm close to
- perfectly and so we can't get people to use algorithms 100% we can get them to use algorithms 99% and that bat
- massively improved their judgments so tell me a little bit about so with this paper like applying it in real life like
- how could if I'm a business owner or even someone who's going to be charged with using one of these algorithms how
- might I apply this research in real life so the the overarching lesson would be that you don't simply impose a
- monolithic model or blackbox model and say this is how you use judgment people will fight that this is how you this is
- how you should cause by your decision making people will fight that you want to let them have discretion and that's
- going to look different in different places so consider graduate school making admissions they they rank their
- applicants and at some point they cut a line and they make exceptions they move people around you can automate some of
- that process even if you use their judgment to provide inputs to the model you can use them a model automatic model
- to say these are the folks that you should take on one hand you could say here's a model this is what it says take
- it or leave it we're going to automate the process you'll basically have a revolt on your
- hand but if you say here's a model its advisory we suggest that you consider it if you want to move things around move
- them around we've actually worked with schools on in exactly this way and what you find is that they're a little
- skeptical early on they lean on the model some and over time they lean on the model more and eventually they're
- practically using the entire model as it is even though they have discretion to change as much as they want so sort of a
- get-to-know-you process there's very much a good to know you process with it and now how important is it with these
- models I mean just I would also make the presentation if it will be important that you make sure that people know that
- they have this control but you're sort of presenting it in such a way that like here's what you can
- control and here's how much control or can it be a little more available Matt yeah I basically think the important
- thing is to avoid an all-or-nothing framing like you are all you you have to stick with the algorithm 100% of the
- time if people think that based on how you've described it they are going to push back but instead you can frame it
- as you know even like ninety nine percent of time we're gonna go with the algorithm but you have the option to
- change the algorithm or to not go with the algorithm at a given moment I think that that's going to go and make people
- a lot more amenable to using it I mean in other context of this might matter is like self-driving cars I mean you can
- imagine people saying like I'm not feeling comfortable being in a self-driving car if they have no control
- whatsoever but if they say well there's this sort of thing you can do it's a little bit difficult and unusual but
- there's this thing you can do to gain control over the car in circumstances where you might need to do it now we
- found that people never need to use this but it does exist we would predict that
- in that circumstance you will be much more amenable to getting in getting in a self-driving car because there's some
- control it's like autopilot is usually safer than real pilots but people want a pilot there even though lots of plane
- crashes are due to pilot error they feel better about that and so I think our research sort of speaks to that a bit
- now are there other stories in the news that might apply to this research Grenoble self-driving cars have
- obviously been in the news quite a bit how about election forecasts yeah so I think you know back in November of
- course we had a presidential election that surprised the world and there are a bunch of people out there predicting
- based on past polling information what the election was going to to look like like probably the most famous cases made
- silver who's who writes for fivethirtyeight.com and he basically said that there is a
- 70% chance that Hillary Clinton would win the election and a 30% chance that Donald Trump would win and of course you
- know Donald Trump won and there was a lot of pushback against Nate Silver at the end of it being like you know you
- were you know you were wrong and and we think in part of your model with your model your model was wrong
- and the thing is it wasn't necessarily wrong 30% happens 30% of the time but I think that it's it's the kind of thing
- where when individual I think when individual pundants go out there and say one thing is going
- to happen I don't think they get as much blowback as when the person who uses statistics and the model and an
- algorithm that people expect to be right 100% of the time when they actually wind
- up getting it wrong in that particular case so I think the blowback that we've seen in in the direction of nate silver
- has been in line with what we found before so it circles back to their first paper where people are just much harder
- on models and algorithms from there than they are on people they're just more forgiving they we've explored it a
- little bit but the bottom line is that they're held to a higher standard yeah now does that I mean does that make
- sense or is that I mean I would think I can see what people would feel that way although no one's perfect or anything is
- perfect well you want there a variety of reasons we think people do this one of them is that they believe that people
- can improve over time whereas a model is relatively fixed and that's the way they
- feel in a way both of those things are not necessarily true models can improve over time and people don't necessarily
- improve over time so the psychology of that I think is compelling but it's not necessarily correct the certainly would
- be some settings where people can improve more than a model but we think that people have that intuition more
- than actually should now is anything - kind of sets this research apart from other work that's been done in this area
- that we're not the first to talk about the difference between model judgment and human judgment it's been established
- for decades now that models are quite good we're relatively early into trying to understand why that is and then how
- you fix it yeah so right so not many people have documented previously the reasons why people are averse to using
- algorithms there's been some anecdotal research there's been some writings about how people don't like these things
- but no one's really looked at it systematically before and so that's that's what we sort of started and again
- back to the motivation motivation was we work with organizations we want them to
- use more models we need to know how to break down that is you don't you can't prescribe
- anything into you better understand of why and exist because you give someone a model and I don't use it you might as
- well know delirious frustrated we did never wrote a few times and now is there anything what's next for this research
- we took a couple of ideas are you know we continue to play with some factors that might contribute to the people's
- reluctance to use acronyms but we also want more real-world tests of it so if we go work with professionals with real
- money at stake do they fall in the same biases and is there other ways for us to
- help them so we have a couple of organizations we've talked to over time and they're interested in running
- experiments on their employees or their own their customers to see if what we see in the lab takes place in the field
- as well great thank you both so much for being with us today yeah you [Music]
Brief Summary
Wharton professors discuss algorithm aversion and how control increases reliance on algorithms in decision-making.
Key points
- Algorithm Aversion Explained. Exploring why people hesitate to trust algorithms despite their effectiveness.
- The Power of Control. Giving users a little control increases their willingness to use algorithms.
Episode highlights
- Algorithm Aversion ExplainedExploring why people hesitate to trust algorithms despite their effectiveness.“People want algorithms to be perfect.”1:50Feb 13, 2017
- The Power of ControlGiving users a little control increases their willingness to use algorithms.“As long as you give people a little bit of control, they’re more likely to use them.”2:36Feb 13, 2017
Tension
40/ 100Low tension





