Is AI Replacing Human Thinking? The Rise of "Cognitive Surrender"

Knowledge at Wharton

Ask about Knowledge at Wharton

Episode Description

ABOUT THE EPISODE Gideon Nave explains that the real danger of AI may not be that it outsmarts humans, but that humans increasingly stop thinking for themselves. Steve Shaw discusses their research showing that shows people readily “cognitively surrender,” often adopting AI answers, even incorrect ones, with high confidence. They explain how this emerging “third system” of artificial cognition is reshaping decision-making in business, education, and everyday life. The conversation explores when outsourcing thought can be helpful—and when it may quietly erode critical thinking skills.

ABOUT THE PODCAST Every day, business scholars answer pressing questions in their research - but what do their insights mean for you? In this podcast, Wharton faculty dive into what inspired their studies and how their findings resonate with the world today. Learn how research insights translate into knowledge you can use, with host Dan Loney.

#DecisionScience #BehavioralScience #AIPsychology #DecisionMakingSkills #AIandSociety #TechnologyAndSociety ----- The Wharton School of the University of Pennsylvania is committed to sharing its intellectual capital through Knowledge at Wharton, the school’s online business analysis journal.

Launched in 1999, Knowledge at Wharton offers free access to:

- Articles, podcasts, and videos highlighting Wharton faculty research and analysis of current business trends - Interviews with book authors from Wharton School Press - In-depth series of curated content like Wharton Executive Education’s Nano Tools for Leaders - A searchable database of more than 10,000 articles covering all aspects of business - A weekly newsletter that delivers Knowledge at Wharton insights directly to your inbox

Visit the Knowledge at Wharton homepage: https://whr.tn/3qUpSBE Subscribe to the Knowledge at Wharton YouTube channel: https://whr.tn/3DEAgAU

Read the full transcriptHide the full transcript
  1. You know, I'm just worried about the future. I think we all focus on this point where this singularity is going to happen,
  2. when AI will really outsmart us. And everybody thinks that this point will come from AI getting better and better. But there is
  3. an alternative story here of humans becoming more and more relying— reliant on AI. And just like we now have right now, an
  4. air conditioner that can set our temperature easily, and we can move from one place to another without using any physical
  5. activity, just like many of us have lost something because of this maybe cultural or technological evolution, we may
  6. lose, as a species, something very critical to our existence, which is our capacity to think.
  7. Welcome to <i>The Ripple Effect</i>, the podcast that takes you on a journey through the minds of Wharton faculty.
  8. I'm your host, Dan Loney. And in each episode, we'll be diving deep into the inspiration behind the groundbreaking research
  9. that Wharton professors have conducted, and exploring how their findings resonate with the world today.
  10. We are certainly seeing how much artificial intelligence is impacting our lives with the advent of the technology and how
  11. it is changing things like our work. But how does it change our decision process? That's a question that is addressed in
  12. recent research, and pleasure to have the authors of that research joining us here today. Gideon Nave, who's an Associate
  13. Professor of Marketing here at the Wharton School, and Steven Shaw, who is a postdoctoral researcher here at Wharton.
  14. Gentlemen, great to have you both with us. Thanks for your time today. Thanks for having us.
  15. Let me start out, I guess, what was the genesis of wanting to do this specific type of
  16. research? Steven, I'll start with you. Looking at the decision process. I mean, we see about it from a business perspective, and
  17. you know, the day to day impact there, but the decision process is probably just as important.
  18. Yeah. I mean, I think, you know, we can observe things in nature. And just observing how integrated AI has
  19. become in our daily lives, we sort of felt that the ability to actually outsource thinking hadn't really been
  20. studied itself. So it's sort of a profound idea. A bit provocative, I would say, in the paper, that— you know, with these
  21. AI tools that are available, they're so ingrained in our daily lives and decision processes that we now have the
  22. option or ability to actually outsource thinking itself. Gideon, what was it that drew your attention to this side of
  23. artificial intelligence? You know, I've been studying decision processes for a while,
  24. and we know that there are certain theories that account for how people make decisions and how they use all the
  25. sorts of types of decision processes, from intuition to more deliberate analytical thinking. And I think that, you
  26. know, technology in the past was integrated as a kind of a block that you can maybe offload some of your cognition into, perform
  27. certain tasks, like using a calculator or using a GPS to navigate. But to me, it sounds like the current theories of how
  28. humans make judgments and decisions must be going through some update once we have these devices that really, as Steve
  29. said, can really replace thinking itself. And it's very important to start trying to account for this theoretically and define
  30. all sorts of tendencies and constructs so we can really study it in the future. And Steven, I guess this is kind of one of the next steps. When
  31. you think about how artificial intelligence is truly impacting our lives, we probably think about people's reasoning, and
  32. have thought about it off and on for many decades. But obviously not in the perspective of how artificial intelligence is going
  33. to impact that process. Yeah, yeah. And that's, again, one of the arguments that we
  34. make in the paper is really, we have these classic models. For example, the dual process model that Giddy has talked about,
  35. intuitions and deliberations. But the reliance that we see, and the ability and the integration of AI that we see in society now
  36. just really changes the options that we have for making decisions. And it does so so dramatically that we need to
  37. update our— many of our old models in cognitive— cognitive science and psychology and marketing.
  38. Right. And so Gideon, it's no longer kind of a dual process. It's a three-tiered process that we probably have to bring into
  39. play here, correct? Yeah, we added kind of artificial cognition as a third model. And I think what's interesting is not only
  40. that we have this system and it adds to it, but the existence of this system, just its mere existence, changes how we engage
  41. our other thinking modes. It changes how we use intuition, it changes how we use deliberation, and it also, surprisingly, also
  42. changes how confident we are in our responses, even ones that are not really critically examined by ourselves. So there
  43. are a lot of interesting effects going on here. And some of them are maybe a bit dark, going to the future, if we think that
  44. these really are going to affect the tendencies in behavior of humans nowadays. Steven, does— does having a dual process now become— adding this
  45. third component— does the dual process run into any roadblocks along the way, as AI is becoming a greater component of our lives?
  46. Sure, yeah. And that's, you know, one of the big things we show experimentally in the paper. We call it cognitive
  47. surrender. Basically, you know, with access— just simply having the option of having AI available for decision making—
  48. people can surrender their thought to AI and let it sort of think for them. So they're basically subverting the whole
  49. internal brain set of processes, subverting and substituting system one, system two, and instead, adopting answers from
  50. what we call system three. And as Giddy said, we saw— even, you know, when cognitive surrender is engaged, people adopt those
  51. answers and are more confident in those answers. Do they rely on them more often, then?
  52. We saw— so in our experiments, we saw— it was optional. You know, we just put ChatGPT in a window. And they were doing some
  53. sort of logic and reasoning questions. We said, you can use AI if you want to, but you don't have to. And so we saw over 50%
  54. of the time, they consulted ChatGPT. And once they consulted ChatGPT, adoption rates were very high, even when AI was
  55. incorrect, gave them the incorrect answer, which we experimentally manipulated, people adopted the answer over
  56. 80% of the time. Giddy, one of the things that I know I have talked with many people about is the business side of how bringing AI into the
  57. business world is going to impact so many different processes on a day-to-day basis in companies. One of the things
  58. that is brought up is how it impacts labor and how people kind of interact with artificial intelligence. It seems like part
  59. of this research also can kind of lead us down the path to better understand what that relationship is going to be like
  60. with companies and their employees, and what's going to be expected from them as we move forward because of the AI
  61. component and the decision process being impacted. Yeah. I mean, let's say it this way. If we are completely surrendering
  62. our thinking to AI, what value do we bring to a company? It's not clear. So even this tendency to surrender, maybe, something
  63. that in the future, companies would want to consider before they're hiring somebody. I don't want to have some person that
  64. just basically is giving me what the AI can already give me. I can get this by myself. And that also tells us, you
  65. know, if we think of the education of the next business leaders, obviously this is a skill that we want to make sure
  66. that people have and not lose, the capacity to think critically, the capacity to be able to check what the AI is
  67. giving you. It's become more and more important over time. So this is kind of a muscle— muscle that we have, that hopefully we
  68. are not going to lose over time. How much of a challenge, Steven, potentially, is that? Because I
  69. think at times we do feel like AI, is this— is this beautiful benefit that we now have in our lives. But we still need to have
  70. that critical thinking as we move forward on so many things, not only in our business lives, but in our personal lives as well.
  71. Yeah. And that's the key question, right? I mean, how do we maintain critical thinking skills in the age of AI? And you
  72. said, you know, how fundamental is that? Well, I mean, I think we're only at the beginning of the age of AI, right? And this
  73. technological integration is just getting started. Right now, we are constrained by communicating with the LLMs
  74. through our phones or our computers. And as those barriers reduce, right, that integration is just going to become,
  75. you know, stronger. What did you take away from doing this research, Steven, that really resonated with you? Either— maybe even something that
  76. you didn't even expect you were going to see play out? How readily people were willing to cognitively surrender.That
  77. was pretty shocking. And how well the experiments worked, you know, based on our theoretical contribution of this tri-system
  78. theory of cognition. Gideon, what about you? You know, I'm just worried about the future. I think we all focus on this point
  79. where this singularity is going to happen, when AI will really outsmart us. And everybody thinks that this point will come
  80. from AI getting better and better. But there is an alternative story here of humans becoming more and more relying—
  81. reliant on AI. And just like we now have, right now, an air conditioner that can set our temperature easily, and we can
  82. move from one place to another without using any physical activity, just like many of us have lost something because of
  83. this, maybe, cultural or technological evolution we may lose, as a species, something very critical to our existence, which
  84. is our capacity to think. But going back to something you said a little while ago, then
  85. the learning process for people in general is going to change, because we have this component, and it is going to be a part of
  86. almost everything we do moving forward. We may or we may not. You know, typically, we know that the
  87. technology— technological development moves much faster than any policy change. It's difficult for policy and
  88. educational systems to respond very quickly to what is happening here. I don't know what going to be the
  89. answer. And of course, there are a lot of different perspectives and different disagreements. What we suggested is provocative
  90. to some degree. It's not obvious that everybody would favor it. Even, you know, you can say, "I want freedom. Let the market
  91. solve itself." And with freedom, sometimes you get outcomes that are, at the end of the day, determined by power imbalances.
  92. I don't know where we are headed here. We are, as academics— are here to maybe try to blow the whistle on it. We
  93. can't do much more than that. Steve, having done this research now, is there a next logical
  94. step that maybe you would like to take as you maybe delve deeper into this further?
  95. Sure. Yeah. I mean, so in, we— one of the part of— the part of the title of the paper is "the rise of cognitive surrender," right?
  96. So we now know that this phenomenon exists. And the question is, when is it adaptive? When is it good to
  97. outsource your thought to AI, right? AI gives us access to super intelligence. There are many instances where, you know,
  98. turning off thought can be a good thing. But in a lot of high stakes contexts— education, healthcare, right? We don't want
  99. that to be happening. And so how do we fight the rise of cognitive surrender in those contexts? Is it on the user
  100. side, on the human side, through AI literacy and training, or is it on the UX design side by putting in different types of
  101. prompting or roadblocks that make sure or try to induce critical thinking, you know, as those decisions are being made?
  102. Well, I guess for either one of you, that brings up— not that this is specifically in your research, but Gideon, it
  103. brings up kind of the regulatory side of this, of just how much of a role regulatory may have in a lot of these developments as
  104. we move forward. We're going to see, at the end of the day, how the software companies or the AI companies will respond to this, how
  105. policymakers will respond to this, how educational institutions will respond to this. I think what we have, but
  106. that is quite nice in this paper, is that we have a very clear method of measuring cognitive surrender. And now we
  107. can try to take into the lab all sorts of interventions and see how we can maybe move it around. And that's going to be quite a
  108. useful research tool. Gentlemen, I appreciate the time. Good luck with the research as you move forward. All the best.
  109. Thanks for having us. - Thanks, Dan. Thank you. Gideon Nave, Associate Professor
  110. of Marketing here at the Wharton School, and Stephen Shaw, postdoctoral researcher here at Wharton.
  111. Thank you for listening to <i>The Ripple Effect</i>. We hope you found this episode
  112. informative and engaging. Don't forget to subscribe and leave us a review so that we can continue to bring you the best insight
  113. from the Wharton School.

Brief Summary

AI's rise may lead to cognitive surrender, impacting critical thinking and decision-making processes.

Key points

  • The Ripple Effect Podcast. A journey through the minds of Wharton faculty exploring AI's impact on decision-making.
  • Cognitive Surrender. Research reveals how people increasingly rely on AI for decision-making, risking critical thinking skills.

Tension

50/ 100Some tension

Related episodes