Why AI Regulation Can’t Wait

Knowledge at Wharton

Ask about Knowledge at Wharton

Episode Description

ABOUT THE EPISODE AI safety concerns are no longer just hypothetical. As AI systems become increasingly capable, the risks involving cybersecurity, biosecurity and unintended behavior are becoming harder to ignore.

Kevin Werbach, Professor of Legal Studies and Business Ethics at the Wharton School, explains the different categories of AI risk and why companies alone may not be able to manage them.

Werbach argues that governments need to establish standards and safeguards for frontier AI, drawing comparisons to the way financial institutions are regulated to protect against systemic risks.

ABOUT THE PODCAST This Week in Business features interviews with Wharton faculty about the latest news, fascinating trends, and issues impacting both consumers and the business world. Episodes are recorded at the Wharton School and published twice per week.

----- The Wharton School of the University of Pennsylvania is committed to sharing its intellectual capital through Knowledge at Wharton, the school’s online business analysis journal.

Launched in 1999, Knowledge at Wharton offers free access to:

- Articles, podcasts, and videos highlighting Wharton faculty research and analysis of current business trends - Interviews with book authors from Wharton School Press - In-depth series of curated content like Wharton Executive Education’s Nano Tools for Leaders - A searchable database of more than 10,000 articles covering all aspects of business - A weekly newsletter that delivers Knowledge at Wharton insights directly to your inbox

Visit the Knowledge at Wharton homepage: https://whr.tn/3qUpSBE Subscribe to the Knowledge at Wharton YouTube channel: https://whr.tn/3DEAgAU

Read the full transcriptHide the full transcript
  1. Recently, much of the discussion around artificial intelligence has been about the potential concern of what
  2. it could do to our society. It's brought up questions about whether or not the speed at which these technologies are being
  3. developed need to slow down. Kevin Werbach is professor of legal studies and business ethics here at the Wharton School.
  4. Kevin, obviously it is drawing a lot of attention right now. Where do you stand on this?
  5. Well, the AI companies, the major frontier labs like Anthropic and OpenAI have been saying for
  6. some time that they were going to get to a point where there were serious safety
  7. risks of the technologies that they were developing. And now we're at that point.
  8. So absolutely, these are serious concerns, but we need to break down what exactly we're talking
  9. about. There's really a variety of different kinds of risks here that people are lumping together.
  10. So what was that concern they originally had, even going back some time, that they were
  11. going to reach this point? I think about it as really two different concerns which are important to tease apart.
  12. One is the so-called existential risk of AI. And many of the people in leadership at
  13. all of the frontier AI companies in the United States come out of a community that
  14. has been arguing for 10 to 20 years, way before the release of Chad GPT and the maturation of generative AI, that at some
  15. point AI would be so powerful they would risk destroying all of humanity. Now, of course, these are also the people
  16. building it. And they were in this dynamic of saying, it's super dangerous, but we have to move
  17. fast to build it because you can't trust the other guy. That's still an argument that gets made.
  18. And I think about that as existential risk. And that's, you know, when you have people
  19. talking about the P doom, the probability that, you know, all life on earth would cease
  20. because of AI. I don't believe that that is a material risk, although it's certainly worth thinking about.
  21. And certainly that community has been very accurate in predicting over an extended period of time,
  22. the improvement of AI. Then there's a related set of concerns, which I think deserve to be taken much more
  23. seriously in the near term, which is severe or catastrophic risk. So, for example, the models we have today
  24. are incredibly good at cyber capabilities and hacking. And they are incredibly good at being able
  25. to do things like create new molecules, which might be used, for example, in a bioterrorism
  26. attack. And there are all kinds of realistic scenarios that one can think of where the capabilities
  27. we have now or nearly in the future could lead to very significant harms. I'm not saying we're definitely there, but we've
  28. seen, you know, at least with the cyber attacks, it actually happening. And I think that's the category where we
  29. can talk realistically about AI safety today. So, then that, I guess, puts us in
  30. the realm of the focus of who really has to look at the concerns here. Obviously, the companies themselves being one, but the
  31. regulatory side being the other part of this, correct? Absolutely. The companies all understand that these are serious
  32. risks and they have many very good people working on AI safety, but it's imperfect.
  33. We are now finding that it was not just a handful of autonomous agents that were
  34. in testing that engaged in cyber attacks of third-party sites, but literally tens of thousands
  35. of incidents where these agents did things that were not intended. Now, none of those were causing any serious
  36. harm, but it's beyond the point where the companies themselves really can be confident that they
  37. know what to do. And, again, there's this game theoretic problem that, you know, there's a tragedy of the commons.
  38. They all want to slow down, but they can't slow down if their competitors aren't.
  39. We need government to be involved here. We need government to do what government is
  40. good at doing, which is setting standards, which is defining as a matter of public policy,
  41. where do we make the trade-offs, building up the capability, and creating structures for the
  42. industry. And, unfortunately, we've had too much of government in recent years saying, we want to get
  43. out of the way of AI for innovation, which is a legitimate impetus, but using that
  44. as an excuse to really not try to put into place the safeguards that we need. So, when the regulatory side of this is
  45. put into place, what do you think are some of the components that we will end up seeing?
  46. And this is kind of part of a discussion you and I have had for more than a decade.
  47. Can we put a global component together? Because this is going to affect not just
  48. more than the United States, this is going to affect countries all around the world.
  49. When we talk about AI regulation, so I talked about the existential risk and then the
  50. severe or catastrophic risk. There's a third category of risk, which is something that actually a lot of AI regulations
  51. have been focused on, which I think of as more mundane or ordinary risk. We know that thousands and thousands of companies,
  52. virtually every company is deploying AI. There are risks that the AI makes errors.
  53. There are risks of bias. There are risks of privacy harms and so forth. There are risks of psychological effects.
  54. These are also serious issues, but they can be dealt with by what we can think of as normal regulatory structures.
  55. We could get into the details, but that's something where, again, it's important to say we
  56. need regulatory agencies, we need enforcement of existing laws against AI, but that's not the problem
  57. we're talking about here. We're talking about here at the frontier safety level.
  58. For that, we need to look to the way that we have addressed potentially catastrophic risks
  59. in other areas. Some of those are examples like nuclear safety, but actually I think that the one that
  60. is most apropos here is how we regulate banks. Now, banks are not going to create something
  61. that's going to destroy all of humanity, but a bank run on one relatively small bank
  62. can destabilize confidence in the financial system and can literally crash the entire U.S. and
  63. global economy. We've seen it happen many times. So, we've had for decades to build a
  64. structure to address bank safety. It's called the system of bank supervision, and
  65. I think we can look to that model as well as other models from financial regulation
  66. to build structures for AI. Second part of what you asked is globally, and it's true.
  67. There's no world government. There's no way that we will have a completely enforceable global structure, but that being said,
  68. if we're talking about frontier AI, there's only two players in the game. Every single frontier model is either in the
  69. United States or China, and China is not the U.S. China is in many ways an adversary to the U.S. China is
  70. also terrified of the risks of these technologies. China, if anything, is more concerned about instability,
  71. is more concerned about private technology that's out of the control of the government.
  72. So, it is in their own self-interest to put into place mechanisms to address these
  73. kinds of issues. It's not going to be perfect, but I think if we have legitimate, thoughtful people in
  74. the United States government, which there's some in this administration, but there are other voices saying,
  75. oh, we just need to move forward as fast as we can, I think there is actually room for coordination with China here.
  76. So, going back to what you were saying a moment ago about the banking regulation, and
  77. we've obviously seen, as you alluded to, so many examples over the course of time, and
  78. we have learned from that, and we have adjusted from that, and so it makes me feel like, especially with something like hugging phase
  79. occurring, we are in the early days of truly understanding what happens and how we address
  80. the issues moving forward. That's true, but that's always true. And when you drill down, the hugging phase
  81. incident, where an open AI model and testing went out and hacked hugging phase, that's a
  82. cybersecurity incident. And we've been dealing with cybersecurity for really two decades, especially the last decade on the
  83. internet. There are people and organizations with expertise, both in how you secure what you're testing and
  84. how you secure and harden other sites. And I'm not saying that open AI does not have a serious cybersecurity program, but I
  85. think we need to make sure to really bring in those who understand that issue. And then, you know, there are other issues
  86. as well. Again, if we're talking about biosecurity, which is very scary, although, you know, so far we
  87. have not seen anything that is anywhere near a real threat, but there are lots of
  88. people with expertise on biosecurity and so forth. So, really, you know, what we need to
  89. do is figure out, first of all, you know, how to enable and encourage the companies
  90. to act on their own, because they do take this seriously, how to get them to coordinate.
  91. The companies say they need an antitrust exemption. I'm not sure if that's the case, but
  92. we need to create an environment where they can actually work together. And then we need some leadership.
  93. We need government to say, you know, this is a red line, and, you know, we need to ensure that these protections are in
  94. place and ensure that they get companies to comply. Soterios Johnson Kevin, great to talk to you.
  95. Great insight. Thank you, sir. Absolutely. Thank you, Dan. You got it. Kevin Werbach, who is a professor of legal
  96. studies and business ethics here at the Wharton School.

Brief Summary

Kevin Werbach discusses AI safety risks, regulatory needs, and global cooperation to manage potential harms.

Key points

  • Existential Risks of AI. Kevin Werbach explains the potential existential risks posed by AI technologies.
  • Need for Regulatory Involvement. The discussion emphasizes the necessity of government regulation in AI safety.

Related episodes