Can AI Companies Really Be Trusted to Regulate Themselves?

Knowledge at Wharton

Ask about Knowledge at Wharton

Episode Description

ABOUT THE EPISODE

AI is advancing faster than governments can figure out how to govern it. So who should be responsible for managing the risks?

Kevin Werbach, Wharton Professor and Chair of the Department of Legal Studies and Business Ethics, joins us to explore AI governance, accountability, and regulation—and why the challenge isn't simply creating “ethical AI,” but making sure the people and organizations building it are accountable.

The conversation examines the risks of increasingly powerful AI, the difficulty of creating global rules, whether AI companies can effectively regulate themselves, and why governments may need a more constructive role in overseeing the technology.

As AI becomes more powerful, the question isn't just what can we build? It's who is accountable when something goes wrong?

ABOUT THE PODCAST This Week in Business features interviews with Wharton faculty about the latest news, fascinating trends, and issues impacting both consumers and the business world. Episodes are recorded at the Wharton School and published twice per week.

#AI #ArtificialIntelligence #AIGovernance #AIRegulation #AISafety #AI Ethics #TechPolicy #AIAccountability #AI Risk #FutureOfAI #OpenAI #SamAltman

----- The Wharton School of the University of Pennsylvania is committed to sharing its intellectual capital through Knowledge at Wharton, the school’s online business analysis journal.

Launched in 1999, Knowledge at Wharton offers free access to:

- Articles, podcasts, and videos highlighting Wharton faculty research and analysis of current business trends - Interviews with book authors from Wharton School Press - In-depth series of curated content like Wharton Executive Education’s Nano Tools for Leaders - A searchable database of more than 10,000 articles covering all aspects of business - A weekly newsletter that delivers Knowledge at Wharton insights directly to your inbox

Visit the Knowledge at Wharton homepage: https://whr.tn/3qUpSBE Subscribe to the Knowledge at Wharton YouTube channel: https://whr.tn/3DEAgAU

Read the full transcriptHide the full transcript
  1. Well, as AI continues to grow in our lives, there are still many questions lingering around
  2. the role that governance can play. Wharton Professor Kevin Werbach is the Chair of
  3. the Legal Studies and Business Ethics Department, and he joins us to discuss right now.
  4. Kevin, great to catch up with you. How are you, sir? I'm great. Good to talk to you as well.
  5. So we hear all of these things about the development of AI. It doesn't feel like we hear a lot
  6. about the governance side of this. What's happening in that realm right now? I don't know.
  7. It depends who you're talking to. I hear a lot about the governance of it, and there's actually a great deal of
  8. talk going on in Washington and other capitals and in the areas where people are developing
  9. AI about this. Broadly, what's going on is two sets of conversations, which I'm realizing increasingly have been conflated
  10. in a way that's not helpful. One is a longstanding set of conversations about risks of AI.
  11. What if the AI systems are biased? What about privacy issues? What about people who are potentially addicted or
  12. have mental health responses to AI companion apps? Those are what I think of as the
  13. deployment issues that are widespread and very significant. There's a like anthropic and open AI about
  14. potentially catastrophic risks of those frontier AI models, as well as geopolitical questions between the U
  15. .S. and China. So when you hear the term ethical AI, I think a lot of people would sit
  16. back and say, well, why isn't it ethical just kind of across the board? But you and I have talked about a
  17. variety of different topics in the past, and in many cases, you do find instances where
  18. the ethics do get pushed to the side at times. Absolutely. A couple things. One is it's not the AI being ethical
  19. that in the first instance I think we should focus on. It's the humans being ethical and being responsible
  20. in how they design the AI and how they use the AI systems. I think also it's important to distinguish there's
  21. AI ethics, which has been a field for a very long time, and there's a lot of excellent, very sophisticated work in different issues
  22. about ethical questions around AI. But then there's also what I prefer to call accountable AI.
  23. Sometimes people talk about responsible AI in other terms, but that's really focused on businesses and
  24. other organizations that are developing and deploying these AI systems. What should they do?
  25. And just as with the larger question of how should you be responsible in business, it's
  26. not a simple matter of just saying, well, identify some ethical principles. It's a question of, broadly, how do you
  27. think about effectively operating a business while responding to very legitimate public and ethical and other
  28. kinds of concerns about responsibility. And so that doesn't happen automatically with AI.
  29. It doesn't happen automatically anywhere else in business. So when you talk about, though, the development
  30. of AI, I think a lot of people would also expect that there has to be a global component to it.
  31. And you mentioned that obviously so many countries are focused on that. Is there a recognition or is there a
  32. want, maybe is even a better way to ask it, to kind of keep the development of AI and how we use AI under
  33. one umbrella for all countries? Or is this similar to what we've seen with the use of the internet and GDPR
  34. in Europe? And they have one set of rules and we have one set of rules. How is that, do you think, going to
  35. play out? AI is global. And so, as you point out, we need to think about AI governance globally.
  36. But that being said, there are proposals that keep getting thrown out. Well, what if we got all the governments
  37. of the world together and they enforce a treaty that would very much limit particular uses
  38. of AI? Most of that I don't think is realistic. There are some areas, again, of these catastrophic
  39. risks where just as the U.S. and the Soviet Union during the Cold War collaborated
  40. on nuclear safety, the U.S. and China and other major governments where there is significant
  41. AI development will have reason to, and I think will find common ground on, for example,
  42. how do we ensure that an autonomous AI system will not launch a nuclear weapon? These are broad questions.
  43. But what you're asking about is really a different set of issues, which is AI is
  44. a global phenomenon. It's being deployed by firms all over the world. And no one country can regulate all of
  45. that. That's true. But that's true about lots of things. And every major country is going to have
  46. to think about an AI framework within its own context. Those don't have to be identical.
  47. But there do have to be certain baseline principles. And we need to think about what's an
  48. appropriate regime. So, you know, I've been writing about this question, looking at different models in the U
  49. .S. for what it would look like. I think in the U.S., we need to realize, first of all, there's a lot
  50. of background law that applies. If you discriminate with an AI system, you're discriminating after existing anti-discrimination laws.
  51. And also in the U.S., private liability is a big factor. People, there's thousands of lawsuits already against AI
  52. companies without even having a law passed in the U.S. about AI. So there's much to be said about that.
  53. But ultimately, I think it's a fantasy to think there's going to be some global agreement
  54. that's going to be reached on all these issues. So is there, though, at least the recognition
  55. by the firms that are involved in the development of the AI, the anthropics, the open
  56. AIs of the world, that they have to have this as part of the buildout as they move forward?
  57. And I ask that because we had the recent case with Hugging Face that ended up obviously being basically a backdoor hack.
  58. That was really a seminal moment when open AI announced that one of its pre-deployment
  59. models had autonomously, in testing, gone out and hacked into Hugging Face. Because that's a scenario that people have talked
  60. about for a while, that clearly is a scenario that is scary and problematic. Yeah.
  61. But it was seen as a hypothetical. Now it's real. And AI is only going to get more
  62. powerful. So in that case, open AI did announce what happened. They are doing an investigation.
  63. They are pausing and limiting some of the development of their next generation models.
  64. And so are many of the other AI companies. So to your question, the major frontier AI
  65. labs in the U.S. have from the beginning had a commitment to addressing at least
  66. the catastrophic risks of AI development. But there's always a tension because they're in
  67. a race to develop as fast as they can against each other and against these companies
  68. in China. So no company wants to make something that is going to cause massive harm, even if
  69. their own self-interest, if you don't want to have that. But exactly what it means and what are
  70. the mechanisms in this kind of challenging game theory environment that would effectively limit the risk
  71. of AI, those are the hard questions. And so, for example, just to throw in one thing, my colleague Peter Conte-Brown and
  72. I have written about using the model that we use for regulating banks. Banks are similar in that if there's a
  73. run on one bank, it can cascade and actually cause a global financial collapse. And so we have a whole regime of
  74. bank supervision that provides a flexible expert mechanism for dealing with those issues.
  75. We propose doing something similar for AI. But that's just one proposal. I think companies, their hearts are in the
  76. right place, but that alone is not enough. Well, and also, I think we also have to look at it from the perspective of
  77. the public in general, is also that we think of AI as a tool to be able to help us out, find information, and
  78. be able to support the work that we're doing. I think a lot of people believe that's
  79. the path that is going to be most used by the public as we move forward. The question is, are the companies involved of
  80. the belief that that's the best path, or do they have grander views of what AI should be as we move forward?
  81. And there's not one answer to that. When you look at a company, I might ask, what does Sam Altman think as the
  82. CEO of OpenAI? And even just asking what Sam thinks is not an obvious question. We can say, what did Sam say?
  83. But part of what he says is not necessarily what's in his heart of hearts. He's one person though.
  84. What Sam says in his public statements is not an answer to what does OpenAI, the
  85. institution, think. It's complicated. I will say it again. If we're talking about the major U.S.-based
  86. frontier AI labs, all of them have many people at the very senior levels who I truly believe are very seriously concerned about risks
  87. of AI. And in fact, again, that is the ethos of how many of these companies, including Google,
  88. DeepMind, Anthropic, and OpenAI got founded. But that alone does not mean, okay, there's
  89. not going to be any problems. First of all, there's a lot of debate about what risks we should focus on.
  90. And there's this question of, if you do think that a super intelligent AI could cause
  91. catastrophic harm, is the right strategy for you to say, let's not build it until we're
  92. really, really confident because someone else is going to build it. And we don't trust them as much as
  93. we trust us. That's the game theory tension that has been there all along with these companies.
  94. So yes, I do believe, and I've engaged with many people in these organizations, they do
  95. appreciate that these are technologies that have many downside risks, and things need to be done
  96. to address that. But that alone is not enough. And this is the classic messy political process.
  97. And it's the process where, unfortunately, the U .S. government has not been terribly helpful because
  98. it's careened from saying, just let them do whatever they want, because we need that renovation
  99. to, oh, my God, we're scared about cyber risks. And so we need to shut down some
  100. of these models under some system where we're not even going to disclose what the rules
  101. are. We need adults in the room from governments that are helping this process with industry.
  102. Kevin, always great to catch up with you. Thanks for your time and your insight.
  103. All the best. Thanks very much, Dan. Kevin Werbach, who is a professor in the Department of Legal Studies and Business Ethics.
  104. He's also chair of that department here at the Wharton School.

Brief Summary

Kevin Werbach discusses AI governance, ethical responsibilities, and the challenges of global regulation in AI development.

Key points

  • Ethics vs. Accountability in AI. The conversation shifts to the distinction between AI ethics and accountable AI practices.
  • The Role of Governance in AI. Kevin Werbach discusses the critical need for governance as AI technology evolves.
  • Real Risks of AI Development. A recent incident with Hugging Face highlights the real dangers of AI systems.

Related episodes