Why AI Regulation Can’t Wait
Ask about Knowledge at Wharton
Episode Description
ABOUT THE EPISODE AI safety concerns are no longer just hypothetical. As AI systems become increasingly capable, the risks involving cybersecurity, biosecurity and unintended behavior are becoming harder to ignore.
Kevin Werbach, Professor of Legal Studies and Business Ethics at the Wharton School, explains the different categories of AI risk and why companies alone may not be able to manage them.
Werbach argues that governments need to establish standards and safeguards for frontier AI, drawing comparisons to the way financial institutions are regulated to protect against systemic risks.
ABOUT THE PODCAST This Week in Business features interviews with Wharton faculty about the latest news, fascinating trends, and issues impacting both consumers and the business world. Episodes are recorded at the Wharton School and published twice per week.
----- The Wharton School of the University of Pennsylvania is committed to sharing its intellectual capital through Knowledge at Wharton, the school’s online business analysis journal.
Launched in 1999, Knowledge at Wharton offers free access to:
- Articles, podcasts, and videos highlighting Wharton faculty research and analysis of current business trends - Interviews with book authors from Wharton School Press - In-depth series of curated content like Wharton Executive Education’s Nano Tools for Leaders - A searchable database of more than 10,000 articles covering all aspects of business - A weekly newsletter that delivers Knowledge at Wharton insights directly to your inbox
Visit the Knowledge at Wharton homepage: https://whr.tn/3qUpSBE Subscribe to the Knowledge at Wharton YouTube channel: https://whr.tn/3DEAgAU
Read the full transcriptHide the full transcript
- Recently, much of the discussion around artificial intelligence has been about the potential concern of what
- it could do to our society. It's brought up questions about whether or not the speed at which these technologies are being
- developed need to slow down. Kevin Werbach is professor of legal studies and business ethics here at the Wharton School.
- Kevin, obviously it is drawing a lot of attention right now. Where do you stand on this?
- Well, the AI companies, the major frontier labs like Anthropic and OpenAI have been saying for
- some time that they were going to get to a point where there were serious safety
- risks of the technologies that they were developing. And now we're at that point.
- So absolutely, these are serious concerns, but we need to break down what exactly we're talking
- about. There's really a variety of different kinds of risks here that people are lumping together.
- So what was that concern they originally had, even going back some time, that they were
- going to reach this point? I think about it as really two different concerns which are important to tease apart.
- One is the so-called existential risk of AI. And many of the people in leadership at
- all of the frontier AI companies in the United States come out of a community that
- has been arguing for 10 to 20 years, way before the release of Chad GPT and the maturation of generative AI, that at some
- point AI would be so powerful they would risk destroying all of humanity. Now, of course, these are also the people
- building it. And they were in this dynamic of saying, it's super dangerous, but we have to move
- fast to build it because you can't trust the other guy. That's still an argument that gets made.
- And I think about that as existential risk. And that's, you know, when you have people
- talking about the P doom, the probability that, you know, all life on earth would cease
- because of AI. I don't believe that that is a material risk, although it's certainly worth thinking about.
- And certainly that community has been very accurate in predicting over an extended period of time,
- the improvement of AI. Then there's a related set of concerns, which I think deserve to be taken much more
- seriously in the near term, which is severe or catastrophic risk. So, for example, the models we have today
- are incredibly good at cyber capabilities and hacking. And they are incredibly good at being able
- to do things like create new molecules, which might be used, for example, in a bioterrorism
- attack. And there are all kinds of realistic scenarios that one can think of where the capabilities
- we have now or nearly in the future could lead to very significant harms. I'm not saying we're definitely there, but we've
- seen, you know, at least with the cyber attacks, it actually happening. And I think that's the category where we
- can talk realistically about AI safety today. So, then that, I guess, puts us in
- the realm of the focus of who really has to look at the concerns here. Obviously, the companies themselves being one, but the
- regulatory side being the other part of this, correct? Absolutely. The companies all understand that these are serious
- risks and they have many very good people working on AI safety, but it's imperfect.
- We are now finding that it was not just a handful of autonomous agents that were
- in testing that engaged in cyber attacks of third-party sites, but literally tens of thousands
- of incidents where these agents did things that were not intended. Now, none of those were causing any serious
- harm, but it's beyond the point where the companies themselves really can be confident that they
- know what to do. And, again, there's this game theoretic problem that, you know, there's a tragedy of the commons.
- They all want to slow down, but they can't slow down if their competitors aren't.
- We need government to be involved here. We need government to do what government is
- good at doing, which is setting standards, which is defining as a matter of public policy,
- where do we make the trade-offs, building up the capability, and creating structures for the
- industry. And, unfortunately, we've had too much of government in recent years saying, we want to get
- out of the way of AI for innovation, which is a legitimate impetus, but using that
- as an excuse to really not try to put into place the safeguards that we need. So, when the regulatory side of this is
- put into place, what do you think are some of the components that we will end up seeing?
- And this is kind of part of a discussion you and I have had for more than a decade.
- Can we put a global component together? Because this is going to affect not just
- more than the United States, this is going to affect countries all around the world.
- When we talk about AI regulation, so I talked about the existential risk and then the
- severe or catastrophic risk. There's a third category of risk, which is something that actually a lot of AI regulations
- have been focused on, which I think of as more mundane or ordinary risk. We know that thousands and thousands of companies,
- virtually every company is deploying AI. There are risks that the AI makes errors.
- There are risks of bias. There are risks of privacy harms and so forth. There are risks of psychological effects.
- These are also serious issues, but they can be dealt with by what we can think of as normal regulatory structures.
- We could get into the details, but that's something where, again, it's important to say we
- need regulatory agencies, we need enforcement of existing laws against AI, but that's not the problem
- we're talking about here. We're talking about here at the frontier safety level.
- For that, we need to look to the way that we have addressed potentially catastrophic risks
- in other areas. Some of those are examples like nuclear safety, but actually I think that the one that
- is most apropos here is how we regulate banks. Now, banks are not going to create something
- that's going to destroy all of humanity, but a bank run on one relatively small bank
- can destabilize confidence in the financial system and can literally crash the entire U.S. and
- global economy. We've seen it happen many times. So, we've had for decades to build a
- structure to address bank safety. It's called the system of bank supervision, and
- I think we can look to that model as well as other models from financial regulation
- to build structures for AI. Second part of what you asked is globally, and it's true.
- There's no world government. There's no way that we will have a completely enforceable global structure, but that being said,
- if we're talking about frontier AI, there's only two players in the game. Every single frontier model is either in the
- United States or China, and China is not the U.S. China is in many ways an adversary to the U.S. China is
- also terrified of the risks of these technologies. China, if anything, is more concerned about instability,
- is more concerned about private technology that's out of the control of the government.
- So, it is in their own self-interest to put into place mechanisms to address these
- kinds of issues. It's not going to be perfect, but I think if we have legitimate, thoughtful people in
- the United States government, which there's some in this administration, but there are other voices saying,
- oh, we just need to move forward as fast as we can, I think there is actually room for coordination with China here.
- So, going back to what you were saying a moment ago about the banking regulation, and
- we've obviously seen, as you alluded to, so many examples over the course of time, and
- we have learned from that, and we have adjusted from that, and so it makes me feel like, especially with something like hugging phase
- occurring, we are in the early days of truly understanding what happens and how we address
- the issues moving forward. That's true, but that's always true. And when you drill down, the hugging phase
- incident, where an open AI model and testing went out and hacked hugging phase, that's a
- cybersecurity incident. And we've been dealing with cybersecurity for really two decades, especially the last decade on the
- internet. There are people and organizations with expertise, both in how you secure what you're testing and
- how you secure and harden other sites. And I'm not saying that open AI does not have a serious cybersecurity program, but I
- think we need to make sure to really bring in those who understand that issue. And then, you know, there are other issues
- as well. Again, if we're talking about biosecurity, which is very scary, although, you know, so far we
- have not seen anything that is anywhere near a real threat, but there are lots of
- people with expertise on biosecurity and so forth. So, really, you know, what we need to
- do is figure out, first of all, you know, how to enable and encourage the companies
- to act on their own, because they do take this seriously, how to get them to coordinate.
- The companies say they need an antitrust exemption. I'm not sure if that's the case, but
- we need to create an environment where they can actually work together. And then we need some leadership.
- We need government to say, you know, this is a red line, and, you know, we need to ensure that these protections are in
- place and ensure that they get companies to comply. Soterios Johnson Kevin, great to talk to you.
- Great insight. Thank you, sir. Absolutely. Thank you, Dan. You got it. Kevin Werbach, who is a professor of legal
- studies and business ethics here at the Wharton School.
Brief Summary
Kevin Werbach discusses AI safety risks, regulatory needs, and global cooperation to manage potential harms.
Key points
- Existential Risks of AI. Kevin Werbach explains the potential existential risks posed by AI technologies.
- Need for Regulatory Involvement. The discussion emphasizes the necessity of government regulation in AI safety.
Episode highlights
- Existential Risks of AIKevin Werbach explains the potential existential risks posed by AI technologies.“AI could risk destroying all of humanity.”1:34Oct 7, 2026
- Need for Regulatory InvolvementThe discussion emphasizes the necessity of government regulation in AI safety.“We need government to be involved here.”4:15Oct 7, 2026





