AI in Finance: How to Approach Financial Regulation

Knowledge at Wharton

Ask about Knowledge at Wharton

Episode Description

"The Regulatory Challenges of AI in Finance": Episode 3 of 4 in our "Future of Finance" series, hosted by Wharton School finance professor Itay Goldstein

Artificial intelligence is transforming the world of investing. Financial leaders are looking to AI technology to identify and create new sources of value – but at what risk?

In this episode, host Itay Goldstein discusses the benefits of AI in the investment industry – and the ways AI might lead to unintentional collusion. Goldstein is joined by Tobias Adrian, Financial Counsellor and Director of the Monetary and Capital Markets Department of the International Monetary Fund).

----- The Wharton School of the University of Pennsylvania is committed to sharing its intellectual capital through Knowledge at Wharton, the school’s online business analysis journal.

Launched in 1999, Knowledge at Wharton offers free access to:

- Articles, podcasts, and videos highlighting Wharton faculty research and analysis of current business trends - Interviews with book authors from Wharton School Press - In-depth series of curated content like Wharton Executive Education’s Nano Tools for Leaders - A searchable database of more than 10,000 articles covering all aspects of business - A weekly newsletter that delivers Knowledge at Wharton insights directly to your inbox

Visit the Knowledge at Wharton homepage: https://whr.tn/3qUpSBE Subscribe to the Knowledge at Wharton YouTube channel: https://whr.tn/3DEAgAU 📂 Finance & Accounting;Fintech;AI 🏷️ Future of Finance;PTW Related Articles

Read the full transcriptHide the full transcript
  1. Itay Goldstein: Hello. This is <i>The Future of Finance</i> miniseries here at the Wharton School of the University of Pennsylvania.
  2. I am Itay Goldstein. I'm a Finance Professor and the Chair of the Finance Department. The topic that we are going to dive
  3. into today is AI in Finance. AI stands for artificial intelligence, and AI is promising to change the way that the
  4. financial industry works in many different ways. There are many opportunities, many risks, and there are regulatory concerns,
  5. and we're going to talk about all this with Tobias Adrian. Tobias is the financial counselor and the Director of
  6. the Monetary and Capital Markets Department at the International Monetary Fund, IMF. Hello,Tobias.
  7. Hi, Itay. - So it's great to have you, and I look forward to hearing your insights. So your division at the IMF has written a report
  8. about the use of AI in finance. Can you tell us a little more about what is at stake? What are the opportunities? What are the
  9. risks? What issues do you see? Yeah, thanks so much for having me with you today, Itay. So we
  10. are— we have a report, which is called the Global Financial Stability Report, that has a special focus on artificial
  11. intelligence, or AI in finance. So there are different modes of AI, and some AI techniques have been deployed in finance for
  12. many years. In fact, many decades. And have really reshaped the way that finance worked already. And here I'm
  13. particularly thinking about algorithmic trading, which is primarily based on machine learning techniques. So when
  14. we're looking at liquid capital markets today, we already see a tremendous impact of artificial intelligence on how those
  15. markets are working. So you know, trading activity in— in places such as equity markets in in the US and other advanced
  16. economies, or Treasury markets, the most liquid securities in those markets are already largely traded in algorithmic
  17. fashion based on artificial intelligence. And, you know, really trading at very high frequencies. What is new in
  18. recent years is, of course, the arrival of generative artificial intelligence. And that is, you know, quite different from the
  19. machine learning that has been previously deployed. And so the focus on— of our report is really on understanding to what extent
  20. generative AI, including the large language models that many of the listeners today will have used already— to what extent is
  21. this going to impact the financial industry? And our particular focus is on trading. So, on capital markets activity.
  22. And so I'm happy to go deeper into that. There are also other areas of finance that are already being impacted. And also
  23. happy to talk about that a little bit. Yeah. So you know, one of the things that we think about when
  24. we think about AI is the agency. That now the AI will basically be a decision maker and will start doing things on their own.
  25. I wonder if some of this is also reflected in the way you're thinking about it, and in your report.
  26. Yeah, absolutely. So it is certainly true that in algorithmic trading, you do see a lot of automated decision
  27. making at very high frequencies. So there could be, you know, a very large number— you know, thousands of millions of decisions
  28. that are— that are fully automated. And, you know, to what extent that will be changed or influenced by generative AI is
  29. still work in progress. So let me dig a little bit deeper about, you know, what generative AI is, and how it is being used
  30. already, and how it may be used going forward. So, generative AI really is using very large models that are calibrated to
  31. very, very large data in order to elicit answers that are somewhat similar to the way that human intelligence works, which
  32. is why the term "artificial intelligence" is being used. And in the large language models, the interaction with those
  33. models is via natural language. So the logic is quite different from programming traditionally, where traditionally, you write
  34. code, then the computer is doing something, and it's generating output. Here you have— so, like a model that has been calibrated,
  35. and then you're working with using the information from that model in order to generate output. And I think the
  36. challenge here is twofold. So number one, we don't fully understand how these models really work. They are extremely
  37. powerful, but they're a little bit of a blank box at the moment. So a lot of research today is actually discovering
  38. what kind of knowledge is embedded in the generative AI models. And so we are still learning about the technology
  39. that is very powerful, but it's still very recent. As a result, the financial sector is still in the process of understanding how
  40. those models can be used across the financial sector and for trading activity in particular. And so one policy concern is the
  41. lack of explainability and the unpredictability, to some degree, right? Because we know they are so, like— you know, highly
  42. complex information is used in a way that may be working similar to the human brain. But how it performs in terms of trading,
  43. for example, when new situations are feeding into— into financial market activity, we don't fully understand. And so there's a
  44. certain amount of unpredictability and lack of explainability. And, you know, what the impact is, is— is work
  45. in progress. So our sense is that there are tremendous opportunities. So we already see some areas where we think the
  46. new technology has a first order impact. So for example, when complex reports are being published. Say, an SEC filing of
  47. a publicly traded company, right? These can be very thick, you know, 100 pages long reports. And a generative AI model, you
  48. know, can very quickly extract information from the report. Another example is statements by monetary policy makers like the
  49. Federal Open Market Committee of the Federal Reserve. When they publish their policy decision that comes alongside the
  50. statement, and the statement now can be analyzed very quickly by generative AI. And so the informational efficiency of
  51. markets can be increased, in principle. And I think we see some evidence that that may be the case. So it could be very
  52. good for market efficiency. But then there are other areas where we sort of don't know exactly how much efficiency would be
  53. improved and what potential pitfalls could be. So there are efficiency gains, but there are also new risks that we have to
  54. take into account. So how widely used is it at this point? So our understanding is that— so, like, every financial firm is
  55. exploring it. And there are many startups that are being built around the technology. And in some areas, we do see traction.
  56. So for example, what we are hearing is that to do compliance checks, right, or credit risk analysis— so, you know, analytics,
  57. where, you know, complex data, some of it from the public domain, like from the internet, some of it from reports such as
  58. credit reports, have to be combined, and activity has to be detected. So for example, for payments activity, right? You
  59. know, financial institutions put a lot of effort into understanding whether a certain payment may be legitimate or
  60. illegitimate. We understand that in those areas, newer AI models can be very effective. And detection rate for fraudulent
  61. activity can— can increase tremendously. And our understanding is that there is already a deployment that is
  62. very beneficial. On trading— so, on the capital market activity, we think that is more of an emerging area. And it is not
  63. clear to us that it's very widely used at the moment, say, for high frequency trading. So what we have heard is that, you
  64. know, to venture into new potential trading activities, generative AI can be very good because, you know, it can
  65. generate, say, new trading strategies very quickly, right? So it understands— so, like, the trading strategies that are out
  66. there, or the potential trading strategies that are out there, and it can give you trading ideas, and then you can work
  67. with those ideas, right? It can literally generate code or— but— you know, to what extent that is then actually being used, we
  68. don't fully, fully know. So— so what we're hearing from market intelligence is that it's helpful, but how it's being used
  69. is still somewhat work in progress. So, you know, when you think about the financial firm— so when you think about a risk
  70. manager in a financial firm, you would certainly have some sort of concerns about accountability and decision-
  71. making accuracy, right? Because in traditional automated trading, you can go through the code, right? And you understand
  72. how the code works. And in a generative AI, right, I mean, it's not— you know, you can't trace back the code as to what
  73. decisions are being taken. Now, if the code— if the model generates code, then that you can check. So it is— it is a bit
  74. of a complex undertaking. So this lack of explainability, the magnitude of unpredictability and this lack of— of
  75. accountability and decision making are certainly issues. Let me— let me flag two more things. So you know, there is some
  76. literature that— that is documenting some bias that's not necessarily related to financial activity. But we don't know
  77. whether— so, like, the time series properties of what is coming out of the model has good econometric properties from a trading
  78. perspective, right? So when you think about trading strategies or capital market, market making, right, you really want to make
  79. sure that, you know, your— your strategy is resilient relative to extreme events in the future, relative to structural changes.
  80. And we don't fully understand to what extent— so, like, the length of the data to which the model is calibrated, the kind of data,
  81. you know, to what extent that is so, like— a good predictor for— for future performance. And, you know another— another issue for
  82. financial firms that we've heard is that they— you know, the technology firms ultimately develop and operate the large
  83. models. And the financial firms are users of the models. But it's— it's— it's not— you know, and then they can possibly use
  84. variants of the model or calibrate the model to specific data. But there's a certain amount of reliance on these
  85. outside parties that are developing models that are not specifically generated for the financial industry. So the
  86. reliance on third-party service providers that may not have the same challenges in mind when generating the models as a financial
  87. firm or trading firm would have— you know, that generates a whole new set of issues in terms of third-party dependency that the
  88. financial industry is trying to tackle. So basically, what you're saying is there is potential here for
  89. better information processing that is going to contribute to market efficiency, and this is something that we all want. On
  90. the negative side, on the risk side, I hear you saying mostly that it's hard to explain what's going on. It's kind of a black
  91. box. It is delegated to other parties, and we don't know exactly what they're going to do with it. So it's mostly, I would
  92. say, kind of the risk of the unknown that— things that we can't fully think about right now might happen because of
  93. deploying these new machines and potentially new agents in the trading process. Exactly. So it is a bit of a black box.
  94. So, you know, we may start to use it, but we may not fully understand where results are
  95. coming from and how they're going to behave in the future. So the risk management around using generative AI is certainly
  96. a key priority. I would add another aspect, which is the malicious use of— of AI. You know, it is not easily
  97. verifiable for use of AI, what kind of information has been fed into the model, and what is the relative importance of— so, like,
  98. good data and malicious data. So it is, in principle, possible that the models were calibrated to malicious data where, you
  99. know, some actors may want to manipulate the outcome of— of the— of the model. And, you know, it's difficult to sort of assess
  100. to what extent your model is robust to such forms of manipulation. I think that is another area of concern, and I
  101. think the financial industry is working on that, but it's certainly not an easy challenge to overcome due to the
  102. complexity and the magnitude of the models. You know, another issue, of course, is cyber vulnerability, right? So we all
  103. know that operational risk through cyber attacks is a first order concern for the financial sector. And, you know, in fact,
  104. regulators are asking financial services firms to hold capital and liquidity against cyber incidents. And you know, IT
  105. departments in financial sector firms put a huge amount of resources into protecting firms against cyber attacks. Now, I
  106. already gave an example where generative AI is used to detect, say, malicious or criminal activity in, say, financial
  107. flows. The challenge is that, of course, generative AI can also be used for criminal purposes, right? So you may use it for
  108. better defense, but you can also— it can also be used for more malicious and more effective attacks. So you know, we worry
  109. that cyber incidents may become more effective. And let me give you an example of that. You and I and the listeners are probably
  110. receiving all kinds of emails that are trying to phish our— our confidential information. Are trying to gain access to our
  111. computer systems. And you know, in the past, oftentimes you can sort of fairly easily detect these phishing attacks, say
  112. through emails or social media. There are oftentimes spelling mistakes or things don't look quite right. But generative AI
  113. could be very good at actually fixing those issues, and so make those phishing emails or text messages, or in whatever other
  114. way you get information, you know, seem— seem more real. And, you know, in cyber— in cyber attacks, it's typically the
  115. interface between technology and the human element that is the vulnerability. The cyber criminals tend to exploit the
  116. weak— the human weaknesses, in order to gain access to the technology. And so that could be much more effective. So on the
  117. one hand, as a sort of, you know, risk manager or IT department in the financial firm, you know, you have better
  118. tools to protect yourself. But probably the attacks are also more effective. So, you know that's another— another first
  119. order concern that we hear from the industry. Yeah. So when you're thinking about regulators, they are in a pretty
  120. tough position right now, because, on the one hand, opportunity is coming from AI that we all recognize. On the
  121. other hand, if you think about the risks, the risks are pretty big, but they're not easy to define. They're not easy to
  122. quantify. A lot of it is just that, you know, it's hard to explain. It's a black box. We kind of don't know exactly what
  123. to— what to expect. So what do you think regulators should do in light of all this?
  124. Yeah, so I think the first order work that regulators have to do is just to understand. You know, do outreach to firms. You know,
  125. meet with participants, to understand what is happening. And our chapter is trying to do some of that. We did a lot of
  126. market outreach just to understand where financial sector firms are using generative AI, what they are
  127. exploring, and what they see as risks and challenges, right? And you know, in many ways, we're at the junction time where— so, like,
  128. the perspective of the policy maker and the perspective of the financial firms is not that different, right? In—
  129. traditionally, in regulation, you often have sort of a big conflict in between the policy objective and the individual
  130. firm perspective, because the policy objectives are not necessarily closely aligned with the individual firm perspective.
  131. But in this area, I think there's actually a fairly close alignment so that conversations are fairly straightforward. Now,
  132. having said that, a lot of it is work in progress, right? I mean, the technology, we are getting updates that are first order
  133. improvements. And, you know, the technology is evolving very quickly, and the applications are evolving very quickly. So it
  134. is— it is still working progress. So, you know, for example, in the US, the Securities and Exchange
  135. Commission proposed a rule in July 2023, last year, on the use of predictive analytics by broker dealers and investment
  136. advisors, which would require regulated entities to take steps to address conflicts of interest in the use of predictive data.
  137. And, you know, this is still in the— in the rule making progress. But that would be one example where there's a— there's
  138. sort of like a regulatory step by a security market regulator that— that has been undertaken. You know, we're working closely with the
  139. Financial Stability Board and the International Organization of Security Commissions, as well as the Bank for International
  140. Settlement, in order to, you know, continue to— to assess, you know, the developments.
  141. So is there room for more coordination across regulators, across different countries, given that I think the nature
  142. of technology is that the cross border spillovers are very significant? So, is that
  143. something that you're trying to do here? Yeah. So I would say that for financial regulators, there's a
  144. very well-developed mode of cooperation across countries and across different types of regulatory bodies. So for
  145. example, the Basel Committee, which works on regulating banks and the security market regulators— you know, which is
  146. this IASCO, they are both members of the Financial Stability Board. So the Financial Stability Board is
  147. bringing together all the different regulatory bodies, including also insurance regulators, for example, in
  148. order to have sort of like an umbrella of policy framework. So— so that is very well developed I think in this area of— of
  149. generative AI and large language models, there are other regulatory issues that don't fall within the financial sector
  150. powers and mandates, right? So, for example, the use of data, right? So as a financial sector firm, you may use a model that
  151. has been calibrated to data where there could be, potentially, governance problems around the data in certain jurisdictions,
  152. right? So, you know, in some jurisdictions, for example, individuals have a right to withdraw their data from being
  153. used in the public domain. Now, what does that mean for the calibration of the model? If there's an individual that was
  154. being used— whose data was being used for the calibration that is then withdrawing that data, right? So these are sort of
  155. complicated model that are falling to some degree outside of the policy framework and the powers of the financial
  156. regulators, but that are highly relevant for the— for the financial regulatory sphere. So it's the interdependence between data
  157. models and the technological infrastructure that is— that is at issue here. So— so I think data, intellectual property, you
  158. know, those are— those are some of the issues. So should we worry that the next financial crisis is going to be
  159. a result of AI, or am I going too far with that? It's— it's a little bit too early to say.
  160. I— I do think we have made first order progress in financial regulation, you know, particularly in the
  161. regulation of banks and insurance companies. But also, you know, important security market participants such as
  162. broker dealers and— and— and funds, open end funds, closed end funds. So I do think we're in a much better place than we
  163. were ten or 15 years ago. And you know, for example, with respect to this operational risk, there has been a lot of progress. You
  164. know, that is true for banks, for funds, but also for infrastructures. I think the challenge is really around those
  165. issues that are not readily captured in the existing regulatory approaches. And I think regulators themselves are
  166. working on upscaling their own technology in order to meet these technological challenges. So we hear quite a bit about, you
  167. know, regulatory upskilling. So for example, SupTech and Reg- Tech. So, where supervisors and regulators are using, themselves,
  168. technologies to detect risks better. So, you know, on the one hand, there could be new risks. On the other hand, policy makers
  169. could be more effective. But let me— let me be a little bit more specific about, you know, some regulatory initiatives that we
  170. have seen already. So for example, in Hong Kong, there is an AI governance framework that the Hong Kong monetary authority
  171. has put out, which is really about the risk management and the reliability of AI systems used in the banking sector.
  172. That's one concrete example. The second one in the UK, the Financial Conduct Authority has implemented requirements for the
  173. explainability of AI-driven decision making. So for example, in credit scoring. So, it's fine to use those models, but you
  174. need to have sort of a rationale or an explainability around that. And that, in turn, helps to mitigate the risks associated
  175. with the opaqueness of the models. In Singapore, there are principles called the Fairness, Ethics, Accountability and
  176. Transparency principles for AI in finance. And that really helps the financial institutions in Singapore to have a
  177. responsible usage of AI systems. And that is particularly focused also on bias and discrimination. And I would finally mention, in
  178. the European Union, there's the General Data Protection Regulation, GDPR, and that has been adapted to the use of AI
  179. practices in financial services, particularly for focusing on the on data protection and privacy practices. So this is some of
  180. the examples I alluded to earlier, aimed at reducing the risk of data breaches and the misuse of personal information.
  181. Okay. All very fascinating. Certainly a lot more to hear about and learn about from this AI revolution in financial
  182. markets and all the challenges it poses when thinking about risks and regulation. Thank you very much, Tobias, for all the
  183. great work that you're doing at the IMF and for joining us for this podcast. Thanks so much, Itay. Very good to see you.
  184. And warm regards from Washington to Pennsylvania. Thank you. And thank you, everyone, for listening.

Brief Summary

Itay Goldstein and Tobias Adrian discuss AI's impact on finance, focusing on opportunities, risks, and regulatory challenges.

Key points

  • AI's Impact on Finance. AI is reshaping finance through algorithmic trading and generative models, but risks remain.
  • The Black Box of AI. Generative AI presents challenges in understanding decision-making processes and accountability.
  • Regulatory Challenges Ahead. Regulators face difficulties in defining and quantifying the risks posed by AI in finance.
  • AI Regulation in Finance. Exploring the need for explainability in AI-driven decision making in finance.
  • Fairness and Ethics in AI. Singapore's principles focus on responsible AI usage to combat bias.
  • GDPR and AI. The adaptation of GDPR to AI practices enhances data protection.

Tension

45/ 100Some tension

Related episodes