Diversity Is Critical for the Future of AI — Leading Diversity at Work

Knowledge at Wharton

Ask about Knowledge at Wharton

Episode Description

EPISODE OVERVIEW In this episode of Leading Diversity at Work, Wharton’s Stephanie Creary speaks with Broderick Turner -- a Virginia Tech marketing professor who also runs the school’s Technology, Race, and Prejudice (T.R.A.P.) Lab -- and Karim Ginena -- a social scientist and founder of RAI Audit -- on how to use AI while thinking critically about its flaws.

SERIES OVERVIEW Leading Diversity at Work is a podcast series featuring Wharton management professor Stephanie Creary in conversation with experts in diversity, equity and inclusion. Learn more at http://whr.tn/leadingdiversity.

PODCAST OVERVIEW The Knowledge at Wharton podcast features interviews with Wharton faculty and industry leaders about the business landscape and the impact of their work. Conversations range from the latest research and books by faculty authors, to Leading Diversity at Work with professor Stephanie Creary, and Dollars and Change with professor Katherine Klein.

#AI #artificialintelligence #dei #diversity ----- The Wharton School of the University of Pennsylvania is committed to sharing its intellectual capital through Knowledge at Wharton, the school’s online business analysis journal.

Launched in 1999, Knowledge at Wharton offers free access to:

- Articles, podcasts, and videos highlighting Wharton faculty research and analysis of current business trends - Interviews with book authors from Wharton School Press - In-depth series of curated content like Wharton Executive Education’s Nano Tools for Leaders - A searchable database of more than 10,000 articles covering all aspects of business - A weekly newsletter that delivers Knowledge at Wharton insights directly to your inbox

Visit the Knowledge at Wharton homepage: https://whr.tn/3qUpSBE Subscribe to the Knowledge at Wharton YouTube channel: https://whr.tn/3DEAgAU

Read the full transcriptHide the full transcript
  1. this podcast is brought to you by knowled of [Music] Warton hello my name is Stephanie Cy and
  2. I'm an assistant professor of management at the Wharton School of the University
  3. of Pennsylvania and I'm delighted to welcome you to today's episode of the knowledge at War and leading diversity
  4. at work podcast series which is focus focused on responsible and fair air AI in the workplace and the marketplace
  5. joining me today are two very special guests first we have Dr broer Turner who is an assistant professor of marketing
  6. at Virginia Tech Pamplin School of Business and a visiting fellow at Harvard Business school's Institute for
  7. business and Global Society he also runs the technology race and Prejudice lab or
  8. the Trap lab where he and his fellow Trappers are pushing the boundaries on understanding how race and racism
  9. underly many consumer and managerial decisions his main research area focuses on the intersection of marketing
  10. technology racism and emotion next we have Dr kareim Gan who is the founder of RA AI audit an AI governance and
  11. research consultancy with 16 years of experience in ethics and governance spanning both industry and Academia he
  12. recently served as the founding user experience researcher on meta's responsible AI team leading AI fairness
  13. user research at the company he has helped meta our AI team scale its products across the company and bolster
  14. the responsible adoption of AI Dr ganana holds a PhD in management from the University of Virginia's Darden School
  15. of Business specializing in business ethics and organizational behavior and I just realized we've got a Virginia
  16. connection uh across the two of you here who would have who would have thought that that would happen but in any case
  17. welcome bro and Kareem I am so honored to have you with me today for a conversation on responsible and fair AI
  18. in the workplace and in the marketplace so we're hoping to cover you know a lot of interesting ground here today and as
  19. I as talked to brck and cre about this earlier I like to think of myself as a as a novice on these topics so today I'm
  20. going to represent the average uh consumer um and worker who doesn't have a lot of experience talking about um AI
  21. other than what I read about and hear about um on mainstream media and from my colleagues so so we're hoping today that
  22. um this conversation will be helpful to others of you who are just like me who are trying to figure out how relevant uh
  23. a conversation about responsible and fair AI is to us as workers but also as consumers so let's get started talking
  24. about broadly about responsible and fair Ai and so project I'm I'm going to go to
  25. you first and what I'm hoping you can do is is share a bit about how and why you
  26. became involved in this topic and summarize for us some of your current work related to this topic yeah so I got
  27. involved in this topic area uh honestly more than a decade ago when I was teaching high school math uh at a public
  28. high school in Atlanta Georgia where I taught like 90% uh black students and what I wanted
  29. to do while I was there was figure out ways to make those kids' lives better fairer more Equitable uh and
  30. simultaneously I used to teach linear regression and students asked me Mr turn am I ever going to use this and now I
  31. have an answer the answer is now that uh you know so I've moved into this space where I'm thinking a lot about how race
  32. and racism underly uh Market Systems and Technology being a system that matters and our
  33. research group the technology race and Prejudice lab is doing active research now on what levers can be moved to lead
  34. to uh more equality in systems to lead to better outcomes in technology uh this has extended some into some uh company
  35. advisory work where uh we're trying to get company consider that moving uh communities earlier into the development
  36. process leads to better uh Products that come out out and then on the public knowledge side uh we're writing white
  37. papers on uh how different uh identity groups may be impacted by gender of AI for instance so we just uh finished a
  38. report on for Hispanic herited Munch Hispanic herit Heritage Month thank you on how uh in uh how folks with Hispanic
  39. Origins and background should be thinking about gender of AI how do they fit into this interlan system of data
  40. the classification code okay great thank you so much uh so Kareem same question to you can you share a bit about how and
  41. why you became involved in this topic around responsible and fair Ai and summarize some of your current work on
  42. this topic yeah absolutely firstly thank you so much for inviting me I'm excited
  43. about this dialogue with yourself and brri and uh love the Virginia connection Virginia is well represented here today
  44. um so i' I've been involved in ethics and governance um for quite some time you know prior to
  45. pursuing my PhD so I'm not surprised that I ended up in kind of AI governance uh what I'm a little bit surprised about
  46. is actually post pursuing my PhD going back to uh industry initially when I started off my PhD program I had no
  47. intentions uh to go back to industry but in my last year of my PhD which was my sixth year I had received offers
  48. basically from Academia and Industry and uh I went for back to Industry so um firstly as someone who belongs to a
  49. minority group um I was very passionate and still am about having a front seat to these conversations that are
  50. happening in Tech and uh helping shape the uh the technology using you know the latest and greatest research but also
  51. interacting with product teams to be able to do that policy and legal and so forth um the other reason that I chose
  52. to go back was you know as I was thinking through my impact to be honest uh joining a company
  53. like meta's responsible AI team where the company has over three three billion users I think you know it's very
  54. difficult to kind of argue with the extent of impact that you can do uh when you uh you have three billion users and
  55. any kind of like product changes that you make can impact really a lot of people worldwide and and lastly honestly
  56. uh one of the concerns that I've had about uh Academia was you know being stuck in a small college town somewhere
  57. uh where there's not very much diversity and uh that would not be appropriate for
  58. the upbringing of my kids and so that was a a a very uh strong concern uh not having much of an autonomy about where I
  59. live or where I work um so it drove me back to actually go to um to Industry now as far as my research
  60. uh much of what I do for clients is obviously confidential but on the public side of things I've been testing out uh
  61. some popular AI image generators um and I'm finding a few patterns so uh most of
  62. the images uh that are being generated from uh the uh at least the AI generators uh that I have tested tend to
  63. be of people from the white race uh people of Co color are uh greatly overlooked uh these models tend to
  64. associate uh a professional headshot or a professional dress code or a professional hairstyle with those of uh
  65. white people uh women are underrepresented uh and uh when presented as professionals they're often
  66. limited to gendered occupations such as teachers nurses graphic designers and much less likely to produ to appear as
  67. CEOs for example or medical doctors or lawyers um images also tend to spew on the younger side of things and Senior
  68. managers for example are associated with having gray hair which is a sign of aism um and finally you know these
  69. models tend to portray certain populations in demeaning ways for example uh if if you ask Del Bing to
  70. portray Turks uh it will uh provide you with a picture of a Stern turkey uh a Stern turkey dressed in a turban right
  71. um whereas if you're ask it to produce for example an image of an American or a French person or something like that
  72. will do a much better job right um they might also refuse to produce images of certain populations For No Good Reason
  73. such as if you ask it to produce um an image of a Muslim or a Jew uh while presenting followers of other religions
  74. so definitely this technology is extremely powerful uh I am not a doomsayer per se as it uh relating to
  75. the technology but I believe uh that you know we have to do our prudent and due diligence in order
  76. for us to be able to direct the trajectory of this technology in ways um such that we're maximizing the benefits
  77. while minimizing the harms my my favorite example of uh that gender of AI uh art is when people do prompts for
  78. Jesus in the temple flipping the table that they literally get Jesus doing like gymnastics doing a backflip over a table
  79. uh so yeah text text has no meaning it's it's just funny absolutely so we're gonna
  80. definitely dive into there's a lot of to unpack there and certainly the implications of this I mean I think of
  81. where I sit I can automatically connect those dots but I think as we begin to talk about some of the challenges it'll
  82. be important for people to understand what could possibly go wrong when you you're not represented in how these
  83. models are being uh trained I I think to me that's an obvious answer but I don't
  84. think that's obvious to everybody what happens when you lack representation in in the data amongst the data or you're
  85. misrepresented right who you are and the the the groups that you're part of are misrepresented in in how the model is
  86. being trained so we're going to talk about that in a minute but let me just and probably this is part of the answer
  87. to your question here Kareem um let's let's start with business and employers first right and and I can imagine I have
  88. some sense of this from my own work that businesses and employers um are likely and can be struggling to understand
  89. their conversation not just in a role about AI use but fair and responsible AI use and so obviously without you know
  90. saying anything confidential can you talk generally about how you through your work are helping businesses
  91. employers make sense of their roles and responsibilities on this topic yeah I think it's important to be
  92. clear that um companies are accountable for how their AI systems operate and that responsibility can be offloaded
  93. right it can be considered an externality of some sort they have obligations to their customers and it's
  94. just a matter of time uh before legislation comes into effect in this space you know obviously we have right
  95. now the uh EU AI act which we're waiting it's just a matter of time before enforcement happens and there's a lot of
  96. talk uh about legislation uh Congress and so forth here in the United States um and honestly if come companies want
  97. to build successful products in this AI era era they must invest in responsible AI infrastructure right um doing so
  98. requires a commitment from the board of directors from senior leadership but ALS
  99. ultimately what it does is it pays off uh in terms of earning customer trust and confidence right it it just doesn't
  100. cut it for you to actually have an AI strategy uh without thinking through the the trust layer or the responsible layer
  101. um you know issues of like fairness and privacy and um robustness and and so forth right because if if you're a
  102. product manager for example you're trying to produce the best products no one really wants to deal with a company
  103. whose products are reaching you know their privacy uh you know their their privacy obligations and stuff like that
  104. like if you're using personal data uh of your users to train your models or if your products don't work for certain
  105. segments of the population or exposes your users to specific types of harms you know cutting Corners like that
  106. really does a disservice uh to your stakeholders of course uh but it also exposes you to legal and reputational uh
  107. risk and it's just a bad way of doing business it's just a matter of time before you know uh better companies are
  108. able to gain more market share because they're taking this more um seriously and investing in setting up the
  109. infrastructure yeah you know a lot of what I hear you saying is you know sometimes it's easy to think that ethics
  110. and responsibility questions of responsibility isn't the job of a corporation but you take a strong stance
  111. and certainly through your own dissertation work and your own scholarship is that these are things
  112. that are not nice to haves um there must haves and and you know while it's it shouldn't always come down to this we do
  113. have a system called the legal system which is set in place to help adjudicate these issues when it does feel like it's
  114. a great area even though a gray area excuse me even might be black and white with respect to businesses and their
  115. responsibility to ensure that they're not inducing harm uh through their activities brri I mean certainly I know
  116. that you do work with companies as well I I would also like you to put on your researcher and educator hat um and help
  117. us to understand like the nature of this conversation from a scientific perspective and also in the work that
  118. you're doing you know as a professor as part of your fellowship I'm actually thinking a lot about chat GPT these days
  119. certainly because as an educator this is sort of the one this is probably the most I know about this conversation
  120. around Should students be able to use GPT to complete their assignments and it seems to me like it's such a polarizing
  121. sets of issues um so I am just just curious as you think about the the places where you set project um how
  122. you're thinking about the issue around businesses and employers and Ed education's role in this conversation so
  123. I think a couple of thoughts right so first uh to piggy back on Kareem when I talk to both students and businesses I
  124. think of myself as an educator no matter where I am right and I have the same thing I tell them uh which is if you get
  125. this wrong right you don't include uh a wide SWA of human beings in the creation
  126. of your technology products when you fail because it's not an if but when you fail you lose money
  127. because you spent all this money on development without considering uh the human beings at the end and when the
  128. product gets released the product fails again because people don't use it right so as a really simple example uh those
  129. automatic uh faucets where you put put your hand under it did not include uh like darker melanated people
  130. in the data set in the training set for when they were testing out this automatic hand washer and so while it
  131. sailed sold fine in North America when it went further out into uh places closer to the Equator where people were
  132. Browner regardless of uh ethnic origin they didn't sell any because the product did not work they would install it and
  133. people put their hand under the thing and I know if you're a black or a brown person you have done this at the airport
  134. bathroom been upset right but if your country is majority melanated then no one bought them and so have they
  135. included those folks earlier in the development process like all that money they spent on development uh would have
  136. been worth it because they could have sold uh more products and so when I'm talking to my students and Business
  137. Leaders who in some ways are my students uh uh I go look this is why we do it like I'm I'm going to speak to the same
  138. incentives that you have you want to make more money you want to keep your job uh you want to do right by your
  139. shareholders then you need to move uh the process or move the people earlier into the development of this process now
  140. in terms of this question around chat GPT and uh the way that uh people are using this inside and outside the
  141. classroom uh we developed the triap lab this these three questions call the 3D model of Equitable Tech adoption right
  142. and I'm going to share those questions with you and this will think help you understand help people understand what
  143. do I do with any Tech that comes into my business or into my classroom those three questions question one what does
  144. this product actually do not what do you want it to do not what it might do in the future not what it could do if we
  145. had uh a billion more hours of Compu like what does it actually do today second question is who does this product
  146. disempower all right every technology will increase power for some and decrease power for others so ask
  147. yourself who does this disempower and then the third question is what is the daily use of this product not the edge
  148. case don't tell me that you know we're going to uh I give you a per example for Hispanic Heritage Month there is uh some
  149. question that maybe we should include in facial recognition whether or not uh this person is Hispanic or this person
  150. is uh Latino and the sales pitch for this is that this will help us when we do the census to identify who is
  151. Hispanic or Latino so we have a better count right now I can get into like the research and on this and why this is
  152. problematic or some of the philosophy of like how this goes arve but let's talk about that case they're saying we're
  153. going to use this for the census the census happens every 10 years all right there is no government service that's
  154. going to buy this expensive technology and then only use it once every 10 years so then what's the most likely daily use
  155. case right of a technology that uh like says this person Hispanic or not right are they going to install it at Borders
  156. probably uh would they install where you hand over your uh passport probably right and is that problematic definitely
  157. right and so this is how I'm thinking about these things uh and in the case of chat GPT your students should ask
  158. themselves what does this actually do right is this writing papers or is it just creating plausible sounding
  159. sentences right and if it's just creating plausible sounding sentences o that could get real bad real fast uh if
  160. that plausibility has no relationship to accuracy all right so you know feel free
  161. to use it and feel free to fail that's on you uh that's come down to it so you're
  162. doing a great job both of you already of raising some of these key challenges but
  163. I'd like to explore these more and certainly you know we don't have time for tap tens so we'll do and I'm sure
  164. that there are 10 on each of your lists but let's think about this in terms of the top two or three issues from where
  165. you sit so project let's go back to you you can those that you've already talked
  166. about or you know invite others into the conversation what would you say from where you sit are the top two to three
  167. issues with respect to responsible and fair AI in the workplace and the marketplace uh two things uh the number
  168. one issue is that the human beings that end up using the AI or being users of machine learning or being users of this
  169. generative AI aren't necessarily the people that are included in the data all right aren't necessarily the people that
  170. are included in the classification of the data and definitely aren't the people that are setting the rules and
  171. doing the coding of the data and so the most pressing uh issue is to get those people into those spaces right to get
  172. representative data to get representative uh classification of that data to get representativeness in the
  173. actual coders that decide the rules for what we see and then that way the stuff that actually comes out will be closer
  174. to fair and Equitable because those people would be in the room so that's one the second thing is that all of my
  175. research uh is around this space is really to demystify this black box there is nothing magic all right going on inside
  176. of these statistical models they are statistical models and if you learned in high school y equal MX plus b then you
  177. like have the building block that you need to understand how these systems work uh we can talk about this if you
  178. ever come hang out on the Trap lab but trust me if you know y equals MX plus b all right then you too can start to
  179. understand that it's not magic going on inside of these systems but just a bunch
  180. of opinions uh around this commodify human labor and the data the classification and the code and it's
  181. interesting you know we've sort of hinted around sort of like language here um in this case I think I find myself
  182. personally feeling like it does sound like it's magic because the the language of AI and technology that we're using to
  183. talk about these things sounds very foreign to me and so I think that is what gives it its Mystique is if I can
  184. talk about it in a way that doesn't coincide with lay ter terminology that regular people use then it sounds like
  185. it's inaccessible to me right it's like it's a is that a power move I don't know
  186. but language is powerful and language can be used as a way to exclude and in this case you know I think what you're
  187. helping us to understand brck is you many of you because you learned y equals MX plus b you know that's a language
  188. that you spoke at some point and to the extent that we give people a grounding for understanding this new technology
  189. through something that they already understand language that they already possess then the I think the Mystique
  190. arounds it disappear the power differential between the creators of the technology and the consumers and the
  191. employees who were trying to figure out what does this mean for them that starts
  192. to be reduced as well is that sort of along some of the same lines R of of what you're suggesting that's exactly it
  193. right so like if you hear the term oh chat gbd says they have a billion parameter model right you'll go oh are
  194. big words billion parameter what does that mean let's explain it right let's break it down real quick so let's go
  195. back to this yals MX plus b b is call it our Y intercept some error term we're not going to worry about that we're just
  196. going to focus on the yals MX Y is an output every computer every machine right they're all uh based on the
  197. touring machine nothing really changes they can only do what they've been told to do why is going to be the output that
  198. comes out all right that could be plausible sounding sentences that could be art of Jesus flipping over
  199. a table whatever all right and then M and X is what matters X is inputs so if I'm deciding I'm building a system
  200. that's going decide whether or not uh someone gets parole for instance those X's might be ZIP code right problematic
  201. those X's might be past criminal history those X's might be height right those X
  202. could be anything then the m is what matters right the m is a essentially the slope of that line we learned this in uh
  203. uh 10th grade but that slope is an opinion it is some developers opinion or if it's unsupervised it's still some
  204. developers opinion on how much that X matters how much does it matter that I live in the zip code how much does it
  205. matter that you know I'm six foot6 right how much does it matter that my name is
  206. broaderick and then like that opinion gets added in so each one of those mxs we can call that
  207. a parameter so if it's a billion things has a billion opinions but they all get filtered out and come out to this output
  208. why that's it we have now learned machine learning congratulations everybody right like Pat yourselves on
  209. the back if you learned why goes MX plus b you too uh can start to understand what's going on these systems you can do
  210. like Kareem or myself and run audits where you basically test these systems uh with a bunch of stimuli to see what
  211. comes out to explain uh this thing is maybe leading to inequality because of uh this weirdness right and so let's
  212. just demystify the whole thing it's not I mean it's complex let me not poo poo my computer science folks out there but
  213. uh we can share a language that inside of you know this increasing complexity comes down to a pretty simple building
  214. block and you know the building block already if you made it through high school and our Wharton grads or NBAs and
  215. undergrads I know uh that you learned y equals MX plus b I have to tell you it's
  216. been a long time since I learned why equals MX plus b I'm not sure my high school teacher was as great as you were
  217. explaining that but I feel like I have in the last three minutes a much better understanding of what it is that people
  218. are trying to suggest is sophisticated um which often I think sometimes feels inaccessible but I think you made me
  219. believe and I'm sure others believe that they too can understand sort of like what the big deal here is and assess for
  220. themselves is it a big deal or is it just more uh of what we already know Kareem let me turn to you and let's get
  221. your top two to three issues I don't know how much they overlap I know earlier you spoke about you know issues
  222. around representation in the data not sure if that's one of your top two or three issues if you have others can you
  223. share with us um from your Vantage Point what are some of the key challenges yeah so first thing I agree
  224. with h bradrick in that you know transparency is a is a major issue right the Splat boox problem and trying to
  225. basically understand uh what companies are doing are companies being transparent about how they're training
  226. these models what data sets they're using are they explaining to their users what is happening behind the scenes
  227. obviously there's an optimal level of transparency as well like transparency after specific um you know uh particular
  228. level might get too in depth to the extent that the uh average user might zone out right it's irrelevant in
  229. information right you don't want to inundate your user with with uh too much technical details that they really zone
  230. out um so for me the the first pressing challenge is addressing uh unfairness in
  231. AI systems right so if you're training data like Rodrick has mentioned is missing certain populations or if your
  232. data is Mis mislabeled obviously that can give rise to bias and can have adverse effects on um you know certain
  233. segments of the population uh this particularly becomes uh you know problematic in issues of like Health
  234. Care employment uh you know uh criminal justice where these decisions are consequential for people right um so if
  235. obviously if these uh these issues of bias are not left unaddressed uh they can perpetuate uh unfairness in society
  236. at a very high rate right we're not just talking about your prototypical kind of
  237. BU we're talking at like an exponential rate with these automated decision systems which is why they um they can be
  238. very dangerous right so the second problem I see is what is called the hallucination problem which is pretty
  239. much like making up stuff right like rodick had mentioned these llms learn to predict the next word right or phrase in
  240. a sentence so um they can misrepresent facts uh they can also tell you a very good story uh that has a very good
  241. narrative uh that is extremely plausible uh but is misleading right and I think this is a particularly big problem uh
  242. given that we as human beings we have like an automation bias where uh we have a propensity to kind of favor
  243. suggestions from an automated decision system uh and to ignore um contradictory information that uh that you know we
  244. might know right we just can't kind of defer to uh this automated decision system right um the third one I would
  245. say is kind of like data privacy and security concerns and this involves things like unauthorized access uh of
  246. data scraping of data that for example the the the company or the llm uh might not have uh access to uh you know or
  247. consent to use uh things like data leakage for example where a large language model might um you know
  248. present some information that a user had used um and we have cases like that coming up in in the media where Samsung
  249. for example has banned uh the use of chat GPT VI its uh employees because of the fact that uh the the llm was uh you
  250. know revealing some of these Trade Secrets uh so things like malicious use and manipulation attempts by nefarious
  251. actors these are all very serious concerns as well yeah so I as as I listen to you all
  252. talk about these concerns it sort of raises the concerns that I've just had again just as an an employee right and
  253. certainly as as a consumer and and certainly what came across my uh social media feed recently was uh in the last
  254. couple weeks there was a uh lawsuit class action lawsuit uh filed in the Northern District of
  255. California um the plaintiffs are open Ai and chat GPT and the Microsoft cor Corporation um and one of the big areas
  256. in included in this um lawsuit is invasion of privacy and it's it's pretty compelling again I'm not a lawyer no and
  257. we none of us knows how this is going to pan out but just looking at it to understand the the lengths at which
  258. people people um don't understand how their data is being accessed and utilized without their permission um is
  259. it can be concerning and so I would encourage people to it's a it was a nice tutorial I would say if you will for me
  260. in privacy concerns around this topic but I'm going to certainly um put punt back to the two of you you come and if
  261. you've had a chance to like look at anything in relation to this class action lawsuit I'm wondering if you're
  262. surprised by this lawsuit or if you just thought that this was going to somebody
  263. it was just a matter of time before it happen any thoughts on that baric and then I'll come back to you kareim uh
  264. yeah so clearly it was only a matter of time and we think about again what do these systems actually do all right so
  265. these large language models train uh on previous data and training just means that they take in a bunch of data so
  266. they then connected uh together but whose data did they take where did that data come from now there is some
  267. indication that you know when you're training on a huge Corpus of uh data that you're goingon to tend towards the
  268. cheapest and freest sources right this is why we get some a lot of weirdness that comes out of these systems in terms
  269. of bias because they're trained on the free parts of the internet uh a lot of the time and what is over represented in
  270. the free Parts the internet Stephanie the free parts are people who I'm trying to think about social media
  271. as an example as a source of data and then I'm thinking about who the users might be um I'm thinking about um not
  272. sure if this is where you're going but I think about all the young people who use
  273. social Med put all of their information resources yeah so we so we have that right we have an over
  274. representation of younger folks all right so we're going to have weird age related things that don't really exist
  275. in the data the other thing that gets over represented on the internet on the free Parts at least is propaganda right
  276. so some websites where they'll say really negative things about presidents for instance are free whereas if I go to
  277. I don't know the New York Times I can read four articles before I get to a firewall and they're like no sir you're
  278. done right uh and so there's going to be some weirdness that's in the data from that the other that's free on the
  279. internet or over represented for free on the internet is pornography right so if
  280. I'm taking in porn and propaganda into these systems right then I'm going to get all types of weirdness out and the
  281. only way to improve these things is to get access to uh you know data that has closer to accuracy that has time spin
  282. into it you want to read one of our research articles Stephanie how much does it cost if you're like not
  283. affiliate with school it's like thousands of dollars right and so if you're one of these companies and you
  284. want to improve your data how do you do it do you spend thousands of dollars per
  285. article from Stephanie Cur or you steal it right is that like a a rhetorical question you can't say but I'll answer
  286. the question the answer is you're probably you're probably not gonna spend thousands of dollars per article per
  287. Professor If instead you can steal it right and this goes for any of we'll call it better data uh because I need a
  288. bunch of it to get to some version of it's not accurate but a better uh distribution of data and so yeah of
  289. course they're getting sued because to make the system better they had to take this stuff in and that we do have
  290. intellectual property laws somebody's going to have to pay or they're gonna have to change the law Kareem do you
  291. want to chime in on this conversation before we move to the how do we fix it yeah absolutely I I think um you know
  292. lawsuits are just going to mushroom from here onwards uh it's just a matter of time honestly you know every day we're
  293. hearing about another lawsuit uh you know whether it's relating to privacy or discrimination or um other aspects um so
  294. to me I'm I'm not too surprised I think uh these companies are you know obviously I I can't speak on on their
  295. behalf but you know we have a responsib ility to make sure that we're training our llms on reliable data sources uh we
  296. need to kind of fine-tune these llms we need to have like some kind of a factchecking mechanism to ensure uh you
  297. know if they're producing uh garbage that you know they're they're being kind of they're receiving feedback and and
  298. they're improving we have to have human reviewers in the loop uh that can play a
  299. role in in correcting like kind of the the trajectory of of these llms um you know perhaps even including like
  300. confidence scores to allow users to kind of understand Engage The reliability of
  301. the information that they're getting right and uh and even to uh provide users with a mechanism to provide
  302. feedback for the llm as to whether you know the response that I got was terrible or not so that uh they're
  303. taking in another signal from their users so I think in conjunction all of these different measures uh hopefully
  304. over the course of time and as long as there's you know strong Buy in from leadership to kind of improve these
  305. models I think um companies can do a better job at uh protecting the privacy of people but also ensuring that you
  306. know whatever gets propagated uh in terms of outputs is more reliable okay uh any other suggestions Solutions um
  307. around that or the other challenges that you and brck brought up today um in like
  308. we think about like one or two key things that can be done uh for various audiences and I think I'm thinking of
  309. lots of audiences I'm thinking of um you know employers and institutions I'm thinking of consumers and employees as
  310. well yeah there's I let kareim go first on that one yeah I think there's the grass is so green um there is so much
  311. that can be done in this space um so firstly there needs to be uh legislation there needs to be comprehensive AI
  312. legislation and enforcement to protect public interests uh not sufficient for companies to have voluntary commitments
  313. that's good but it uh it's not sufficient second thing is you know there needs to be um when we're thinking
  314. about data protection laws uh they need to be strengthened to include you know AI systems and Ure that um the Privacy
  315. rights of users and so forth uh you know we spoke about transparency and that companies uh need to disclose how AI
  316. systems have been built what data sources they've been using and to kind of um to take a crack at the blackbox
  317. problem uh stakeholder engagement is really important you know as we're thinking through diverse audiences and
  318. data uh sources uh you need to engage with different stakeholders I always say when you're working in this
  319. space you really need to get it like be cross functional because there's a lot of barbwire in this space you know
  320. whether it is you know privacy legal regulatory you're working with ethicists product managers you're working with uh
  321. data scientists researchers uh you know Eng it is extremely cross functional space and it has to be that way because
  322. it is a soot technical problem that uh takes all the great minds to be able to attack it from different ways you need
  323. to have diverse teams for example so that you have representation people can identify and rectify bias effectively
  324. right training data needs to be more inclusive make sure you know like bradrick was saying who's missing out
  325. from this data like who are we not seeing you know I always say that um this technology is not neutral um this
  326. technology is already has a point of view and its point of view is what it has been trained on right so if it has
  327. been missing people of color for example then that's its point of view right it's
  328. not like it's coming as a blank slate no it is already has a point of view and so
  329. uh that's a little bit of mythbusting so I mean we need to kind of like take inclusive data and inclusive uh
  330. inclusivity into consideration uh throughout the product development life cycle right we need to have like audits
  331. that are done on uh these these models and and ensure that you know we have uh results and that can kind of feed into
  332. how we can improve uh human oversight for example I mentioned this earlier that there needs to be a human in the
  333. loop to be able to kind of uh you know spot and uh improve the trajectory of uh outcomes I mean there's so much that can
  334. be done I can go on and on and on but start yeah so much brck your turn what would
  335. you say um are some key specific things that can be done so I'm going to break this down into uh different market
  336. segments because that's how my brain is wired so we're going to talk about what can companies do what can consumers do
  337. what can researchers do and then what can your students do all right so first for companies consider that 3D model
  338. that I laid out earlier before you roll out a product what is this product do who does it disempower uh and what uh is
  339. its daily use all right if you need help answering these questions then call RA audits uh you know Kore will pick up the
  340. phone uh or you can holler at the folks over here in the Trap lab right and we can help you think through some of these
  341. things uh as we're trying to make this technology better and more Equitable all right two consumers what can you do do
  342. not accept that It's Magic there is nothing magical about these systems these systems are just people and their
  343. opinions of people if you learned yals MX plus b then you have learned the building blocks of every machine
  344. Learning System all right so if there is some weird outcome that comes out when you're on Facebook or Twitter or you
  345. notice some weirdness on your uh when you you put out an application for a loan trust your gut all right it is weird and
  346. say something because again the only way to improve these things as well is uh uh
  347. for them to update their model and it's you you are right that something's wrong
  348. all right it is not magic is not your fault is them uh don't accept magic for researchers if you are interested in
  349. working on these topics and in this space go to jointhe trap.com and come hang out with us uh we meet online once
  350. a week over Zoom Wednesdays at 1 pm uh and then finally for students this is going to be a weird one uh because in
  351. some ways this rise of uh generative Ai and writing and art uh makes people believe that we are like this close to
  352. uh having AI that can write beautiful books or make beautiful art uh and I'm going to challenge that all right we
  353. they also told us we were this close to having fully self-driving cars uh within
  354. a year 10 years ago and they said this every year and it's not happening anytime soon all right I'm going to say
  355. the same thing is the case for art and writing and uh creativity I think that they will be a
  356. premium uh on people that can actually Express themselves clearly and accurately and honestly and so if you
  357. are currently a student uh anywhere if you're in a b school if you're in a college if you're in high school and
  358. you're listening to this I don't know why you're listening to a Wharton podcast in high school but maybe you
  359. know you're one of those kids uh get really into building your uh toolbox of creativity right get really into uh
  360. creative writing right get really into making more art because there will be a value on the ACT actual human element
  361. that comes out of this if uh what everyone else is doing is just plugging in some chatbot that gives you uh
  362. plausible sentences that aren't that good if you can write better sentences you'll
  363. win thank you and and I'm just going to tell you project we actually do have knowledge at war in high school so your
  364. high school learners are going to learn a lot from what you all share today as much as your you know fully experienced
  365. senior leaders will um as well uh so so broject and Kem this has been fantastic I feel so much more knowledgeable as
  366. somebody who believes that I know a lot about a lot of things I I know that I don't know a lot about this topic and so
  367. the past amount of time 45 minutes that we've been chatting have been amazing and just I think helping me to feel more
  368. empowered um as a researcher a consumer and a worker around sort of what exactly
  369. is happening and and how I need to pay attention um to to what is being shared um so I want to thank you uh so much for
  370. sharing your insights and your expertise with us uh and to all the uh leading diversity at work podcast listeners we
  371. truly appreciate you for being here uh so that's all for now thanks to our audience for joining us and listening to
  372. this episode of the knowledge at Warton leading diversity atw work podcast series goodbye for
  373. now for more insight from knowledge at Wharton please visit knowledge. won. [Music]
  374. EDU

Brief Summary

Experts discuss the importance of responsible AI, representation, and transparency in technology's impact on society.

Key points

  • Introduction to Responsible AI. Stephanie Cy welcomes guests to discuss responsible AI in the workplace.
  • Dr. Broer Turner's Research. Dr. Turner shares insights on race and technology's impact on consumer decisions.
  • Dr. Kareem Gan's AI Governance. Dr. Gan discusses his role in AI governance and the importance of ethical practices.
  • The Importance of Representation. The conversation highlights the need for diverse representation in AI development.
  • Demystifying AI Models. Understanding AI isn't magic; it's about basic math principles like y = MX + b.
  • The Hallucination Problem. AI can misrepresent facts, leading to dangerous consequences in decision-making.

Tension

45/ 100Some tension

Related episodes