Diversity Is Critical for the Future of AI — Leading Diversity at Work
Ask about Knowledge at Wharton
Episode Description
EPISODE OVERVIEW In this episode of Leading Diversity at Work, Wharton’s Stephanie Creary speaks with Broderick Turner -- a Virginia Tech marketing professor who also runs the school’s Technology, Race, and Prejudice (T.R.A.P.) Lab -- and Karim Ginena -- a social scientist and founder of RAI Audit -- on how to use AI while thinking critically about its flaws.
SERIES OVERVIEW Leading Diversity at Work is a podcast series featuring Wharton management professor Stephanie Creary in conversation with experts in diversity, equity and inclusion. Learn more at http://whr.tn/leadingdiversity.
PODCAST OVERVIEW The Knowledge at Wharton podcast features interviews with Wharton faculty and industry leaders about the business landscape and the impact of their work. Conversations range from the latest research and books by faculty authors, to Leading Diversity at Work with professor Stephanie Creary, and Dollars and Change with professor Katherine Klein.
#AI #artificialintelligence #dei #diversity ----- The Wharton School of the University of Pennsylvania is committed to sharing its intellectual capital through Knowledge at Wharton, the school’s online business analysis journal.
Launched in 1999, Knowledge at Wharton offers free access to:
- Articles, podcasts, and videos highlighting Wharton faculty research and analysis of current business trends - Interviews with book authors from Wharton School Press - In-depth series of curated content like Wharton Executive Education’s Nano Tools for Leaders - A searchable database of more than 10,000 articles covering all aspects of business - A weekly newsletter that delivers Knowledge at Wharton insights directly to your inbox
Visit the Knowledge at Wharton homepage: https://whr.tn/3qUpSBE Subscribe to the Knowledge at Wharton YouTube channel: https://whr.tn/3DEAgAU
Read the full transcriptHide the full transcript
- this podcast is brought to you by knowled of [Music] Warton hello my name is Stephanie Cy and
- I'm an assistant professor of management at the Wharton School of the University
- of Pennsylvania and I'm delighted to welcome you to today's episode of the knowledge at War and leading diversity
- at work podcast series which is focus focused on responsible and fair air AI in the workplace and the marketplace
- joining me today are two very special guests first we have Dr broer Turner who is an assistant professor of marketing
- at Virginia Tech Pamplin School of Business and a visiting fellow at Harvard Business school's Institute for
- business and Global Society he also runs the technology race and Prejudice lab or
- the Trap lab where he and his fellow Trappers are pushing the boundaries on understanding how race and racism
- underly many consumer and managerial decisions his main research area focuses on the intersection of marketing
- technology racism and emotion next we have Dr kareim Gan who is the founder of RA AI audit an AI governance and
- research consultancy with 16 years of experience in ethics and governance spanning both industry and Academia he
- recently served as the founding user experience researcher on meta's responsible AI team leading AI fairness
- user research at the company he has helped meta our AI team scale its products across the company and bolster
- the responsible adoption of AI Dr ganana holds a PhD in management from the University of Virginia's Darden School
- of Business specializing in business ethics and organizational behavior and I just realized we've got a Virginia
- connection uh across the two of you here who would have who would have thought that that would happen but in any case
- welcome bro and Kareem I am so honored to have you with me today for a conversation on responsible and fair AI
- in the workplace and in the marketplace so we're hoping to cover you know a lot of interesting ground here today and as
- I as talked to brck and cre about this earlier I like to think of myself as a as a novice on these topics so today I'm
- going to represent the average uh consumer um and worker who doesn't have a lot of experience talking about um AI
- other than what I read about and hear about um on mainstream media and from my colleagues so so we're hoping today that
- um this conversation will be helpful to others of you who are just like me who are trying to figure out how relevant uh
- a conversation about responsible and fair AI is to us as workers but also as consumers so let's get started talking
- about broadly about responsible and fair Ai and so project I'm I'm going to go to
- you first and what I'm hoping you can do is is share a bit about how and why you
- became involved in this topic and summarize for us some of your current work related to this topic yeah so I got
- involved in this topic area uh honestly more than a decade ago when I was teaching high school math uh at a public
- high school in Atlanta Georgia where I taught like 90% uh black students and what I wanted
- to do while I was there was figure out ways to make those kids' lives better fairer more Equitable uh and
- simultaneously I used to teach linear regression and students asked me Mr turn am I ever going to use this and now I
- have an answer the answer is now that uh you know so I've moved into this space where I'm thinking a lot about how race
- and racism underly uh Market Systems and Technology being a system that matters and our
- research group the technology race and Prejudice lab is doing active research now on what levers can be moved to lead
- to uh more equality in systems to lead to better outcomes in technology uh this has extended some into some uh company
- advisory work where uh we're trying to get company consider that moving uh communities earlier into the development
- process leads to better uh Products that come out out and then on the public knowledge side uh we're writing white
- papers on uh how different uh identity groups may be impacted by gender of AI for instance so we just uh finished a
- report on for Hispanic herited Munch Hispanic herit Heritage Month thank you on how uh in uh how folks with Hispanic
- Origins and background should be thinking about gender of AI how do they fit into this interlan system of data
- the classification code okay great thank you so much uh so Kareem same question to you can you share a bit about how and
- why you became involved in this topic around responsible and fair Ai and summarize some of your current work on
- this topic yeah absolutely firstly thank you so much for inviting me I'm excited
- about this dialogue with yourself and brri and uh love the Virginia connection Virginia is well represented here today
- um so i' I've been involved in ethics and governance um for quite some time you know prior to
- pursuing my PhD so I'm not surprised that I ended up in kind of AI governance uh what I'm a little bit surprised about
- is actually post pursuing my PhD going back to uh industry initially when I started off my PhD program I had no
- intentions uh to go back to industry but in my last year of my PhD which was my sixth year I had received offers
- basically from Academia and Industry and uh I went for back to Industry so um firstly as someone who belongs to a
- minority group um I was very passionate and still am about having a front seat to these conversations that are
- happening in Tech and uh helping shape the uh the technology using you know the latest and greatest research but also
- interacting with product teams to be able to do that policy and legal and so forth um the other reason that I chose
- to go back was you know as I was thinking through my impact to be honest uh joining a company
- like meta's responsible AI team where the company has over three three billion users I think you know it's very
- difficult to kind of argue with the extent of impact that you can do uh when you uh you have three billion users and
- any kind of like product changes that you make can impact really a lot of people worldwide and and lastly honestly
- uh one of the concerns that I've had about uh Academia was you know being stuck in a small college town somewhere
- uh where there's not very much diversity and uh that would not be appropriate for
- the upbringing of my kids and so that was a a a very uh strong concern uh not having much of an autonomy about where I
- live or where I work um so it drove me back to actually go to um to Industry now as far as my research
- uh much of what I do for clients is obviously confidential but on the public side of things I've been testing out uh
- some popular AI image generators um and I'm finding a few patterns so uh most of
- the images uh that are being generated from uh the uh at least the AI generators uh that I have tested tend to
- be of people from the white race uh people of Co color are uh greatly overlooked uh these models tend to
- associate uh a professional headshot or a professional dress code or a professional hairstyle with those of uh
- white people uh women are underrepresented uh and uh when presented as professionals they're often
- limited to gendered occupations such as teachers nurses graphic designers and much less likely to produ to appear as
- CEOs for example or medical doctors or lawyers um images also tend to spew on the younger side of things and Senior
- managers for example are associated with having gray hair which is a sign of aism um and finally you know these
- models tend to portray certain populations in demeaning ways for example uh if if you ask Del Bing to
- portray Turks uh it will uh provide you with a picture of a Stern turkey uh a Stern turkey dressed in a turban right
- um whereas if you're ask it to produce for example an image of an American or a French person or something like that
- will do a much better job right um they might also refuse to produce images of certain populations For No Good Reason
- such as if you ask it to produce um an image of a Muslim or a Jew uh while presenting followers of other religions
- so definitely this technology is extremely powerful uh I am not a doomsayer per se as it uh relating to
- the technology but I believe uh that you know we have to do our prudent and due diligence in order
- for us to be able to direct the trajectory of this technology in ways um such that we're maximizing the benefits
- while minimizing the harms my my favorite example of uh that gender of AI uh art is when people do prompts for
- Jesus in the temple flipping the table that they literally get Jesus doing like gymnastics doing a backflip over a table
- uh so yeah text text has no meaning it's it's just funny absolutely so we're gonna
- definitely dive into there's a lot of to unpack there and certainly the implications of this I mean I think of
- where I sit I can automatically connect those dots but I think as we begin to talk about some of the challenges it'll
- be important for people to understand what could possibly go wrong when you you're not represented in how these
- models are being uh trained I I think to me that's an obvious answer but I don't
- think that's obvious to everybody what happens when you lack representation in in the data amongst the data or you're
- misrepresented right who you are and the the the groups that you're part of are misrepresented in in how the model is
- being trained so we're going to talk about that in a minute but let me just and probably this is part of the answer
- to your question here Kareem um let's let's start with business and employers first right and and I can imagine I have
- some sense of this from my own work that businesses and employers um are likely and can be struggling to understand
- their conversation not just in a role about AI use but fair and responsible AI use and so obviously without you know
- saying anything confidential can you talk generally about how you through your work are helping businesses
- employers make sense of their roles and responsibilities on this topic yeah I think it's important to be
- clear that um companies are accountable for how their AI systems operate and that responsibility can be offloaded
- right it can be considered an externality of some sort they have obligations to their customers and it's
- just a matter of time uh before legislation comes into effect in this space you know obviously we have right
- now the uh EU AI act which we're waiting it's just a matter of time before enforcement happens and there's a lot of
- talk uh about legislation uh Congress and so forth here in the United States um and honestly if come companies want
- to build successful products in this AI era era they must invest in responsible AI infrastructure right um doing so
- requires a commitment from the board of directors from senior leadership but ALS
- ultimately what it does is it pays off uh in terms of earning customer trust and confidence right it it just doesn't
- cut it for you to actually have an AI strategy uh without thinking through the the trust layer or the responsible layer
- um you know issues of like fairness and privacy and um robustness and and so forth right because if if you're a
- product manager for example you're trying to produce the best products no one really wants to deal with a company
- whose products are reaching you know their privacy uh you know their their privacy obligations and stuff like that
- like if you're using personal data uh of your users to train your models or if your products don't work for certain
- segments of the population or exposes your users to specific types of harms you know cutting Corners like that
- really does a disservice uh to your stakeholders of course uh but it also exposes you to legal and reputational uh
- risk and it's just a bad way of doing business it's just a matter of time before you know uh better companies are
- able to gain more market share because they're taking this more um seriously and investing in setting up the
- infrastructure yeah you know a lot of what I hear you saying is you know sometimes it's easy to think that ethics
- and responsibility questions of responsibility isn't the job of a corporation but you take a strong stance
- and certainly through your own dissertation work and your own scholarship is that these are things
- that are not nice to haves um there must haves and and you know while it's it shouldn't always come down to this we do
- have a system called the legal system which is set in place to help adjudicate these issues when it does feel like it's
- a great area even though a gray area excuse me even might be black and white with respect to businesses and their
- responsibility to ensure that they're not inducing harm uh through their activities brri I mean certainly I know
- that you do work with companies as well I I would also like you to put on your researcher and educator hat um and help
- us to understand like the nature of this conversation from a scientific perspective and also in the work that
- you're doing you know as a professor as part of your fellowship I'm actually thinking a lot about chat GPT these days
- certainly because as an educator this is sort of the one this is probably the most I know about this conversation
- around Should students be able to use GPT to complete their assignments and it seems to me like it's such a polarizing
- sets of issues um so I am just just curious as you think about the the places where you set project um how
- you're thinking about the issue around businesses and employers and Ed education's role in this conversation so
- I think a couple of thoughts right so first uh to piggy back on Kareem when I talk to both students and businesses I
- think of myself as an educator no matter where I am right and I have the same thing I tell them uh which is if you get
- this wrong right you don't include uh a wide SWA of human beings in the creation
- of your technology products when you fail because it's not an if but when you fail you lose money
- because you spent all this money on development without considering uh the human beings at the end and when the
- product gets released the product fails again because people don't use it right so as a really simple example uh those
- automatic uh faucets where you put put your hand under it did not include uh like darker melanated people
- in the data set in the training set for when they were testing out this automatic hand washer and so while it
- sailed sold fine in North America when it went further out into uh places closer to the Equator where people were
- Browner regardless of uh ethnic origin they didn't sell any because the product did not work they would install it and
- people put their hand under the thing and I know if you're a black or a brown person you have done this at the airport
- bathroom been upset right but if your country is majority melanated then no one bought them and so have they
- included those folks earlier in the development process like all that money they spent on development uh would have
- been worth it because they could have sold uh more products and so when I'm talking to my students and Business
- Leaders who in some ways are my students uh uh I go look this is why we do it like I'm I'm going to speak to the same
- incentives that you have you want to make more money you want to keep your job uh you want to do right by your
- shareholders then you need to move uh the process or move the people earlier into the development of this process now
- in terms of this question around chat GPT and uh the way that uh people are using this inside and outside the
- classroom uh we developed the triap lab this these three questions call the 3D model of Equitable Tech adoption right
- and I'm going to share those questions with you and this will think help you understand help people understand what
- do I do with any Tech that comes into my business or into my classroom those three questions question one what does
- this product actually do not what do you want it to do not what it might do in the future not what it could do if we
- had uh a billion more hours of Compu like what does it actually do today second question is who does this product
- disempower all right every technology will increase power for some and decrease power for others so ask
- yourself who does this disempower and then the third question is what is the daily use of this product not the edge
- case don't tell me that you know we're going to uh I give you a per example for Hispanic Heritage Month there is uh some
- question that maybe we should include in facial recognition whether or not uh this person is Hispanic or this person
- is uh Latino and the sales pitch for this is that this will help us when we do the census to identify who is
- Hispanic or Latino so we have a better count right now I can get into like the research and on this and why this is
- problematic or some of the philosophy of like how this goes arve but let's talk about that case they're saying we're
- going to use this for the census the census happens every 10 years all right there is no government service that's
- going to buy this expensive technology and then only use it once every 10 years so then what's the most likely daily use
- case right of a technology that uh like says this person Hispanic or not right are they going to install it at Borders
- probably uh would they install where you hand over your uh passport probably right and is that problematic definitely
- right and so this is how I'm thinking about these things uh and in the case of chat GPT your students should ask
- themselves what does this actually do right is this writing papers or is it just creating plausible sounding
- sentences right and if it's just creating plausible sounding sentences o that could get real bad real fast uh if
- that plausibility has no relationship to accuracy all right so you know feel free
- to use it and feel free to fail that's on you uh that's come down to it so you're
- doing a great job both of you already of raising some of these key challenges but
- I'd like to explore these more and certainly you know we don't have time for tap tens so we'll do and I'm sure
- that there are 10 on each of your lists but let's think about this in terms of the top two or three issues from where
- you sit so project let's go back to you you can those that you've already talked
- about or you know invite others into the conversation what would you say from where you sit are the top two to three
- issues with respect to responsible and fair AI in the workplace and the marketplace uh two things uh the number
- one issue is that the human beings that end up using the AI or being users of machine learning or being users of this
- generative AI aren't necessarily the people that are included in the data all right aren't necessarily the people that
- are included in the classification of the data and definitely aren't the people that are setting the rules and
- doing the coding of the data and so the most pressing uh issue is to get those people into those spaces right to get
- representative data to get representative uh classification of that data to get representativeness in the
- actual coders that decide the rules for what we see and then that way the stuff that actually comes out will be closer
- to fair and Equitable because those people would be in the room so that's one the second thing is that all of my
- research uh is around this space is really to demystify this black box there is nothing magic all right going on inside
- of these statistical models they are statistical models and if you learned in high school y equal MX plus b then you
- like have the building block that you need to understand how these systems work uh we can talk about this if you
- ever come hang out on the Trap lab but trust me if you know y equals MX plus b all right then you too can start to
- understand that it's not magic going on inside of these systems but just a bunch
- of opinions uh around this commodify human labor and the data the classification and the code and it's
- interesting you know we've sort of hinted around sort of like language here um in this case I think I find myself
- personally feeling like it does sound like it's magic because the the language of AI and technology that we're using to
- talk about these things sounds very foreign to me and so I think that is what gives it its Mystique is if I can
- talk about it in a way that doesn't coincide with lay ter terminology that regular people use then it sounds like
- it's inaccessible to me right it's like it's a is that a power move I don't know
- but language is powerful and language can be used as a way to exclude and in this case you know I think what you're
- helping us to understand brck is you many of you because you learned y equals MX plus b you know that's a language
- that you spoke at some point and to the extent that we give people a grounding for understanding this new technology
- through something that they already understand language that they already possess then the I think the Mystique
- arounds it disappear the power differential between the creators of the technology and the consumers and the
- employees who were trying to figure out what does this mean for them that starts
- to be reduced as well is that sort of along some of the same lines R of of what you're suggesting that's exactly it
- right so like if you hear the term oh chat gbd says they have a billion parameter model right you'll go oh are
- big words billion parameter what does that mean let's explain it right let's break it down real quick so let's go
- back to this yals MX plus b b is call it our Y intercept some error term we're not going to worry about that we're just
- going to focus on the yals MX Y is an output every computer every machine right they're all uh based on the
- touring machine nothing really changes they can only do what they've been told to do why is going to be the output that
- comes out all right that could be plausible sounding sentences that could be art of Jesus flipping over
- a table whatever all right and then M and X is what matters X is inputs so if I'm deciding I'm building a system
- that's going decide whether or not uh someone gets parole for instance those X's might be ZIP code right problematic
- those X's might be past criminal history those X's might be height right those X
- could be anything then the m is what matters right the m is a essentially the slope of that line we learned this in uh
- uh 10th grade but that slope is an opinion it is some developers opinion or if it's unsupervised it's still some
- developers opinion on how much that X matters how much does it matter that I live in the zip code how much does it
- matter that you know I'm six foot6 right how much does it matter that my name is
- broaderick and then like that opinion gets added in so each one of those mxs we can call that
- a parameter so if it's a billion things has a billion opinions but they all get filtered out and come out to this output
- why that's it we have now learned machine learning congratulations everybody right like Pat yourselves on
- the back if you learned why goes MX plus b you too uh can start to understand what's going on these systems you can do
- like Kareem or myself and run audits where you basically test these systems uh with a bunch of stimuli to see what
- comes out to explain uh this thing is maybe leading to inequality because of uh this weirdness right and so let's
- just demystify the whole thing it's not I mean it's complex let me not poo poo my computer science folks out there but
- uh we can share a language that inside of you know this increasing complexity comes down to a pretty simple building
- block and you know the building block already if you made it through high school and our Wharton grads or NBAs and
- undergrads I know uh that you learned y equals MX plus b I have to tell you it's
- been a long time since I learned why equals MX plus b I'm not sure my high school teacher was as great as you were
- explaining that but I feel like I have in the last three minutes a much better understanding of what it is that people
- are trying to suggest is sophisticated um which often I think sometimes feels inaccessible but I think you made me
- believe and I'm sure others believe that they too can understand sort of like what the big deal here is and assess for
- themselves is it a big deal or is it just more uh of what we already know Kareem let me turn to you and let's get
- your top two to three issues I don't know how much they overlap I know earlier you spoke about you know issues
- around representation in the data not sure if that's one of your top two or three issues if you have others can you
- share with us um from your Vantage Point what are some of the key challenges yeah so first thing I agree
- with h bradrick in that you know transparency is a is a major issue right the Splat boox problem and trying to
- basically understand uh what companies are doing are companies being transparent about how they're training
- these models what data sets they're using are they explaining to their users what is happening behind the scenes
- obviously there's an optimal level of transparency as well like transparency after specific um you know uh particular
- level might get too in depth to the extent that the uh average user might zone out right it's irrelevant in
- information right you don't want to inundate your user with with uh too much technical details that they really zone
- out um so for me the the first pressing challenge is addressing uh unfairness in
- AI systems right so if you're training data like Rodrick has mentioned is missing certain populations or if your
- data is Mis mislabeled obviously that can give rise to bias and can have adverse effects on um you know certain
- segments of the population uh this particularly becomes uh you know problematic in issues of like Health
- Care employment uh you know uh criminal justice where these decisions are consequential for people right um so if
- obviously if these uh these issues of bias are not left unaddressed uh they can perpetuate uh unfairness in society
- at a very high rate right we're not just talking about your prototypical kind of
- BU we're talking at like an exponential rate with these automated decision systems which is why they um they can be
- very dangerous right so the second problem I see is what is called the hallucination problem which is pretty
- much like making up stuff right like rodick had mentioned these llms learn to predict the next word right or phrase in
- a sentence so um they can misrepresent facts uh they can also tell you a very good story uh that has a very good
- narrative uh that is extremely plausible uh but is misleading right and I think this is a particularly big problem uh
- given that we as human beings we have like an automation bias where uh we have a propensity to kind of favor
- suggestions from an automated decision system uh and to ignore um contradictory information that uh that you know we
- might know right we just can't kind of defer to uh this automated decision system right um the third one I would
- say is kind of like data privacy and security concerns and this involves things like unauthorized access uh of
- data scraping of data that for example the the the company or the llm uh might not have uh access to uh you know or
- consent to use uh things like data leakage for example where a large language model might um you know
- present some information that a user had used um and we have cases like that coming up in in the media where Samsung
- for example has banned uh the use of chat GPT VI its uh employees because of the fact that uh the the llm was uh you
- know revealing some of these Trade Secrets uh so things like malicious use and manipulation attempts by nefarious
- actors these are all very serious concerns as well yeah so I as as I listen to you all
- talk about these concerns it sort of raises the concerns that I've just had again just as an an employee right and
- certainly as as a consumer and and certainly what came across my uh social media feed recently was uh in the last
- couple weeks there was a uh lawsuit class action lawsuit uh filed in the Northern District of
- California um the plaintiffs are open Ai and chat GPT and the Microsoft cor Corporation um and one of the big areas
- in included in this um lawsuit is invasion of privacy and it's it's pretty compelling again I'm not a lawyer no and
- we none of us knows how this is going to pan out but just looking at it to understand the the lengths at which
- people people um don't understand how their data is being accessed and utilized without their permission um is
- it can be concerning and so I would encourage people to it's a it was a nice tutorial I would say if you will for me
- in privacy concerns around this topic but I'm going to certainly um put punt back to the two of you you come and if
- you've had a chance to like look at anything in relation to this class action lawsuit I'm wondering if you're
- surprised by this lawsuit or if you just thought that this was going to somebody
- it was just a matter of time before it happen any thoughts on that baric and then I'll come back to you kareim uh
- yeah so clearly it was only a matter of time and we think about again what do these systems actually do all right so
- these large language models train uh on previous data and training just means that they take in a bunch of data so
- they then connected uh together but whose data did they take where did that data come from now there is some
- indication that you know when you're training on a huge Corpus of uh data that you're goingon to tend towards the
- cheapest and freest sources right this is why we get some a lot of weirdness that comes out of these systems in terms
- of bias because they're trained on the free parts of the internet uh a lot of the time and what is over represented in
- the free Parts the internet Stephanie the free parts are people who I'm trying to think about social media
- as an example as a source of data and then I'm thinking about who the users might be um I'm thinking about um not
- sure if this is where you're going but I think about all the young people who use
- social Med put all of their information resources yeah so we so we have that right we have an over
- representation of younger folks all right so we're going to have weird age related things that don't really exist
- in the data the other thing that gets over represented on the internet on the free Parts at least is propaganda right
- so some websites where they'll say really negative things about presidents for instance are free whereas if I go to
- I don't know the New York Times I can read four articles before I get to a firewall and they're like no sir you're
- done right uh and so there's going to be some weirdness that's in the data from that the other that's free on the
- internet or over represented for free on the internet is pornography right so if
- I'm taking in porn and propaganda into these systems right then I'm going to get all types of weirdness out and the
- only way to improve these things is to get access to uh you know data that has closer to accuracy that has time spin
- into it you want to read one of our research articles Stephanie how much does it cost if you're like not
- affiliate with school it's like thousands of dollars right and so if you're one of these companies and you
- want to improve your data how do you do it do you spend thousands of dollars per
- article from Stephanie Cur or you steal it right is that like a a rhetorical question you can't say but I'll answer
- the question the answer is you're probably you're probably not gonna spend thousands of dollars per article per
- Professor If instead you can steal it right and this goes for any of we'll call it better data uh because I need a
- bunch of it to get to some version of it's not accurate but a better uh distribution of data and so yeah of
- course they're getting sued because to make the system better they had to take this stuff in and that we do have
- intellectual property laws somebody's going to have to pay or they're gonna have to change the law Kareem do you
- want to chime in on this conversation before we move to the how do we fix it yeah absolutely I I think um you know
- lawsuits are just going to mushroom from here onwards uh it's just a matter of time honestly you know every day we're
- hearing about another lawsuit uh you know whether it's relating to privacy or discrimination or um other aspects um so
- to me I'm I'm not too surprised I think uh these companies are you know obviously I I can't speak on on their
- behalf but you know we have a responsib ility to make sure that we're training our llms on reliable data sources uh we
- need to kind of fine-tune these llms we need to have like some kind of a factchecking mechanism to ensure uh you
- know if they're producing uh garbage that you know they're they're being kind of they're receiving feedback and and
- they're improving we have to have human reviewers in the loop uh that can play a
- role in in correcting like kind of the the trajectory of of these llms um you know perhaps even including like
- confidence scores to allow users to kind of understand Engage The reliability of
- the information that they're getting right and uh and even to uh provide users with a mechanism to provide
- feedback for the llm as to whether you know the response that I got was terrible or not so that uh they're
- taking in another signal from their users so I think in conjunction all of these different measures uh hopefully
- over the course of time and as long as there's you know strong Buy in from leadership to kind of improve these
- models I think um companies can do a better job at uh protecting the privacy of people but also ensuring that you
- know whatever gets propagated uh in terms of outputs is more reliable okay uh any other suggestions Solutions um
- around that or the other challenges that you and brck brought up today um in like
- we think about like one or two key things that can be done uh for various audiences and I think I'm thinking of
- lots of audiences I'm thinking of um you know employers and institutions I'm thinking of consumers and employees as
- well yeah there's I let kareim go first on that one yeah I think there's the grass is so green um there is so much
- that can be done in this space um so firstly there needs to be uh legislation there needs to be comprehensive AI
- legislation and enforcement to protect public interests uh not sufficient for companies to have voluntary commitments
- that's good but it uh it's not sufficient second thing is you know there needs to be um when we're thinking
- about data protection laws uh they need to be strengthened to include you know AI systems and Ure that um the Privacy
- rights of users and so forth uh you know we spoke about transparency and that companies uh need to disclose how AI
- systems have been built what data sources they've been using and to kind of um to take a crack at the blackbox
- problem uh stakeholder engagement is really important you know as we're thinking through diverse audiences and
- data uh sources uh you need to engage with different stakeholders I always say when you're working in this
- space you really need to get it like be cross functional because there's a lot of barbwire in this space you know
- whether it is you know privacy legal regulatory you're working with ethicists product managers you're working with uh
- data scientists researchers uh you know Eng it is extremely cross functional space and it has to be that way because
- it is a soot technical problem that uh takes all the great minds to be able to attack it from different ways you need
- to have diverse teams for example so that you have representation people can identify and rectify bias effectively
- right training data needs to be more inclusive make sure you know like bradrick was saying who's missing out
- from this data like who are we not seeing you know I always say that um this technology is not neutral um this
- technology is already has a point of view and its point of view is what it has been trained on right so if it has
- been missing people of color for example then that's its point of view right it's
- not like it's coming as a blank slate no it is already has a point of view and so
- uh that's a little bit of mythbusting so I mean we need to kind of like take inclusive data and inclusive uh
- inclusivity into consideration uh throughout the product development life cycle right we need to have like audits
- that are done on uh these these models and and ensure that you know we have uh results and that can kind of feed into
- how we can improve uh human oversight for example I mentioned this earlier that there needs to be a human in the
- loop to be able to kind of uh you know spot and uh improve the trajectory of uh outcomes I mean there's so much that can
- be done I can go on and on and on but start yeah so much brck your turn what would
- you say um are some key specific things that can be done so I'm going to break this down into uh different market
- segments because that's how my brain is wired so we're going to talk about what can companies do what can consumers do
- what can researchers do and then what can your students do all right so first for companies consider that 3D model
- that I laid out earlier before you roll out a product what is this product do who does it disempower uh and what uh is
- its daily use all right if you need help answering these questions then call RA audits uh you know Kore will pick up the
- phone uh or you can holler at the folks over here in the Trap lab right and we can help you think through some of these
- things uh as we're trying to make this technology better and more Equitable all right two consumers what can you do do
- not accept that It's Magic there is nothing magical about these systems these systems are just people and their
- opinions of people if you learned yals MX plus b then you have learned the building blocks of every machine
- Learning System all right so if there is some weird outcome that comes out when you're on Facebook or Twitter or you
- notice some weirdness on your uh when you you put out an application for a loan trust your gut all right it is weird and
- say something because again the only way to improve these things as well is uh uh
- for them to update their model and it's you you are right that something's wrong
- all right it is not magic is not your fault is them uh don't accept magic for researchers if you are interested in
- working on these topics and in this space go to jointhe trap.com and come hang out with us uh we meet online once
- a week over Zoom Wednesdays at 1 pm uh and then finally for students this is going to be a weird one uh because in
- some ways this rise of uh generative Ai and writing and art uh makes people believe that we are like this close to
- uh having AI that can write beautiful books or make beautiful art uh and I'm going to challenge that all right we
- they also told us we were this close to having fully self-driving cars uh within
- a year 10 years ago and they said this every year and it's not happening anytime soon all right I'm going to say
- the same thing is the case for art and writing and uh creativity I think that they will be a
- premium uh on people that can actually Express themselves clearly and accurately and honestly and so if you
- are currently a student uh anywhere if you're in a b school if you're in a college if you're in high school and
- you're listening to this I don't know why you're listening to a Wharton podcast in high school but maybe you
- know you're one of those kids uh get really into building your uh toolbox of creativity right get really into uh
- creative writing right get really into making more art because there will be a value on the ACT actual human element
- that comes out of this if uh what everyone else is doing is just plugging in some chatbot that gives you uh
- plausible sentences that aren't that good if you can write better sentences you'll
- win thank you and and I'm just going to tell you project we actually do have knowledge at war in high school so your
- high school learners are going to learn a lot from what you all share today as much as your you know fully experienced
- senior leaders will um as well uh so so broject and Kem this has been fantastic I feel so much more knowledgeable as
- somebody who believes that I know a lot about a lot of things I I know that I don't know a lot about this topic and so
- the past amount of time 45 minutes that we've been chatting have been amazing and just I think helping me to feel more
- empowered um as a researcher a consumer and a worker around sort of what exactly
- is happening and and how I need to pay attention um to to what is being shared um so I want to thank you uh so much for
- sharing your insights and your expertise with us uh and to all the uh leading diversity at work podcast listeners we
- truly appreciate you for being here uh so that's all for now thanks to our audience for joining us and listening to
- this episode of the knowledge at Warton leading diversity atw work podcast series goodbye for
- now for more insight from knowledge at Wharton please visit knowledge. won. [Music]
- EDU
Brief Summary
Experts discuss the importance of responsible AI, representation, and transparency in technology's impact on society.
Key points
- Introduction to Responsible AI. Stephanie Cy welcomes guests to discuss responsible AI in the workplace.
- Dr. Broer Turner's Research. Dr. Turner shares insights on race and technology's impact on consumer decisions.
- Dr. Kareem Gan's AI Governance. Dr. Gan discusses his role in AI governance and the importance of ethical practices.
- The Importance of Representation. The conversation highlights the need for diverse representation in AI development.
- Demystifying AI Models. Understanding AI isn't magic; it's about basic math principles like y = MX + b.
- The Hallucination Problem. AI can misrepresent facts, leading to dangerous consequences in decision-making.
Episode highlights
- Introduction to Responsible AIStephanie Cy welcomes guests to discuss responsible AI in the workplace.0:23Nov 9, 2023
- Dr. Broer Turner's ResearchDr. Turner shares insights on race and technology's impact on consumer decisions.1:07Nov 9, 2023
- Dr. Kareem Gan's AI GovernanceDr. Gan discusses his role in AI governance and the importance of ethical practices.1:16Nov 9, 2023
- The Importance of RepresentationThe conversation highlights the need for diverse representation in AI development.21:01Nov 9, 2023
- Demystifying AI ModelsUnderstanding AI isn't magic; it's about basic math principles like y = MX + b.“If you learned y equals MX plus b, you can understand these systems.”25:39Nov 9, 2023
- The Hallucination ProblemAI can misrepresent facts, leading to dangerous consequences in decision-making.“These LLMs can misrepresent facts and tell plausible but misleading stories.”29:11Nov 9, 2023
- Class Action LawsuitA lawsuit against OpenAI raises serious privacy concerns regarding data usage.“People don’t understand how their data is being accessed and utilized without permission.”31:27Nov 9, 2023
- The Illusion of AI CreativityGenerative AI may not be as close to creating art as we think. 'They told us we were this close to self-driving cars... and it’s not happening anytime soon.'“They also told us we were this close to having fully self-driving cars.”44:01Nov 9, 2023





