Baseball, Bias and Decision-Making

Knowledge at Wharton

Ask about Knowledge at Wharton

Episode Description

Wharton's Etan Green finds that umpires adjust their calls based on what information they think they know when a play is in motion. More: http://knlg.net/2ekcW3z πŸ“‚ Management 🏷️ Research

Read the full transcriptHide the full transcript
  1. today we're speaking with Wharton professor 8 on green about his research on expert decision-making and
  2. specifically umpire calls in baseball so it sound happy with us thanks for joining us yeah my pleasure so can you
  3. give us an overview of your research yeah so generally what I'm interested in is decision-making by experts expertise
  4. particularly in realms for which there are predictions available from machines and machine based models algorithms or
  5. in the case of umpires in baseball data from stereoscopic cameras behind home plate of every major league ballpark
  6. that we use to benchmark the calls that umpires make and so this is a great setting for studying decision-making by
  7. experts because we have experts who are supposed to abide by a very specific decision rule so the pitcher throws a
  8. pitch if the pitch is in this imaginary box the official strike zone defined by the width of home plate on the floor and
  9. the batter stance then the umpires supposed to call a strike otherwise he's supposed to call a ball and so what we
  10. do is we use these data from the stereoscopic cameras that take a sequence of images of every pitch from
  11. its release from the pitchers hand until it crosses the region above home plate to basically observe to what extent the
  12. umpire abides by this decision rule to make his calls based solely on the location of the pitch and so I think the
  13. most interesting thing that comes out of the data is basically this deviation from that benchmark and very systematic
  14. way and so there's something in baseball called the count the count is keeps track of the sequence of pitches between
  15. a pitcher and a batter over the course of an at-bat if the count reaches for balls that's good for the batter he
  16. walks if it reaches three strikes that's good for the pitch or the batter strikes
  17. out and so what you see is that instead of the umpire just using the location of
  18. the pitch to make as calls pitches at the same location are sometimes called balls or sometimes called strikes
  19. depending on the count and in particular the strike zone expands dramatically when the count favor is the batter and
  20. so when account favors the batter the umpire responds by favoring the pitcher and vice versa when account favors the
  21. pitcher the Empire responds by favoring the batter and it's particularly extreme
  22. so basically you can think about a pitch that crosses say the top boundary of the
  23. official strike zone so this pitch and what I'll call baseline count the count at the beginning of the advance euro
  24. strikes this an umpire calls a strike fifty percent of the time it calls a ball fifty
  25. percent of the time you can think of as being a different between a ball and a strike but when the counts say has three
  26. balls and 0 strikes when it's strongly favored is the batter well then this pitch is almost always called a strike
  27. and the reverse is true in the opposite countless your balls and two strikes the
  28. same pitch at the same location is almost always called a ball so so why does this happen so they're potentially
  29. a number of stories that I can explain this result let me tell you about a particularly interesting and
  30. counterintuitive one and the argument here is that what the pitcher is do or what the umpire is doing is he's trading
  31. off accuracy for bias or rather he's trading off bias for accuracy he's being purposefully biased consciously or
  32. unconsciously so he's varying the strike zone that he enforces with the count he's not making his decisions based
  33. solely on the location of the pitch but the argument is this actually helps them
  34. make more accurate calls and why is this the case well imagine yourself as an umpire you're squatted behind the
  35. catcher you're looking out over his head towards the pitcher the pitcher winds up
  36. he throws a 90 plus mile an hour pitch it's there in an instant it has some lateral movements and vertical movement
  37. you have to decide whether this pitch is inside or outside some imaginary box it's an incredibly difficult problem and
  38. if you relied only on your observation of the location of the pitch you'd probably make mistakes on a regular
  39. basis frequently when the pitch is closed it'd be hard to say whether it was just inside the strike center just
  40. outside the strike zone but fortunately for you you have other information at your disposal you have expectations that
  41. you've built up over many years of being a professional umpire expectations about
  42. where the pitcher is going to throw in a certain count and whether the batter is
  43. going to swing and so for instance you might reasonably expect that when the count has three balls and 0 strikes that
  44. is when favor is the batter that the pitcher is going to try to throw a strike and so if the pitch is close and
  45. you're unsure whether it was just inside the strike zone or just outside the strike zone you may err on the side of
  46. calling a strike the pitch that you expect now think about what happens in an 02 count so in this count you expect
  47. that the batter is going to swing it anything close because if he doesn't he runs the chance
  48. of striking out whereas he can prolong liat bad if he fails the pitching off for instance and so imagine you see a
  49. pitch that appears close to you but the batter chooses not to swing how can you rationalize that decision well you can
  50. rationalize it by saying that he observes something that you didn't that his vantage point was such that he
  51. believed the pitch to be a ball and so you might err on the side of calling a ball and so this Bayesian updating this
  52. basically rational way of processing other information that you have creates this trade-off between bias and accuracy
  53. it helps the umpires become more accurate at the cost of having them systematically change the strike zone
  54. that they enforce with this variable the count that has nothing to do with their
  55. directive so what would you say a business practitioner should take away from your research yeah so I think what
  56. umpires are doing is they're statistically discriminating so they have a directive to make their calls
  57. based solely on the location of the pitch but that's very difficult to do it's very hard to observe the exact
  58. location every time and so what they do instead is they say well this other informations other information is
  59. correlated with the location in the pitch it could help me on average make more accurate calls and so as I said
  60. before they basically trade-off bias for accuracy it's a statistical discrimination at least opportunities
  61. for statistical discrimination are just they're everywhere and they're everywhere in the workplace in
  62. particularly in you know the hiring process so for instance you know when we hire we have a benchmark that sounds
  63. very similar to the umpires directive we want to hire the best person person is going to do the best of the job there's
  64. going to be the best fit but it's hard in the interview process looking at a CV or even interview i'm a person often to
  65. decide who is the best or how good is this person how good is this person going to be in the job and so we may
  66. rely on other factors factors that are either implicitly or explicitly banned that we shouldn't be using perhaps but
  67. factors that we believe perhaps rightly as in the case of the umpires or even erroneously to give us information about
  68. this person's fit and so if we're right we're going to get a little more accuracy but it's going to come look at
  69. the cost of bias it's going to come at the cost of systematic being able to systematically
  70. predict who we hire based on factors that have nothing to do at least directly with the dimension that we're
  71. trying to hire along so what are you going to look at next going to stay stay in baseball or look elsewhere for
  72. research yeah so I mean it's a baseball is an opportunity to use machine based models these cameras to say how good of
  73. a job umpires are doing but I think there are a lot of interesting cases in which the decisions that individuals
  74. make that experts make can be informed by algorithms by machine based predictions and so one of the things
  75. that I'm interested in particular now is making predictions about the election so
  76. it's particularly timely I think a lot of us are interested in the probability that Hillary Clinton is going to wait on
  77. the probability that Donald Trump will be our next president and so one place you may go to get information about this
  78. you may go to 538 nate silver's website and one thing that nate silver is doing this election season that he hasn't done
  79. in previous election seasons is he's providing multiple models so in the past he told you the probability that Hillary
  80. Clinton would win is seventy-seven percent now he's telling you if you believe this model it's seventy two
  81. percent if you believe this model it's eighty-four percent and sometimes there's really quite a deviation between
  82. these two models well what are these two models well basically they're making different assumptions about the world
  83. and your decision as to which model you listen to is really a decision about what what you believe the data
  84. generating process to be what do you believe the world to look like and in particular there's one model that says
  85. we should only listen to the pulse we should only listen to what people are saying right now and there's another
  86. model that says actually there are lots of predictors economic indicators for instance like that historically have
  87. been very predictive of election outcomes and so we should listen to those as well as your decision about
  88. which model to listen to her how to balance these two pieces of information basically comes down to your belief
  89. about whether this election season is totally different from the past in which case you should only listen to the polls
  90. or if you believe that this is just another draw and some stable distribution that is similar to
  91. everything else that's come before and so generally what I'm interested in is how can we frame questions what types
  92. of information can give people to make them think that this moment the present is just like the past and the past is a
  93. good predictor of the present and what types of information how can we frame questions to get people to think
  94. actually know the process is not stationary at all this moment is unique in time great that's fascinating thanks
  95. very much for joining us a time yeah my pleasure thank you you

Brief Summary

Wharton professor discusses umpire decision-making biases in baseball and their implications for accuracy and hiring practices.

Key points

  • The Impact of the Count. How the count influences umpire decisions, affecting the strike zone.
  • Decision-Making in Baseball. Exploring how umpires make calls based on pitch location and game context.
  • Predicting Elections with Models. Using machine-based models to forecast election outcomes and their implications.

Tension

30/ 100Low tension

Related episodes