The 2025 NFL Season Through Data: DVOA, Analytics, Awards, and Predictions
Ask about Knowledge at Wharton
Episode Description
ABOUT THE EPISODE Aaron Schatz, chief analytics officer at FTN Fantasy and founder of Football Outsiders, joins Eric Bradlow to explore how DVOA and play-by-play analytics challenge conventional narratives about the 2025 NFL season, conference strength, playoff probabilities, and the growing influence of data in awards voting. Plus, Eric walks through real-world examples of using generative AI to forecast NBA win totals, college football playoff probabilities, and NFL Super Bowl odds, highlighting how modern models apply Bayesian reasoning, betting markets, and simulation-based analytics.
ABOUT THE PODCAST Sports is a game of numbers. Wharton experts Eric Bradlow, Shane Jensen, Cade Massey, and Adi Wyner team up to tackle the world of sports, from current events to longstanding issues such as: What sports streaks are the most impressive? How do you rank the best players? Can athletes be compared across sports? Moneyball Highlights explains how decision-makers in the game can avoid the common mistakes and embrace the data. Episodes are recorded at the Wharton School.
#NFLAnalytics #FootballAnalytics #SportsData #AIInSports #SportsPodcast #NFLAnalytics ----- The Wharton School of the University of Pennsylvania is committed to sharing its intellectual capital through Knowledge at Wharton, the school’s online business analysis journal.
Launched in 1999, Knowledge at Wharton offers free access to:
- Articles, podcasts, and videos highlighting Wharton faculty research and analysis of current business trends - Interviews with book authors from Wharton School Press - In-depth series of curated content like Wharton Executive Education’s Nano Tools for Leaders - A searchable database of more than 10,000 articles covering all aspects of business - A weekly newsletter that delivers Knowledge at Wharton insights directly to your inbox
Visit the Knowledge at Wharton homepage: https://whr.tn/3qUpSBE Subscribe to the Knowledge at Wharton YouTube channel: https://whr.tn/3DEAgAU
Read the full transcriptHide the full transcript
- Welcome, welcome to Wharton Moneyball, the podcast edition here on the Wharton podcast network. This is Eric Bradlo,
- professor of marketing, statistics, and data science here at the Wharton School.
- As everyone who's listened to us for the last 11 plus years knows, some combination of myself, Kate Massie, Audi
- Winer, Shane Jensen, sports never stops. Data science and statistics never stops.
- So, we're here every week on Wharton Moneyball. I've always said that one of my greatest honors of doing this show
- again for the last 11 plus years is having people that are living, breathing analytics applied to sports all day
- long. I just do it as a part-time job. I got the regular job doing my own research, teaching other stuff. And
- certainly today is no exception. Um, we're joined today by the I don't even know how many times Aaron's been on the
- air. I'm going to say 10 plus. It's got to be more than that because whatever. We're joined today by Aaron Shatz. is
- chief analytics officer of FTN Fantasy. A lot of people also know his work on ESPN. A lot of our listeners certainly
- know him as the founder of uh Football Outsiders. A lot of people also know him as the creator of Advanced Statistics,
- including DVOA. And something I just learned today, and we're going to talk about this later on, is he's one of the
- members of the Associated Press that votes for the AllP Pro team and the regular season awards. So Aaron, uh
- welcome back to uh the Wharton Podcast Network and Wharton Moneyball. Hey, thank you for having me back. Always
- good to be here, man. >> Well, this is obviously a a great NFL season. It's The reason I'm going to say
- that and I'd love your a kind of analytical look at it is, you know, I can't tell you who's going to go I can't
- even tell you in my view who's going to go to the Super Bowl. I think this is just a year with more uncertainty. You
- know, it's like I think Shane read the stat last week. This is like the first time in 20 years there's been no Patrick
- Mahomes or Tom Brady in the Super Bowl. And so I'll just ask you, let's just start at the highest level. How do you
- see the 2025 NFL season right now? Who are the true contenders? And if you had to list, you know, I always like to give
- the I'll call it the paro curve. like how many teams would I have would you have to give me to make it like an even
- bet between making the playoffs and between going to the Super Bowl and not? How do you view the AFC and the NFC
- right now? >> Well, the first thing I'll say is it's a very topsyturvy year. A lot of weird
- stuff has happened. There are a lot of teams where their record doesn't really match what the underlying statistics
- suggest about how good they are, right? like Carolina, New England, Chicago don't seem quite as good as their
- records. Detroit, Kansas City, Indianapolis have played better than their records. Uh, some of these are
- teams you would have expected in the preseason, some of them are not. Uh, I used my current ratings, current
- ratings, and ran a simulation of the whole season. And in that simulation, the amount of times that Baltimore,
- Detroit, Cincinnati, and Kansas City all missed the playoffs was 1.1%. >> Wow. >> And that's not using preseason
- projections. That's using how good the teams have actually been this year. >> So, we always like to ask that question.
- So, now the question I I I'll ask you, which is probably not a surprising question, is it could mean one of a
- couple things. It could mean the thing that we all have in our simulations, which is not enough uncertainty. And if
- the answer is there isn't enough, where is it coming from? Another possibility is um 1.1% of events happen 1.1% of the
- time. So, you know, let's not let's not go overboard that you said 1 in 20 million. So, you didn't say that. The
- third, of course, is your point estimates of the team's strengths could also be off. It's not an uncertainty
- issue. It's kind of >> It's an imperfect science. Yeah. >> Yeah. It's an imperfect science. So
- which of those three explanations or all a combination of all three do you think
- it is? >> I think it's a combination of all three. But the big thing is just that events
- get ordered in different ways. And so teams play one game where they're like outstanding the whole game and then
- another game where they lose by three points. And this year is filled with teams where they've just been either
- really really good in one-score games or really really bad in one-score games. The other thing that's going on is the
- is we'll talk about this in a little bit. I think schedules being very extreme because of the way certain divisions are
- good and bad this year. There are certain teams that have very extremely easy or extremely hard schedules. Um, as
- far as uh the NFC is the better conference, that's the first thing. Uh, I my numbers differ from market. Is the
- NFC the better conference on let's say whether it's average or median or do you think the top end because those are
- different statements than saying the top end of the NFC is better than the top end of the AFC or both.
- I would say both are true, but in particular the top end of the NFC. Um, the market I have Seattle and the
- Rams much farther ahead of the rest of the league than a lot of other people who do advanced analytics.
- Uh, the reason for that is almost everybody else who does advanced analytics uses some form of EPA, right?
- Expected points added with some sort of adjustment. And I don't I use my own stat which is
- DVOA. And because of some of the ways that DVOA works and EPA works, Seattle and
- the Rams come out a lot higher in DVOA than they do in EPA. One, some of it has to do the increasy intricacies of, you
- know, certain amount of yards and a certain down and distance. And some of it is easy to explain stuff like I
- downweight turnovers because they're less predictive. And Seattle has a lot of turnovers. So if you do EPA,
- Seattle's offense is not going to come out as good as they do in DVOA because I'm downweing the turnovers because less
- predictive. So because of that, I have Seattle and the Rams at almost a 5050 shot to win the Super.
- >> I just want to be clear them com just to be clear. Are you saying if you take the
- Rams and Seattle, you have the Rams in Seattle, I have the rest of the league and that's a fair bet. Is that what
- you're saying? >> My numbers would say yes. It's a surp It's surprising and I don't know whether
- maybe I have them a little too high and no one else would have this, but yeah. I
- mean, everybody has right now has Seattle and the Rams as their top two. >> So, just give me an or this is what I
- was going to ask you. whether one's EPA based or DVO based. Can you give me a sense? Or even if I went on to ESPN,
- FBI, whatever it was, how much probability would they give combined? Are you at 50 and they're at 20 or
- you're at 50 and they're at like 35? I I always like to get a sense of the order
- of magnitude of these discrepies differences. I can look up ESPN while we're here and tell you that for Seattle
- and the Rams, they have them winning the Super Bowl a combined 27% of the time. >> Wow. So, you're almost twice as high.
- So, that's we would consider >> past everybody when it comes to those two teams. Yeah.
- >> Yeah. So, I just just for myself maybe I'm thinking about this wrong. I'm pretty sure most of our listeners and
- this is the great thing about Wharton Moneyball is saying, "Wow, would Aaron really give me even money despite like
- someone we both agree someone from the AFC is making the Super Bowl. I don't know who it is, but someone is." And we
- could argue they have almost a 50% chance to win that game. So, in your view, well, let me ask a question. So,
- there's this is the wonderful thing about winning the Super Bowl. We agree two things have to happen for
- you to win the Super Bowl. If you're Rams or Seattle, first you have to win the NFC. Then you have to go to the
- Super Bowl and win it. Which of those two numbers is really, really high? It sounds like it might be both. Like
- here's something that could yield 50%. >> The combined probability is 7070. That
- would get you to 50%. Like they're the one of the two of them has a 70% chance of going in total. And if they go,
- they're a twothirds favorite. Roughly that get you to 50. >> So my ratings are based on the idea of,
- you know, how efficient is a team compared to average. >> Seattle and the Rams are two of the 10
- best teams since 1978. >> Measure. I know I have them way ahead of everybody else. I know exactly how
- you're measuring that. It's based on a play-by-play breakdown and the success on each play is compared to a baseline
- that's adjusted for situation and opponent. And one thing is that I include special
- teams and a lot of people who do advanced stats don't include special teams because special teams is less
- predictive than offense and defense. But it's not not predictive and I think Seattle and Rams game that the fact that
- Seattle has really good special teams. So just to be clear, I compute this >> I compute this metric for every play and
- then I sum it over all the plays. >> Yeah. Yes. >> Okay. So Okay, I see. I just wanted to
- understand the Okay. Wow. >> So just to give you an idea of where the numbers are, Seattle and the Rams are at
- like 42 43%. The next highest team that will make the playoffs is like Jacksonville right now, which is
- at like 20%. Like that's the gap I have between the Rams and Seattle and everybody else. I
- don't know if I would really do the Rams and Seattle versus the Field to win the
- whole thing. >> If you gave me the 49ers though, I might do it. If you gave me the NFC West
- versus everyone else. >> You almost made the whole MS NFC West. >> I might I might do that.
- >> Huh. That's very very very interesting. Um, what aspect? Well, first of all, could you just tell us since you must
- since you're making a statement that they're in the top 10 since 1978. Can you give maybe I I don't know if you
- have this handy, can you tell me since I'm a football historian just like you are, can you tell me who else is in that
- list or do you have that list handy? >> The top three through this is through 15
- games. The top three are the 2007 Patriots. >> They were pretty damn good. The 1991
- Redskins. >> They were good. >> The 1985 Bear. >> I knew the 85 Bears were going to be in
- there somewhere. Wow. Yeah. So, >> you also get some teams that didn't make it all the way like the 2010 Patriots
- and the 1983 Redskins who lost to the Raiders in the Super Bowl and the 9549ers who got upset by the Panthers in
- the divisional round. And then for some reason the last three years I have had a
- ton of teams come out as historically good in a way that hadn't happened for a dozen years before them. The 2023
- Ravens, the 2023 49ers, the 2024 Ravens, like come out as really historically good. And I don't know why because I
- haven't changed what I'm doing. And yet there weren't any teams that were anywhere near that high from 2011 to
- 2022 and then all of a sudden in the last couple of years there have been these teams that are ridiculously high.
- H wow. Um so because if you had asked most people I think I they would say that the Rams and the Seahawks are
- better appear to be better than others this year but not I'll call them. Well, I don't think anybody would put them
- among the historically great teams. >> And it's it's it's a little weird because of the fact that I've had this
- going for a couple of years and those teams have not won the Super Bowl. But it's not like my ratings always come out
- with modern teams that high because from 2011 to 2022, there were no teams that high.
- >> It's just the last three years. >> So, I'm going to ask you a question that every listener here on Wharton
- Moneyball. And again, I'm I'm joined here by Aaron Shats. Aaron is the chief analytics officer at FTN Fantasy.
- Everybody also knows him from his original work at Football Outsiders, his work at DBA,
- his work with DVOA. So, let me just ask you, I'm always going to ask these kind of questions because I know the answer
- is no to each of them, but what the hell? I'm going to answer them anyway. Rams and Seahawks.
- Um, if I'm correct, the quarterback for the Seahawks is Sam Darnold, right? >> Yes.
- So you have no concerns? >> No, I have concerns. >> Okay. So, so let me ask you a different
- question. Is that brought mathematically into your model anywhere that it's Sam Darnold's?
- >> No. And it's something I want to look at doing in the future, which is should I
- have an additional career history uh variable for quarterback that goes past this year all the way back multiple
- years of whether you trust a certain quarterback compared to other quarter. It's clear that Vegas does.
- >> Absolutely. >> Because there's no other reason why they have the Bills as the favorite to come
- out of the AFC. The Bills have not been by any metric the best team in the AFC this year. The only reason to have them
- as the favorite to come out of the is Josh Allen. Is the idea that you have more historical trust in Josh Allen than
- you do in Drake May or Bo Nicks or Trevor Lauren? >> So, which team in the AFC has been the
- best team based on your Jacksonville? Is it Jacksonville? Uh, if you look over the course of the entire season, it's
- Indianapolis. If you weight it towards more recent games, it's Jackson. Where do you have I have debate with my
- kids about this all the time who are big analytics guys as well. Isn't I mean, they they believe
- Houston's a fraud. Where do you guys have Where do you have Houston who I think's won at least eight straight if
- I've got this right? >> We have them uh fifth. >> We have fifth in the NFL. a fifth in the
- NFL. They are number two on defense, but they're only 21st on offense. >> And there is that thing where offense is
- a little bit more predictive than defense. And that is that is true, right? So, you're like, okay, maybe
- they're not quite that good going forward, but that defense is really, really good and the offense has played
- better in recent week. >> Um, and okay, very interesting. So, let me ask you another question. So, you've
- met you made a comment in the notes here about the Patriots and their schedule. Um, could you talk more not only about
- their schedule, but maybe this is why one of the reasons why we have advanced analytics because you can't just look at
- one loss record and so could you talk a little bit more about which teams you see where and you mentioned it a little
- bit when you're in your opening remarks about who's one lost record or how about
- the following. Let's say I did a by a graph of w number of wins on the x- axis and whether it's Aaron Shatz's measure
- of team strength on the y ais. Which teams would have the largest deviation from that 45 degree line?
- >> And a lot of these teams, anybody doing advanced analytics has the dev deviation? the Panthers,
- the Bears, the Patriots, and on the other side, the Lions, the Chiefs, and the Colts. Although obviously the Colts
- and Chiefs have a different quarterback now than they did for most of the season, and that affects how you would
- predict things going forward. But as far as how they've actually played during the year, the Colts and the Chiefs have
- been better than their record. And the Lions have The Lions are going to be one of the top DVOA teams ever to miss the
- playoff. >> Wow. So, I was going to ask you that also. Do you ever as a I'll call it a
- face validity check. Let's imagine that every year DVOA based strength um has lots and lots of outliers or deviations
- where the DVA DVOA based strength does not match let's call it the win loss. Do you ever start to say maybe that's a
- deficiency in DBOA because there's of course there's randomness within a given year but like across a long period of
- time eventually you know if I look in totality it should be properly calibrated.
- >> Yeah. I mean there's a lot of variance in sports and there's a lot of variance
- in the NFL. It's never going to be properly calibrated but no I'm always looking to improve it. I'm always
- looking to improve the predictive ability. I'm always looking to improve, but a lot of these uh it's interesting
- for a lot of the teams that I just described, the year-to-year DVOA is more consistent
- than their year-to-year win loss records. >> Well, that's and that's what you would
- hope. >> Like Detroit, Indianapolis is an exception, but Detroit was good last year. Kansas City was good last year.
- Carolina was not good last year. So, the idea that Carolina is not as good as its
- record makes some sense if you think about who they were last year. And the idea that the Chiefs are better than
- their record makes sense if you think about who they were last year, >> right? Is it Am I correct in saying this
- is just from my I haven't looked at it analytically, but I would just give an opinion that Well, I think I know, maybe
- I don't know. I was going to say is Well, it must be from your answer. The strongest conference division, sorry, in
- football has to be the NFC West, right? Given you just said the Rams and Seattle
- are historically strong. You said you would take the 49ers. I think each of them has at least 11 wins at this point.
- Um, and so that's the strongest and the NFC South is the weakest. >> Yeah. Oh, yeah. The AFC South was
- shockingly strong this year. The AFC South, >> that was going to be my second guess.
- >> Really surprised. Indie Jacksonville really surprised. Um, Houston, I think people thought was going to be, you
- know, just what they are, which is a great defense with no offensive line. Um, but uh right now in my ratings for
- the whole season, so not weighted to recent game. Uh, the Colts, the Texans, and the Jaguars go four, five, six. But
- with the NFC West, those two teams are so far ahead and the 49ers are also good that Yeah. the NFC West is the best
- division. And the whole NFC South is in the bottom half. >> Wow. Yeah, because I you may remember
- I'm a Buccaneers fan through my cousin, etc. Uh what's happened the last three weeks to the Buccaneers has been
- embarrassing. Like I mean >> they're the best team in the division. Why can't they win games? It's very
- frustrating. >> Yeah. I mean they are the best team. That's clear. And now they have all
- their players back. So there's no excuse for that. And they've just they lost at
- home to the let me think if I got this right. I was at the game with the Saints so I know they lost that one and then
- they lost to the Falcons at home and then they just lost to the Panthers on the road. So they just lost to, you
- know, three teams in their division and they all stink. >> Yeah, it's I can't imagine. It's very
- frustrating. Yes. >> And of course they still control their own destiny which is also the crazy
- part. I mean they may end up going eight and nine and winning the division because the accord to you that's likely
- to happen because uh the Panthers play Seattle this week and that's a big game for Seattle. The Bucks play the
- Dolphins. Well, the the Bucks probably >> starting a rookie quarterback. They're gonna be
- >> they should probably beat him. The Bucks would have to win the division at nine
- and eight because somebody's going to have to win the last game of the season between those two teams. But that may be
- a historically weak division winner. And whoever the five seed is um might be happy they're the five seed.
- >> I mean, there are other divisions, the 2020 NFC East, the 2004 NFC West. There
- have been other divisions that are weaker than this, but this is pretty weak. >> Now, the next thing in the notes that
- you helped us uh create here, which I I really appreciate that, is um I assume when I asked you the question about DVOA
- and your it being based on playbyplay um and then you kind of add it up. Um you put in the notes here the s the
- search for historical playby-play data and the upcoming edition edition of the 1977 season. So, is it fair for me to
- say that what you and I completely agree with this by the way that you've built most of your analytical career for the
- NFL on having this playby-play data as opposed to just aggregated statistics and that's why
- >> Yeah, please go ahead. >> When I got started, there were people who were like, "Yeah, we did analysis
- based on playbyplay and there was nothing based on playbyplay in it." And I was like, "Let's do something based on
- playbyplay." And then I started collecting old years and a lot of it's been with the help of Jeremy Snyder who
- people know on social media as quirky research. >> Okay. >> And he has helped me and we have tracked
- down and we keep going back and back and back. So we've got back to 1978 now. We
- have transcribed things off of video. We have found old game books. Um and we have enough to do 1977 this off season.
- And it is no longer just mine. I have shared it with Pro Football Reference. Pro Football Reference now has all the
- playbyplay going back to 1978 on their site, searchable with all the playby-play logs for all the games. I'm
- very proud of doing that. It's been I think it's a really huge addition to the ability to study historic football
- historically. And it's great. It's so much fun for me when I'm like, "Oh, yeah. The best teams to miss the
- playoffs, the 1979 Redskins." Like, I can go back that far and say they're one of the best teams that didn't make the
- playoffs. You know, let me ask you a question. This might seem naive, but I'm sure a lot of our listeners here on
- Morton Moneyball are asking might be thinking the same. What do you exactly mean when you say playbyplay data?
- Because one could dream. Let's let's forget what you have. I'm going to answer what my dream might be and then
- you're going to tell me the data that one actually would have conceptually. You could have video motion data of
- every player and their XY coordinates on every single play and all of and I know
- that's not the data you have but so what do you mean by playbyplay data and how do you compute kind of like an expected
- performance given that data >> for a given play? >> All my years are normalized so that
- average is zero. So the same goes all the way back to 78. Every year is normalized so that I'm comparing teams
- to just that year. And what I have is basically everything that was in the playbyplay up to 2004.
- So like is it a pass or a run? Who was the player? Who was the receiver? What was the down
- and distance? What was uh the time is not in there for a lot of old years until the last two minutes of each half,
- but we have the time to start each drive. And then the further back you go, the more spotty it gets as far as like
- the direction of runs or tackles or passes defense, but we have like who had sacks and who has interceptions and all
- that data. Uh but we have targets, right? Like no one ever had targets for the 80s before before we collected that.
- So, not just receptions, but targets like is in this data. Um, we don't have um yards after the catch and air yards.
- That doesn't start till 2005. >> We don't officially have whether a quarterback run is a scramble, but I've
- gone back to a lot of video and answered whether a lot of those are. Uh, we don't
- have a quarterback knockdowns. That didn't start till 2006, I don't think. And we don't have all the kind of
- charting data that you now get from FTN or SIS or PFF. Not we don't have any of that going back before 2005. But
- everything that people had from 99 to 2004, we basically now have for 78 to 98. >> H I I can imagine that's ext and then
- can you give us a sense of how just you know maybe give me an example of how you
- score a play. So, how is a play score? >> Well, like you know, I give it a certain
- number of success points based on the yards and the down and distance, right? Success in my percentages is you have to
- get 45% of needed yards on first down, 60% on second down, and 100% on third or fourth down, but there's like partial
- numbers. It's it's not it's not it doesn't go zero and one. It goes like zero and 2 and 0.5 and 7, etc. And then
- you get extra bonus for the extra yardage. So a play will come out with something like 2.3 success points if
- it's like a 10 yard pass on first and 10 or something. And then I compare that to
- what's the average for first and 10 passes in that area of the field. >> Yeah. Okay. So that's what I meant. I
- was also asking what do you condition on when you compare a play? So it would be
- similar >> distance area of the field uh and then adjust for the score, right? You know,
- obviously it's easier to gain yardage if you're losing by 40 late in the game. >> So, this is actually a pretty opponent
- adjustment based on the quality of the opponent. So if I just look at this as a big contingency table, this is a very
- highdimensional contingency table because I mean there's not that many downs so that's not high dimension but
- distance maybe have them uh blocked or you know bundled into ranges or something and then there's maybe there's
- score difference and then there's maybe >> what I try to do is smooth the curves.
- >> Okay. So there is a smoothing that's done but I didn't have like I I don't think it makes sense to say that eight
- yard gains are more common than seven yard gains. It's more likely that our sample size isn't big enough. It what
- what's logical is to say that you know you gain one more than you gain two you gain two more than you gain three you
- gain three you know etc. like it's it or I guess four is the average and then the
- curve goes like this, right? But like um you know for judging between first and 20 and first and 25, you sort of group
- that together because there aren't enough actual like >> first and 21s that are out. You know,
- there aren't that many first and sevens that aren't in the red zone. >> Makes a lot of sense. So let me get to
- the last topic for today that I want to talk to you about which is as I mentioned in when I introduced you to
- start with you're one of the I I imagine a tremendous honor for you and one welldeserved as a member of the
- Associated Press that votes uh for both the regular season award and the allp pro team. Um one of the topics you
- brought up in your notes was the growing role of analytics in awards voting. So that could mean many things. It could
- mean um the number of people that you and I out of those 50 would consider as analytics oriented is going up. That's
- one way to measure it. Another way to measure it is um you have knowledge of the analytics that people are using.
- Like here are the stats that people use or those are provided in some way. What did you mean when you said the growing
- role of analytics in awards voting? >> More the first than the second, but they're definitely both true. I was the
- first analytics person to be on the panel. uh starting in 2021, but they've done a
- lot of changing of the panel over the last few years. So now you have Mina Kines
- >> and you have Sam Monson who used to be with PFF and you have Doug Ferrar who's
- more of a film guy but understands analytics used to write for me at Football Outsider.
- The other thing though is if you think about it, people who are not necessarily analytics people have access to more
- analytics now than they used to. Think about a guy like Dan Orlovski, right? Dan is a former quarterback. He
- is a film guy, but he works for ESPN and he's very conversant with all of the ESPN advanced stats that ESPN stats and
- info does. And I'm going to guess that when Dan puts his ballot together, he's going to use some of that stuff. Now,
- he's more probably going to use his eyes and his scouting, but he's going to use
- some of that stuff that didn't exist a few years ago, or at least there was nobody on the panel that would use any
- of that a few years ago. >> Right. >> So, I think even the people who are not necessarily analytics people are going
- to use more analytics than they would have used 10 years ago. >> Can you point I don't want to talk about
- this year because I know that's not only is it off limits, you shouldn't talk about it because you're a voter, but we
- can talk about the past. So can you talk about instances at least in your own mind that somebody was like wow if it
- wasn't for analytics that person never would have been voted all pro or never would have been an or maybe that's off
- limits too just like I was just not that your vote I'm not asking for your vote or anything like that
- >> I'm thinking because I have an answer when it comes to my vote but I'm trying
- I don't know about for the whole allpro team or the whole or specific awards that I think of anyone like that, but I
- can think of one where I was very different. I'm very famous among, you know, people who understand this stuff.
- When Lamar Jackson won MVP in 2023, he got 49 of the 50 votes and I was the exception.
- >> So, can you tell Well, now that you said that, can you tell us about that and
- what what did you vote? And >> I wrote a whole thing about how no matter what stat you looked at, he came
- in below Josh Allen and Dak Prescott. >> My advanced stats, ESPN's advanced stats, PFF grades, like SIS, anything
- you looked at, he came in below those guys. And I was like, I just can't vote for him for MVP when he's not number one
- and or two or two in any staff. And so I went with Josh Allen. Wow. Well, let me just say uh for those
- people that follow us on whether you want to call it X or Twitter, W Moneyball, um there's no doubt I'm going
- to be if we still call it tweeting, I'm going to be tweeting that out because that in itself is fascinating. It's also
- fascinating to me that since let's assume whether people use it or not as a separate issue, that had to have been
- sort of known by the voters. And so that that to me is very interesting. Let me ask I'm just asking you for speculation.
- If that were to happen today, 5 years from today, you agree the vote you're I mean we would both hope the vote
- wouldn't be 49 to1. >> Today it's still there's still narrative, right? Cuz Allen beat Jackson
- last year even though last year the opposite was true. Jackson had better numbers.
- >> Interesting. But in five years, I don't know if somebody whose numbers are as
- low as what Cam Newton's were in 2015 is going to be able to win the MVP. I don't
- know. I mean, you you'll have you'll certainly have places where the person who finishes third or fourth will beat
- the person who finishes first in a stat, especially because different metrics have different players number one. But
- five or six years from now, you may not have a situation where you have a guy who's 10th or 11th in stats, but he gets
- a narrative going, >> right? >> And his team wins a lot of games because of their defense and he gets a narrative
- going and then he wins the award. You may not have that anymore in a few years. Well, let me just my last question now
- is um can you tell me a little bit about what you're what you guys are working on
- that you can talk I'm not necessarily the nuts and bolts but what are you guys doing at FTN Fantasy that our listeners
- might be interested in like or you even personally like what's the big project you're working on now
- >> I mean big projectwise I don't have anything in particular except you know still always looking to improve things
- in the offseason you know improve predictive ability um you know the thing I'm really proud of at FTN is because we
- do all of charting in our FTN stats hub. If you're a subscriber to the site, ftnfantasy.com,
- you get DVOA for all kinds of charting uh categories. So, you can look at who's the best against man coverage or zone
- coverage or with two high safeties and which teams are the best with certain run concepts, outside zone, inside zone,
- power, man, etc. I love that stuff and I love doing stuff to figure out what's real and what is just really variable.
- And then working on the 1977 and then the other thing that FTN is doing that I have nothing to do with but
- we are doing and is cool is basketball charting. and we're going to have a bunch of basketball stuff coming out uh
- for people who enjoy basketball analy well Aaron I'd like to thank you for joining me here today on the podcast
- edition of Wharton Moneyball I've been talking to Aaron Shatz chief analytics officer FTN Fantasy uh apparently it's
- ftnfantasy.com right is the uh is the website you can also read >> called FTN Fantasy we do a lot more than
- just fantasy it's just we had to get it all under one ceiling and that's the ceiling we went
- Well, again, Aaron, thank you again for joining me today on Wharton Moneyball. >> Hey, thanks for having me, man.
- >> Welcome back to Wharton Moneyball here on the Wharton podcast network. This is
- Eric Bradlo, professor of marketing, statistics, and data science here at the Wharton School. Uh, I just got done and
- finished talking to Aaron Shatz of FTN Fantasy, chief analytics officer. Had a great discussion of lots of different
- topics. Um, and uh, some combination of myself, Audi Winer, Shane Jensen, and Cade Massie are here every week on
- Wharton Moneyball. Uh, since it's just me, um, and there's the a quote second half of the show, I did something
- interesting this week, which I hope all of our listeners uh, will want to will be interested in. I decided that since
- I'm Wharton's vice dean of AI and analytics, uh, why not use Generative AI to answer a bunch of questions that I
- have uh, I'd like to hear the answer to. So, I'm going to tell you what I did. I'm going to tell you the the exact
- prompt I typed in, and I'm going to tell you how, in my case, chat GPT. You could
- use your own large language model, but Wharton gives us a free enterprise access to chat GPT. Um, I'll tell you
- exactly what I said, what I asked it, what the prompt was, and what it found. And by the way, just for all of you that
- want to know, this was chat GPT 5.2, which is the newest version. And I also used it in thinking mode trying to get
- the if you'd like the highest quality answer for anybody that wants to reproduce what I did. So the first
- question I asked it was give me a 95% confidence interval for the number of wins for the Oklahoma City Thunder in
- the regular season. So let me say a couple things. I'm interested in that because they're 26 and three. They
- certainly have the potential to break the all-time record of 73-9 by the I think it was 2017 Golden State
- Warriors who lost the title as everyone remembers to LeBron in the finals. Um but I'm also interested into how much
- uncertainty it gives it. Okay. And so the first thing is what this is first something that differs from older
- versions of large language models. You may remember like when they first came out like we've only trained the date up
- until January the 1st of two years ago. Well, that's all gone now. Now, I think everybody knows that the first thing it
- said is as of December 23rd, their record is 26 and three. And they literally played last night. So, they
- know that they played last night. That's in the database. So, that's good. Um, it
- said it leaves 53 games in the regular season. Sounds good. Um, then it did something more spe more specific and
- impressive and it started thinking and it would come up on the screen what it was thinking about. It says, "Look,
- we're going to fit a beta binomial predictive model." So, let me just be clear to everybody what that means. So,
- you can treat the 29 games they're played as a binomial, a coin flipping model. 26 have come up head success.
- Three have come up failed. That means they've lost. That's the binomial part. And then we're going to put a prior
- distribution on that coin flip. And usually most people put a beta distribution. You can view that as an
- equivalent number of wins and losses. So for example, if I said it's a beta coin
- with a 1010, that would say my prior is 10 wins and 10 losses. I would add that to the 26 and three. I'd have 36 wins
- and 13 losses and that would be my predicted win probability for the rest of the season for the 53 games. So first
- I was impressed that it knew about basian inference that well. It knew that doing some sort of shrinkage, which a
- beta binomial is going to do because no one's going to predict that they're going to go, you know, 950 or whatever
- 26 and 3 is for the rest of the season. You're going to have to shrink it down to some prior and a beta binomial gives
- you a formal way to do it because it gives you an effective sample size. We know the observed sample size 29, but
- the prior is going to give us an effective sample size. It then said the 95% interval is from 64 wins to 78 wins.
- Now, first thing that surprised me was the 78 number. I mean, that number I mean, they're not winning 78 games.
- They're not going 52 and one the rest of the season, but I will say it also computed this simulation game by game,
- so it knew the schedule. 64, I think that's just way too low. I mean, you say, well, what about injuries? Well,
- the first like month of the season they played without their second best score, Jaden Jackson. So, they're not I mean,
- maybe um I just thought it was an interesting interval. I guess it's centered around 71 or 70 or 71 which
- seems quite plausible to me that they might do but either way. Um and then lastly it said for a reference a more
- optimistic assuming their current win percentage continues exactly it leads to an interval of 69 to 77 wins which means
- centered at 73 plus or minus 4. So you could view it as a standard deviation of two. You're going to take two standard
- errors of either side. So you're 73 plus or minus 2 * 2 which gives you an interval from 69 to 77. Either way I was
- impressed by its thinking. I was impressed by its use of the beta binomial. I could have queried it more
- to use this prior. It would have done that calculation. It was simulating game by game. Not bad. Not bad. So the next
- question, since we're living in the world of college football, I asked it, what is the probability that the four
- semifinal college football teams are Indiana, OSU, Georgia, and Oregon. So that's the 1,
- two, three, and the five seed. The reason I picked the five seed Oregon is that they're favored over Texas Tech.
- And I said, give me an exact probability estimate and a 95% confidence interval.
- So, first thing it said is um it recognized the matchup, so it told me the matchups. That's not that hard. Um
- it then used the draft kings money line. It could have used lots of different money lines, but that's fine. It then
- gave me the probability of each of the teams that I won winning. It then said, let's assume they're independent. It
- multiplied them together, and it comes up with 17.94%. It then computed a 95% confidence
- interval based on those proportions. Um, it then gave me a range of 16.72 to 19.68%.
- U, that seems a little bit narrow to me. Um, and then it said here, if you want me to compute spreadbased probability
- instead of money lines, it could do that type of computation. So again, this one was interesting because it rep
- recognized that I was looking for a compound event. you know the probability of four things happening because I asked
- for exactly all four of them. It used the it found the money line to compute those probabilities. It multiplied them
- together um and it gave me a confidence interval based on these uh proportions and and what the likely uncertainty is
- in those proportions. So I thought that was pretty impressive. And again it's more to me that it's impressive compared
- to historically. If I had asked it six months ago, a year ago, I don't think it would have given me as precise an
- answer. Um, and last but not least, I I asked it um let's see here. I had one other thing that I asked it for. Ah,
- okay. Um, I asked it the question I always like to ask, and this time I did it for the NFL. I asked it, how many
- teams do I need and what would those teams be that would give a 50% probability to win the Super Bowl? In other words,
- if I thought there were four teams that added up to 50%, I I'm asking it to tell
- me the number four and who those teams are. Now, what's fascinating about this is I didn't I hadn't even thought about
- this at the time. I just asked Aaron Shatz that question if you remember during the first half of the show and he
- said in his mind it's almost two teams the Rams and the Seahawks almost add up to 50% under his DVOA model. Um that's
- not what uh but he also said under using betting lines they would be about 27%. And that's what um chat GPT 5.2 thinking
- found. It said first it's converting bet 365 odds to an implied probability and it gave me the formula of the implied
- probability which is the way we always do it. P equals 100 over the plus number plus 100. So you take if it's plus 300
- 100 over 300 plus 100 is the implied probability. It normalizes across all 32 teams to take care of the vig the
- betting vig that's in there. And then what it says, it gave me the top six teams that give 49.65%.
- Now to Aaron's credit, not surprisingly, the Rams and the Seahawks are at the top. Although according to this, that
- would only be about 22% probability. It then gave the Chiefs, who aren't even going to make the playoffs, the Bills,
- the Patriots, and the Broncos. And those six teams, it adds up to 50%. Now, here's the question. Certainly, I think
- Aaron would take the No, of course, the Chiefs aren't making the playoffs. The Bills, the Patriots, and Broncos are.
- Um, and by the way, the seven team for us Eagles fans are the Eagles here. Um, but I thought this was fascinating.
- Again, I just want to say for our listeners out there, use generative AI. I I want to say I get I this is not a
- specially trained generative AI model. It's the one that if you have chat GPT 5.2 2 and possibly in thinking mode. Um,
- you have the same ability I do to type in what is the number of teams X that are needed to get to 50% win
- probability, order them by the highest probability teams and tell me who the teams are. That's all I typed in and
- this analysis came in and it said here and again it said closest to 50% using the strict order of the teams is the
- number six. That's pretty impressive and this is a statistic we've been talking about to understand uncertainty in the
- league really for the 11 and a half years we've been here on Wharton Moneyball. So those are the three things
- I did on chat GPT 5.2. I'll try to do some more before next week's show and also post some on our Twitter account
- Moneyball. Um, just the last thing I wanted to talk about maybe in the um, in the last couple minutes I'm going to
- talk today is um, since everybody knows I'm a big tennis fan and we're trying to
- get very soon we're going to get Paul Anakone on and he would be great to talk to about this. Um, as many people saw in
- the last week Carlos Alcarez, the number one player in the world, the holder of six majors um, fired his coach. It'll be
- very interesting to see what impact that has on his performance during the year because one of the things we've always
- asked about is the impact uh of coaching and so you know he's keeping a lot of his team but his main coach Juan Carlos
- Ferrero uh former number one in the world uh winner of I think the French um is no longer his coach. It'll very
- interesting to see if there's any drop off or if there's any narrative. So well remember he also has to beat other
- players and he has to beat especially Yanx S. So you know can we decompose whether maybe S's just getting better or
- is it that Alcarez got worse? I think this is the kind of thing that we're going to need advanced analytics for
- because I'm not sure like he's never won the Australian Open. So we said, well, let's say he loses. We can't just say,
- oh well, if he had his other coach. No. So we're going to actually have to dive into the advanced analytics and
- understand, you know, is he actually performing worse? And then of course there's also MA is match to match and
- week-toeek variation. So we're going to have to disentangle that from it. It will not be an easy analysis. We're
- certainly going to have to watch it for some period of time and maybe use last year as a if you like a control group
- and compare his performance at similar tournaments on similar surfaces and see if there's some sort of market decline.
- It's an interesting counterfactual on how you might look at the impact of coaching. So this has been uh this
- week's show of Wharton Moneyball here on the Wharton podcast network. Again, I'm
- Eric Bradlow. On behalf of myself, Kate Massie, Shane Jensen, and Aie Winer like
- to thank you for joining us. I'd like to thank Marissa Rena, like to thank D Patel, and as always, I'd like to thank
- Dion Simpkins, our associate producer, sound engineer, the person that makes all of this happen. So, between now and
- next week, enjoy your statistics, enjoy your sports. We'll see you next week here on the Wharton Podcast Network and
- Wharton Moneyball.
Brief Summary
Aaron Shatz discusses NFL analytics, playoff predictions, and the evolving role of statistics in awards voting.
Key points
- Welcome to Wharton Moneyball. The podcast kicks off with Eric Bradlo introducing the show and its focus on analytics in sports.
- Topsy-Turvy NFL Season. Aaron Shatz analyzes the unpredictability of the current NFL season, highlighting unusual team performances.
- Super Bowl Probabilities. A discussion on the probabilities of Seattle and the Rams winning the Super Bowl, revealing a significant discrepancy with market predictions.
- Seattle and Rams' Historic Ranking. Aaron reveals that Seattle and the Rams are among the top 10 teams since 1978 based on analytics.
- Frustration with Team Performance. The team has lost to three divisional rivals, raising concerns about their performance.
- Weak Division Winner Prediction. Discussion on the likelihood of a team winning the division with a losing record.
Episode highlights
- Topsy-Turvy NFL SeasonAaron Shatz analyzes the unpredictability of the current NFL season, highlighting unusual team performances.“This is a topsy-turvy year.”2:23Jan 5, 2026
- Super Bowl ProbabilitiesA discussion on the probabilities of Seattle and the Rams winning the Super Bowl, revealing a significant discrepancy with market predictions.“Wow, so you're almost twice as high.”7:18Jan 5, 2026
- Seattle and Rams' Historic RankingAaron reveals that Seattle and the Rams are among the top 10 teams since 1978 based on analytics.“Seattle and the Rams are two of the 10 best teams since 1978.”8:36Jan 5, 2026
- Frustration with Team PerformanceThe team has lost to three divisional rivals, raising concerns about their performance.“They all stink.”19:12Jan 5, 2026
- Weak Division Winner PredictionDiscussion on the likelihood of a team winning the division with a losing record.“It's a historically weak division winner.”19:58Jan 5, 2026
- Analytics in Awards VotingThe growing influence of analytics in voting for awards is discussed, highlighting changes in the panel.“Even the people who are not necessarily analytics people are going to use more analytics.”28:51Jan 5, 2026
- Controversial MVP VoteOne analyst explains why he voted against a popular MVP candidate, citing statistical evidence.“I can't vote for him for MVP when he's not number one in any stat.”30:15Jan 5, 2026
- Future of MVP VotingSpeculation on how narrative and statistics will influence MVP voting in the future.“You may not have a situation where you have a guy who's 10th or 11th in stats, but he gets a narrative going.”31:38Jan 5, 2026





