Agreeing and Disagreeing with Bill James

I have long admired Bill James and I enjoy reading his work. I remember when Doug Drinen let me borrow his collection of Abstracts, and the jealousy I felt about growing up without ever knowing of James’s work. In my mind, Bill James thinks like an economist, and I didn’t realize my way of thinking naturally was commonplace in economics until I got to college. Had I found Bill James as a teenager, my life probably would have been a little bit different.

I happened across a post by King Kaufman the other day, which led me to A Bill James Primer, extracted from James’s 1988 Abstract. But, as I read the primer, I was surprised to see how often I disagree with many of the tenets. Though I’ve been writing about these issues for years, I just hadn’t noticed. So, I thought I’d wander through the primer in a blog post. I’ll point out where I agree, disagree, and have qualified agreement and why.

1. Minor league batting statistics will predict major league batting performance with essentially the same reliability as previous major league statistics.

I think that minor-league statistics can predict performance; though, I wouldn’t go so far as to say that they predict with the same reliability as major-league statistics. In Chapter 8 of Hot Stove Economics, I examine how minor-league baseball statistics can be used to predict major-league performance. I find that certain minor-league statistics at the High-A level and above are correlated with major-league performance. However, performances at Low-A and below do not have much ability to predict major-league performance. As a general rule, the closer you get to the big leagues, the more reliable predictor minor-league statistics become. The problem is that because the quality of minor-league players is so widely dispersed, good minor-league performances can be aided by taking advantage of qualities of the competition that won’t survive in the majors. However, it’s rare for someone who performs poorly in the minors and to excel in the majors.

2. Talent in baseball is not normally distributed. It is a pyramid. For every player who is 10 percent above the average player, there are probably twenty players who are 10 pecent below average.

I disagree. Below are two graphs that showing the distribution of hitters and pitchers in baseball, with a minimum of 100 plate appearances or batters faced in 2009. The solid line is league average and the dashed lines bound the standard deviation.

OPS
ERA

For this reason, I don’t believe replacement players are cheap and abundant, and it’s one of the reasons I dislike replacement-level metrics.

3. What a player hits in one ballpark may be radically different from what he would hit in another.

I agree that ballpark effects are real. You can’t judge players without adjusting for park effects. This seems obvious now, but I don’t believe it always was. I think that James was one of the first people to study and measure park effects.

4. Ballplayers, as a group, reach their peak value much earlier and decline much more rapidly than people believe.

Depends on which “people” you are. I think James’s claim that player peak at age 27 is wrong. I believe the methodology of his initial study of the issue is flawed. My own estimates indicate that players peak around 29. Though I disagree with his initial findings, the chapter on aging in his 1982 Abstract was groundbreaking in how he thought about aging functions.


5. Players taken in the June draft coming out of college (or with at least two years of college) perform dramatically better than players drafted out of high school.

It depends on what you mean by dramatic, and how this ought to inform draft strategy. Jim Callis of Baseball America conducted an analysis of how well high school and college players succeeded in the majors. College players did a little bit better—I wouldn’t call the difference “dramatic”—but that’s not surprising and I don’t think the identified difference should affect teams’ drafting strategy.

College prospects are a survivors of a previous high school draft pool. After three or four years of observation, we have a better idea of whom the good players are. It should be no more surprising that college players succeed at a slightly higher rate than high school players than that minor-league players three years from high school are more successful than recently-drafted players. We have better information about them, and time has culled out some non-prospects. But, if a team tried to improve its return to the draft by drafting more college players, it’s not necessarily going to benefit from the previously measured higher success rate of college players. The college draft pool is more certain: not only do we know the good players, we know the bad ones too. Part of the reason the returns to drafting college players are better is that teams stop drafting college players once the good ones are gone.

Let’s say you had to buy a car today. You have time to test-drive three, and you like one and hate the other two. But by the time you go to buy the good car you test-drove it’s gone. Should you buy one of the cars you know you don’t like or go for one you have a chance of liking from a pool of cars you didn’t test drive that were on your list of prospective cars from Consumer Reports? The latter group of cars has more risk, but at least has an upside that the former group doesn’t have.

So, agree with James here on the fact, but I don’t think the higher success rate of college draftees means that teams can improve their drafts by focusing on more college players. However, I’m not sure if James believes anything more than the fact.

6. The chance of getting a good player with a high draft pick is substantial enough that it is clearly a disastrous strategy to give up a first round draft choice to sign a mediocre free agent.

I’d probably agree with this, but I think the fact that the pick will likely yield a player who can be paid far less than he’s worth is more important than the likelihood that the drafted player is good. On talent alone, I expect a mediocre free agent has an expected talent value of close to that of a high draft pick.

7. A power pitcher has a dramatically higher expectation for future wins than does a finesse picther of the same age and ability.

I don’t know.

8. Single season won-lost records have almost no value as an indicator of a pitcher’s contribution to a team.

Agree.

9. The largest variable determining how many runs a team will score is how many times they get their leadoff man on base.

Getting the first man on in the inning is important, but I’m not sure that it’s the largest variable. I’ll concede that I don’t know.

10. A great deal of what is perceived as being pitching is in fact defense.

I find baseball is 13% defense, and so does James. Agreement.

11. True shortage of talent almost never occurs at the left end of the defensive spectrum.

Possibly, but the problem is that for most of the game, players have to play both sides of the ball. And because fielding is such a relatively small part of baseball, purely defensive guys don’t have a lot of value. There is a shortage of guys who are decent hitters and play defense well, especially at defensive positions.

12. Rightward shifts along the defensive spectrum almost never work.

Agree.

13. Our idea of what makes a team good on artificial turf is not supported by any research.

I have no clue, so he’s probably right.

14. When a team improves sharply one season they will almost always decline in the next.

Agree, that’s simple mean reversion.

15. The platoon differential is real and virtually universal.

Agree.

Even as I find differences here, my affection and respect for James as a thinker has never diminished. I agree with Tyler Cowen that James is one of the greatest living social scientists. What I like about James is the way he approaches problems and finds answers. I may not agree with the exact method used or the answer—although; in most cases I do—but I like the frame of mind with which James tackles curiosities. The sabermetric approach that James espouses is right. That his findings sometimes don’t fit with mine, doesn’t bother me.

33 Responses “Agreeing and Disagreeing with Bill James”

  1. Mitch says:

    And what about the research of the past twenty two years? Do you agree or disagree with that?

  2. #13 — artificial turf was much more of an issue in 1988 when James wrote this as it is now. We’ve gone from an era of a large number of multi-purpose facilities that had artificial turf to allow them to quickly turn into a football stadium, to two (both in the AL East — Tropicana in St. Pete and Rogers in Toronto).

  3. LarryM says:

    I’m interested in hearing some further thoughts on #2. This is in my opinion an underexplored topic. On the one hand, some other research I’ve read recently (can’t find the link – it was data showing a high number of IP/PA by sub-replacement level players each year) and my own subjective impressions agree with your skepticism. On the other hand, if you think about the logic of James’ argument, he SHOULD be right. (i.e., major league players represent the extreme right tail of the normal distribution of baseball skills of all adults.) Why does the reality diverge from what we should expect?

  4. Nate says:

    Kind of funny that JC is completely unaware that by adding the 100 min PA requirement to his baseball player sample, he’s looking a very different population distribution.

  5. Pete says:

    JC, that graph in #2 is flawed.

    First of all, it doesn’t address the claim made by James. He said, “Talent in baseball is not normally distributed.” Which is more than just the majors. James’ statement is quite obviously true or there wouldn’t be 750 players in the Majors and thousands in the Minors.

    Second, there is all kinds of selection bias in your graph. 1) It’s only players who make the majors. 2) It’s only players who get 100 PA in the majors. 3) It’s only one season worth of data.

    Your graph doesn’t say anything about the true distribution of baseball talent, only about the talent selected by baseball teams to play and stay in the majors. Basically it says that baseball is efficient at selecting talent because the hordes of players with less than major league talent either don’t make it at all or don’t get much playing time if they do.

    Basically, if you include the major league ability of all professional baseball players, there are thousands more on the left side of your graphs and zero more on the right side.

  6. JC says:

    When selecting sample inclusion cutoffs there is a tradeoff. Setting cutoffs high excludes some player observations, but gets a good representation of true talent. Setting cutoffs low includes more player observations, but because of so few observations it’s less able to capture true ability. I tried both lower and higher cutoffs, and the results looked somewhat similar. 100 PAs is quite low for measuring ability, and as I state in note 61, on page 244 of my book, “Lower cutoffs yielded similar results.” Cutting observations does not induce a distribution that looks like the right tail of a normal distribution, as replacement-level theory assumes.

  7. Chris W says:

    I enjoyed this article, but with regards to your graph for tenet #2, I don’t think Bill was talking about “talent in the MLB with 100 AB.” He’s talking about in the world. So though Replacement and Sub-replacement level talent is unlikely to get 100+ PA in the MLB it would be interesting to study the talent of players in the MiLB (with a corresponding adjustment of their MiLB stats vis a vis MLB correlation) and see how the plot looks then. Also: college baseball, independent leagues, international leagues. I don’t know for sure, but I strongly suspect that Bill is right about this, if we assume he’s talking about talent in the general populace and no just at the highest level.

  8. John says:

    Every Triple-A player who isn’t a prospect working his way up is a potential replacement-level major leaguer, and in fact, a lot of them have spent time in the majors in just that role. When you consider that, it’s clear that replacement-level players are extremely abundant.

    As far as the minor league statistics go, you are aware that Bill James was only speaking about numbers from AA and AAA baseball, aren’t you? He never used numbers from any version of A ball, let alone low-A, in developing his system of translating minor league statistics.

  9. JC says:

    If there was positive skewness, it would be evident in the bottom of major-league players.

    And while I’m on the subject of replacement-level, here is a comment that I made on the subject in a Sons of Sam Horn chat last week.

    Replacement-level thinking is based on the notion that there is a vast supply of equally-talented players at the bottom the league. This is not the case, the distribution of talent across players is bell-shaped. Players who aren’t in the league are a real step-down from major-league players. Now, this doesn’t mean that cheaper inferior players don’t put downward wage pressure on marginal players—I think they do, as did economist Simon Rottenberg in the 1950s—but taking the step of valuing the bottom level of players at the league minimum is wrong. Having the worst player in the league on your team will better your team than just randomly grabbing a minor-league player who earns the minimum. And adding the next best player not in the majors to your roster isn’t easy. There are 29 other teams whose player control covers far beyond their rosters. If you want a guy who you can pay $400K who would net you $1 million in revenue, another team isn’t going to hand him over for free just because he’s not on its big-league roster.

    I’d also like to add that 1/3 of the league is considered to be below replacement level. And 15% supposedly cost their teams more than $1 million. GMs make mistakes, but eight players per team who shouldn’t be there? I don’t buy it.

  10. Pete says:

    I ran the numbers using Fangraphs WAR from 2010. About 28% of pitchers and 37% of hitters were below replacement level. Which is about 33% of all players. However, this includes pitcher hitting, which is almost universally below replacement because Fangraphs does not use a positional adjustment for.

    Also the 28% of pitchers made up 13% of innings pitched and the 37% of hitters made up only 16% of plate appearances.

    But when considering these numbers you have to take several things into account.

    1) If you take out pitcher hitting, about 28% of pitchers and hitters were below replacement level. And they made up 13% of innings pitched and 14% of plate appearances.

    2) Most of these below replacement performances are based on very small sample sizes. The pitchers below replacement level averaged 31.2 innings and the hitters averaged 86 PA. So most of those hitters wouldn’t even make your talent distribution graph. And probably about half of those pitchers wouldn’t show up on your chart.

    3) With many players right around replacement level, you would expect a good portion to have observed value below replacement level though their true value may be right around or slightly above replacement level.

    4) About 60% of hitters and 75% of pitchers below replacement level were below by only a third of a win. Making it highly likely that many of them are actually above replacement level in talent.

    5) If you don’t believe replacement level exists then why do you base your argument on the fact that Fangraphs set their replacement level too high? (Rally WAR at Baseball Reference and WARP at Baseball Prospectus both use lower replacement levels).

  11. LarryM says:

    Pete really starts to move the ball forward on this. I, too, suspect that part of the answer is sample size – i.e., a lot of the players performing below “replacement level” really have replacement level ability, but this fact is masked by luck in small samples. Though I doubt that this explains all of the problem.

    It’s frustrating that this is such a little discussed issue though – I think J.C. has a real point, but it bugs me that he doesn’t go deeper into his analysis (maybe he does in the book, I don’t know), and is IMO overly categorical in his conclusions.

  12. Pete says:

    A quick correction:

    Baseball-Reference WAR actually uses a slightly higher replacement level than Fangraphs (I believe B-R uses a team that would have a .320 winning percentage and Fangraphs is set at .290)

  13. JC says:

    I exclude pitchers from the hitting side. It doesn’t matter to the argument about why 1/3 of the league is below replacement whether or not some players would make the arbitrary sample cutoff that I settled on. That estimate was just to demonstrate that the distribution is not as positively skewed as assumed, the cutoff was set to measure the frequency of talent levels. If there are some measured sub-replacement-level players because of random variation, there will also be some that are positive that should be negative. The best estimate of sub-replacement makeup of the league is about 1/3—that’s a lot of performance below replacement-level, especially given that so much superior talent should be available at the league minimum. Why not just raise replacement level? Because replacement-level as a concept serves no useful purpose if replacement-level players are not scarce.

  14. JC says:

    LarryM,

    I discuss the argument in more depth in my book at the end of Chapter 4. Also, I’ve also written about the subject of replacement-level on the blog over many years. See here, here, and here, for examples.

  15. Michael Poplawski says:

    True replacement-level players don’t get 100 PA in MLB.

    I play baseball, I’m not good enough to play professionally. But I’m in the pyramid of baseball talent. There are fewer people better than me than worse than me, and that is true of anyone who ever learned how to play any sport.

    #2 above is trying to answer a different question than what Bill James was discussing.

  16. Colin Wyers says:

    JC, regardless of where you set the cutoff, you’re still going to have a cutoff of at least one major-league PA or one major-league batter faced. We know that there were at least 9,000 players who played at ANY level of professional baseball (looking at affiliated minors and the major leagues – so really there’s more if you include professional leagues in Japan and Korea, the independent minors, and high-level college programs).

    What would your distribution look like if all of those players were included? Would it still be normal?

  17. Pete says:

    Let’s just tackle what Bill said:

    “For every player who is 10 percent above the average player, there are probably twenty players who are 10 pecent below average.”

    Players 10% above average and 10% below average by OPS+ for the 2006-2010 seasons, cumulative, min 100 PA. (Pitchers excluded) Data from B-R Play Index

    110 or above: 120
    91-109: 174
    90 or below: 253

    About 21 below for every 10 above.

    B-R ERA+ isn’t linear, so I just used data from Fangraphs for 2006-2010 with a minimum of 100 batters faced. Starters and relievers were calculated separately.

    10% Above: 240
    Between 10% Above and 10% Below: 343
    10% Below: 498

    That’s nearly 21 below for every 10 above.

    So Bill was right. It’s a pyramid.

    One year of data is not enough because there’s much more turnover at the bottom than at the top.

  18. JC says:

    Colin,

    That there are a lot of non-major-league players is irrelevant. It’s a quality argument, not a quantity argument. There are many more college students than professors. Future professors will come out of that sample of students, but most are not capable to become professors.

    Pete,

    You have made an elementary error in your analysis. The comparison is 10% better than average versus 10% worse than average, not 10% or greater than average versus 10% or worse than average. OPS and ERA are going to bounded on the good side, you can only be so good. But on the low side you can be very bad. The important point is that the distribution is not sufficiently positively skewed to indicate that there talent at the low end is non-scarce.

    And on a related theme of points,

    If Bill James was referring to baseball talent in the world as a whole or all baseball talent, then this is an odd statement to include in a primer of basic tenets of sabermetrics. Really, there are a lot more people in the world who aren’t as good as the average of people who play baseball? I’m not sure how that knowledge is new or useful. The only sample that is relevant are those at the major-league level (whether or not they actually play). If there is a glut of talent available that is of major-league caliber, then the left tail of the distribution of major-league players would be much fatter than the right tail. It is not. There is not a honey hole from which they can freely draw talent.

  19. Charles says:

    Hey, watch me carefully parse Bill James’ words and ignore the actual meaning of his statement in an effort to drum something up! JC – you’re going to have to interpret the statement as James meant it to be, not as however you want to twist it into. By setting the bar at 100 PA, we are purposefully excluding the true replacement player, for example the guy who gets called up from AAA when a starter goes on the DL for 15 days. The replacement player pool is not those players who have established themselves as major leaguers. It’s the pool of players who could get a shot at the majors. And since JC needs it spelled out, James was not referring to only players who are specifically and exactly 10% better or worse than average, as if players 20% better or worse couldn’t possibly fit into a similar pyramid. But apparently it is much more fun to build up a strawman to create some controversy with than take the statement as it was meant to be. Saying the knowledge of the broader interpretation isn’t new or useful is like saying James’ third statement isn’t new or useful either. Just like ballpark effects, distribution of talent wasn’t well understood over 20 years ago, and James was one of the first to tackle this.

  20. Jim Glass says:

    “We’ve got to get rid if this guy because he’s only average, that’s not good enough to win — and it will be *easy* to replace him with someone better because half of all players are above average!”

    If I’ve heard this nonsense a thousand times I’ve heard it a million. And this is the popular idea James was responding to in 1988 (I have that Abstract too).

    But now the dufuses who believe this have your charts to back them up. “See!”

    Of course, the average pro ballplayer is down in AA somewhere. That’s how the talent is really distributed.

    And you believe that too. Otherwise you wouldn’t say “I don’t believe replacement players are cheap and abundant”.

    Why aren’t they? Because even the marginal MLB/top AAA talent is so far above the average level of all pro players as to be *not* “cheap and abundant”.

  21. Dodger300 says:

    I am shocked that a supposedly knowledgeable person would try to make the silly arguments that JC is making about a “lack” of replacement level players.

    Obviously, you just don’t get it at all.

    Don’t feel bad, though, Bill James writes over a lot of people’s heads.

  22. Mike Fast says:

    To further the point Pete made earlier about turnover in the player population and what you are missing by looking only at a single year, I looked at the hitter population with at least 100 plate appearances between 1994 and 2009. In that population, 404 hitters had an OPS above average and 1133 hitters had an OPS below average. Moreover, 28 hitters had an OPS more than one standard deviation better than average, and 354 hitters had an OPS more than one standard deviation worse than average.

    The distribution is clearly skewed, contrary to what your examination of a single year led you to believe.

  23. Mike Fast says:

    JC, you should look at any hitter who had at least 100 plate appearances in total during the whole time period, not just those who had 100 plate appearances in any given single year. You just repeated your same study with the same error on a larger scale and did not consider the point about turnover that Pete and I have raised.

  24. JC says:

    Aside from the silliness of looking at players who gather 100 PAs over many years, I am not arguing that there are just as many good players as bad players. In fact, I am arguing that there are many more bad players than good players which is why a certain level of talent is not just lying freely available for teams to pick up. I stated this in my response to Pete. In any given season, when you have to dip down for a replacement player you are going to pick up an inferior player. There is not a glut of talent clustered at the hypothesized replacement level.

  25. cliff says:

    Let’s try a different athletic endeavor.

    Sprinting.

    Just for discussion, let’s say that roughly the number of people who have ever run a verified sub 10.00 100 meters is 50. Then, the number sub 10.2 is probably 500. The number sub 10.5 is probably 10000. Etc.

    I don’t understand why that argument is not applicable to baseball.

    I think a problem is that players that we want to judge in terms of “replacement level” (#5 starting pitchers and “worst starting position player”) are actually usually above replacement level.

    For most teams there are about 2 to 3 slots on their roster that call for a replacement player: #4 righthanded reliever, #3 lefthanded reliever, # 5 outfielder, #3 catcher, #2 backup middle infielder. And there are lots and lots of those.

  26. Mike Fast says:

    First, I had included pitchers hitting in my original numbers. I should have removed those. I also realized that my data only ran through 2007 instead of 2009. Neither of those change the conclusion substantially.

    There were 348 hitters with at least 100 PA above average in OPS, and 958 such hitters below average, from 1994-2007.

    I also incorrectly counted the standard deviations. There were 75 hitters more than one standard deviation above average in OPS and 422 hitters more than one standard deviation below average (not including pitchers hitting).

    It appears to me that you simply repeated your previous study and expanded the years rather than considering the issue of turnover that Pete and I have raised. You should look at players that had at least 100 plate appearances over the whole time period 1994-2009 and not restrict yourself to players who had at least 100 PA in a given single year.

    To give a few specific examples, are you including players like Greg LaRocca, Mike Rouse, Fausto Cruz, Les Norman, and Jim Tatum in your sample? They all had 100 PA or more during the period 1994-2009, but spread across more than one year.

  27. Mike Fast says:

    So you don’t think that the players who got 100 PA distributed over two or three years are a good representation of replacement level? Why not? They would seem to be a perfect representation. Yet you purposefully exclude them without giving a good reason, other than an ad hominen allegation of silliness.

    If it radically changes your distribution, which it does, you might want to consider why.

    (My previous post giving details about the 100-PA multiple-season players was cross-posted with your preceding post, in case that isn’t clear from the context.)

  28. JC says:

    Mike,

    As I previously stated (in the comments to this post and in my book), the reason I use the 100-PAs cutoff is that I need a sufficient sample size to measure talent level. There is a tradeoff here, because I lose observations from players below that level. I looked at samples below this cutoff and it did not have a big effect on the distribution.

    When you look at cumulative PAs over more than a year, you are getting lots of small samples that are not akin to the typical groupings of at-bats that players take regularly. They are also spread over time so it’s not clear what level of talent is being represented. That is why I think it would be silly to include them in the sample; however, as I said, “aside from the silliness,” it is not correct to sum the frequency of bad players over time to tell us something about the availability of bottom level talent in the league in any given year. There are going to be a few good players who stick around and the players who do not stick around will obviously be worse. Thus, when looking for how much talent might be around when a team is looking to fill its roster, the fact that there were some bad players in the league several years ago is irrelevant. Those guys are out of the talent-well and gone from baseball. There are lots of players who get to play baseball who are not very good and don’t get to play long. I’m not disputing this. I also responded that it is incorrect to look at cumulative totals above and below benchmarks (e.g., 10% and SD) because there is an upper bound to how good a player can be. The inferior tail of the talent distribution includes more observations, not because it is fatter than the superior tail, but that it is longer.

    The initial James statement that I was responding to claimed that baseball talent was not normally distributed, and that “For every player who is 10 percent above the average player, there are probably twenty players who are 10 pecent below average.” This implies that the frequency of observations below average should be greater than the frequency above in reverse proportions. This is clearly not the case.

  29. Mike Fast says:

    So if you remove a very large number of players who are below average because they aren’t good enough to stick around, then you can get the distribution to approximate a normal distribution? That’s a tautology, and surely you can see that.

    You state, “the reason I use the 100-PAs cutoff is that I need a sufficient sample size to measure talent level.” That’s irrelevant to my point, and I don’t know why you state it. I also used a 100-PA cutoff.

    You state, “I looked at samples below this cutoff and it did not have a big effect on the distribution.” That may be true for the single season level, but it is decidedly untrue at the multi-season level. Thus, it is either irrelevant or misleading to restate that point in addressing my question.

    You state, “When you look at cumulative PAs over more than a year, you are getting lots of small samples that are not akin to the typical groupings of at-bats that players take regularly. They are also spread over time so it’s not clear what level of talent is being represented.” They are mostly spread over two consecutive years. I’m doubtful that talent level changes so radically in the span of one year that this data becomes unusable. In fact, your aging curves suggest that talent level changes very slowly. If you accept that finding, these samples should be very useful to you. To discard a large portion of the data set that radically changes your conclusion, you ought to have a better reason than simply thinking the data might have problems.

    If you include this data, it demonstrates quite clearly that your conclusion is wrong. If you want to exclude this data, you ought to demonstrate with evidence-based tests why you can throw it out and maintain any sort of integrity in your conclusion.

  30. JC says:

    Mike,

    So if you remove a very large number of players who are below average because they aren’t good enough to stick around, then you can get the distribution to approximate a normal distribution? That’s a tautology, and surely you can see that.

    If I had argued that, it would be a tautology, but that was not my argument. I stated, “it is not correct to sum the frequency of bad players over time to tell us something about the availability of bottom level talent in the league in any given year (emphasis added for clarification). There are going to be a few good players who stick around and the players who do not stick around will obviously be worse. Thus, when looking for how much talent might be around when a team is looking to fill its roster, the fact that there were some bad players in the league several years ago is irrelevant.”

    You state, “the reason I use the 100-PAs cutoff is that I need a sufficient sample size to measure talent level.” That’s irrelevant to my point, and I don’t know why you state it. I also used a 100-PA cutoff.

    This was in response to your claim that “So you don’t think that the players who got 100 PA distributed over two or three years are a good representation of replacement level? Why not? They would seem to be a perfect representation. Yet you purposefully exclude them without giving a good reason, other than an ad hominen allegation of silliness.” I was restating my good reason that you had previously ignored.

    You state, “I looked at samples below this cutoff and it did not have a big effect on the distribution.” That may be true for the single season level, but it is decidedly untrue at the multi-season level. Thus, it is either irrelevant or misleading to restate that point in addressing my question.

    And then I argued why it was incorrect to use a cumulative sample, because the issue is the availability of “replacement-level” talent at any given time.

    You state, “When you look at cumulative PAs over more than a year, you are getting lots of small samples that are not akin to the typical groupings of at-bats that players take regularly. They are also spread over time so it’s not clear what level of talent is being represented.” They are mostly spread over two consecutive years. I’m doubtful that talent level changes so radically in the span of one year that this data becomes unusable. In fact, your aging curves suggest that talent level changes very slowly. If you accept that finding, these samples should be very useful to you. To discard a large portion of the data set that radically changes your conclusion, you ought to have a better reason than simply thinking the data might have problems.

    Yes aging changes are minor over time. Age was not mentioned in my argument. I do not think that a few scattered PAs over several years are the same as PAs accrued in a single season when measuring talent. You are free to disagree. But that point was an aside, which I initially stated and restated in a follow-up comment. I took your argument at face value (even though I do think it is silly) and explained why I felt these observations were not relevant to the question at hand. The main reason for not including these observations is because it includes a group of players who are not readily available to serve as replacements. That someone took 150 PAs from 1994–1996 is irrelevant to the availability of marginal players today.

    If you include this data, it demonstrates quite clearly that your conclusion is wrong. If you want to exclude this data, you ought to demonstrate with evidence-based tests why you can throw it out and maintain any sort of integrity in your conclusion.

    Again, my argument is an economic one. Are replacement players cheap and abundant? I say no, because in any given year there does not appear to be a glut of talent at the bottom of the talent spectrum. Many players have come and gone in the past, but they are gone and not part of the available talent pool.

    I have entertained your critiques and explained why I do not agree with them multiple times. This exchange is going nowhere, and I see no reason to continue it.

  31. Donald A Coffin says:

    The question of the distribution of *talent* is, I think, somewhat different from the distribution of *outcomes,* in a situation in which the objective of one participant (say, a hitter) and the objective of another participant (say, the pitcher/defense) are opposed. The question of the distribution of talent can, I think, only be truly answered in one of two circumstances:

    1) All hitters hit against only a set of pitchers backed by a defense of given quality. Then, the only variable is the talent of the hitter. (Transpose this for pitcher/defense–hold the hitter talent constant and vary the pitcher/defense.)

    2) Participants, while competing with each other do not oppose each others’ efforts. Take, for example, the distribution of driving distances on the PGA Tour in 2010. I can’t reproduce the graph here, but it is clearly non-normal. And it is positively-skewed.

    I suspect, but can’t right now, find data for (for example) scores on the Professional Bowlers’ Tour, but I suspect they are also non-normal.

    But I’m not surprised that the distribution of outcomes in MLB is approximately normal, just as I would not surprise that the distribution of outcomesin the NFL is roughly normal. The interaction of two non-normal distributions can easily be normal.

  32. Pete says:

    JC,

    Looking at only the above-average portion of your updated graph (where there should be little to no selection bias because these are the very best players available) we see that for each lower level of performance there is an increasing number of players. As you get below average the efficiency of the league minimizes below average *performances.*

    However, there is still below average *talent* available in increasing numbers. These players just don’t receive major league playing time.

    Surely you agree that there are more players clustered around “22nd-25th man on the roster” talent than “18th-21nd man on the roster” talent.

    Of course, this won’t show up when you look at players who receive 100 MLB PA in a season. Because most of these players are in AAA.