Letter to Bryan Burwell

In today’s St. Louis Post-Dispatch, columnist Bryan Burwell writes the following:

Bill James, baseball’s ultimate seamhead and statistical guru, tried Tuesday to explain to me why Jim Rice should never get into the Hall but Mark McGwire, Sammy Sosa, Barry Bonds and many of their chemically enhanced contemporaries should. It was a dazzling bit of stat geek mumbo-jumbo that basically came down to this:

Stats and baseball’s integrity are very relevant to baseball … unless I don’t want them to be.

This prompted me to write the following to Mr. Burwell in an e-mail:

I believe your assertions about several MLB players using steroids to create a home run chase are misguided. While there is a cloud of suspicion surrounding some players this is hardly evidence that they cheated their way to success. Please read the following paper for a discussion of the statistical variance of home runs (http://www.arthurdevany.com/webstuff/images/HomeRunHitting.pdf). The achieved excellence by the sluggers you mention is all within the natural variance of home run hitting in baseball history.

I don’t expect to hear back, but it bothers me that so many people are willing to convict players of using steroids with evidence that wouldn’t meet the weakest of legal standards. If you have not read De Vany’s paper, which I posted a link to above, you should. Art De Vany is a retired economics professor with an excellent research reputation. I wish more people would take notice of his work. I have yet to see someone successfully refute De Vany’s findings.

7 Responses “Letter to Bryan Burwell”

  1. Guy says:

    “I have yet to see someone successfully refute De Vany’s findings.”

    I agree that drawing conclusions about players’ PED use should require some evidence. But I don’t think you should recommend DeVany’s paper to anyone. It is riddled with poor analysis and faulty reasoning. Among those I recall (I read it when you first posted a link):
    1) he claims there has been no overall increase in HR hitting, which is clearly wrong;
    2) he uses means that take no account of playing time, allowing him to conclude that Bonds/Sosa/Mac are many more SDs above average than is true;
    3) he invents a useless statistic, HR/H, and uses it to ‘prove’ many of his points, w/o any explanation of why it’s a relevant metric;
    4) the whole point of the exercise is to knock down a strawman, i.e. that Bonds/Sosa/Mac must have used PEDs because their HR production would be statistically impossible otherwise. But the serious arguments against these players do not follow this logic — they are based on sudden changes in performance, unusual aging patterns, and other factors (including a quasi-admission in Bonds’ case). To the extent DeVany addresses these issues at all, it’s in a tendentious and unconvincing manner. His argument boils down to “it is statistically possible that these players’ performance did not benefit from PED use.” Well, sure. I don’t think anyone, including the players’ harshest critics, would dispute this claim. But it’s an obvious and irrelevant point.

    No one has bothered to “refute” this because it doesn’t say anything worth responding to. Mr DeVany may have been a great economist — I have no idea — but the paper is an embarassment.

  2. MT Head says:

    “It is said that when Honus Wagner saw Babe Ruth hitting home runs he altered
    his hitting and increased his home run output from less than 10 a year to 40 a year.” The above is a quote from the paper (p. 13). Wagner was 45 and out of the majors by the time Ruth started hitting home runs. He never hit more than 10 in his career and Ruth was still in the Baltimore orphanage when he did that. De Vany may know economics but he doesn’t know baseball.

    “Guy” is absolutely right with #4. In order to convince anyone, an advocate must have convincing explanation for, in particular, “sudden changes in performance, [and]unusual aging patterns.” Bonds’ career arc is unlike any other in history and it is hard to believe that, without chemical assistance, he suddenly figured out to go from greatness to Superman at an age when his body should have been breaking down.

  3. JC says:

    I’m sorry guys, but your analysis is incorrect. I don’t think you understand the paper.

    he claims there has been no overall increase in HR hitting, which is clearly wrong;

    If HRs are Lotka distributed (a severely non-normal distribution) then the increase is within the variance. That’s his argument. He most certainly does not argue that HRs have not increased. They have not increased more than expected given several non-steroid factors. That is a very different argument than your claim.

    he uses means that take no account of playing time, allowing him to conclude that Bonds/Sosa/Mac are many more SDs above average than is true

    Umm, there are some means in the paper. But he also looks at HR totals. How is playing time for any individual player affecting anything? I’m not sure which way this would induce bias in the particular direction you suggest. In his regression model he does account for the total numbers of games played in explaining home runs per season. But overall, I don’t really understand your argument.

    he invents a useless statistic, HR/H, and uses it to ‘prove’ many of his points, w/o any explanation of why it’s a relevant metric

    That is ONE metric he uses. He defends it, and I have no problem with it. What’s wrong with it? I think it’s a pretty good measure of power, and he probably ought to be given credit for thinking of it.

    the whole point of the exercise is to knock down a strawman, i.e. that Bonds/Sosa/Mac must have used PEDs because their HR production would be statistically impossible otherwise.

    After your first “critique” you are in not position to accuse others of strawmen. His point is that what seems to be odd to us in terms of HR hitting actually fits into a pattern that is not so odd. We often see things in terms of normal distributions, because many empirical phenomena in the world fit it. HRs seem odd, because we are more accustomed to using the normal lens.

    To the extent DeVany addresses these issues at all, it’s in a tendentious and unconvincing manner. His argument boils down to “it is statistically possible that these players’ performance did not benefit from PED use.” Well, sure. I don’t think anyone, including the players’ harshest critics, would dispute this claim. But it’s an obvious and irrelevant point.

    No, his argument is that steroids are not necessary to explain the jump in home run totals. That is not only relevant, it’s science.

    Bonds’ career arc is unlike any other in history and it is hard to believe that, without chemical assistance, he suddenly figured out to go from greatness to Superman at an age when his body should have been breaking down.

    See, that’s why the paper is important. You’re arguing that because no other explanation exists, steroids must be the reason. De Vany makes the case that home run hitting is within the expected variance. He also addresses Bonds specifically in the paper. And even if you don’t buy his explanation, one player’s improvement that differs from the ordinary cannot be proof of anything. For example, Julio Franco must be on the juice. This argument has actually been made.

  4. JC says:

    MT,

    You’re right about the Wagner/Ruth thing. Why not send him an e-mail? He sounds to be spouting something he “thinks he knows but ain’t so.”

  5. Repoz says:

    “It is said that when Honus Wagner saw Babe Ruth hitting home runs he altered
    his hitting and increased his home run output from less than 10 a year to 40 a year.”

    I’m sure DeVany meant Rogers Hornsby…a common mistake at the time…:)

  6. Guy says:

    “He most certainly does not argue that HRs have not increased.”

    He makes precisely this claim, many times. Just two examples:
    “There has been no change in MLB home run hitting for 45 years, in spite of the new records. Players hit with no more power now than before.”
    “Why do all the attempts at explaining an increase in MLB home runs seem to fail? I think it is because they are attempting to explain something that has not happened.”

    He is arguing there is no general or systemic increase in HR-hitting, but that is exactly what has happened even after you remove the impact of the “big 3″ (which is trivial) and account for #of teams/games. There have been 9 NL seasons in which teams hit more than 2 HR/G, and 8 of them are the past 8 seasons. 1998 and 1999 were not 1 or 2 outlier years, but part of a stable, high-HR era. We can debate the causes — parks, pitching, bats, balls, PEDs — but not its existence. (Ironically, a higher mean makes it MORE, not less, plausible that the big 3 could perform as they did w/o help from PEDs.)

    “If HRs are Lotka distributed”
    Actually, I don’t see much evidence for this. It’s not like most players hit 4-5 HRs and a handful hit 50. Once you control for playing time, and account for the fact that MLB players are in the ‘right tail,’ the distribution doesn’t look unusual. Among FT players, the mean is around 18, SD of maybe 10 (I’m guessing). Maybe not normal, but nothing like earthquakes or works of musical genius (as DeVany argues).

    “his argument is that steroids are not necessary to explain the jump in home run totals.”
    Agreed. But so what? Hardly anyone claims that the big 3’s HR totals, by themselves, are proof of PED use. But logically, a Lotka distribution, even if true, can’t possibly prove that they hit their HRs w/o help from PEDs — only that they may have. (BTW, MT above makes a much different claim based on Bonds’ career arc — .21 H/HR thru age 34, .36 age 35-40 — which DeVany’s argument doesn’t address, much less refute).

    I could go on — the paper is riddled with problems. But don’t take my word for it, JC. Send the paper to some sabermetric analysts you respect (Tango, DSG, Studes, whomever) and ask for their feedback. I think all will agree the paper is deeply flawed, and contributes little to our understanding of either HR hitting or the impact of PEDs.

    “I’m sorry guys, but your analysis is incorrect. I don’t think you understand the paper.”
    Assuming that people who disagree with you “don’t understand” the argument is patronizing and insulting (and it’s not the first time I’ve seen this on this site). If you want the site frequented by a handful of people who will give you only fawning praise, just say so. But if you want thoughtful debate and discussion, you need to show people the respect they deserve.

  7. JC says:

    Assuming that people who disagree with you “don’t understand” the argument is patronizing and insulting (and it’s not the first time I’ve seen this on this site).

    I’m sorry you feel this way. Maybe you shouldn’t accuse other people who are making very good arguments that you misinterpret as arguing strawmen. Please, you are in no position to play the polite card here.

    It’s clear from your arguments that you do not understand the paper. I’m sorry you take offense to my saying so. What else am I supposed to say? For example, you don’t think HRs are Lotka-distributed. Did you see Figure 1 on page 32? It shows exactly this. Look, I don’t know how else to convince you of your errors, and I don’t plan to spend any more time discussing it. You probably should stop trying to convince me.

    I teach statistics and conduct statistical research for a living. I HAVE to know this stuff. I’m not trying to declare myself to be intellectually superior, I’m just trying to let you know that I’m not making this stuff up. I have no personal stake in this. I don’t know Art De Vany. All I know is that he wrote a very good and interesting paper, and your criticisms are incorrect. He wins. If you were making sound arguments, I would certainly change my opinion, as I have on many occasions in my life. De Vany’s work may ultimately proved to be wrong, but you’re not even putting a dent in his argument.