In an effort to further our debate over what the statistics say about Roger Clemens’s possible steroid use, Dave Berri asked Justin Wolfers and I to address our disagreement on Wages of Wins.

So here we have two of my friends appearing to have a very public disagreement. And this led me to think of my role in life as a uniter (yes, I have always thought of myself as a uniter, not a divider). So last night I sent the following e-mail to both Bradbury and Wolfers.

Would each of you agree with the following statements?

Justin and company are arguing that the statistics do not show Clemens is innocent.

JC is arguing that the statistics do not show that Clemens is guilty.

Both Bradbury and Wolfers graciously responded to my inquiry. And each also agreed to let me post their responses.

Here is an excerpt from Justin.

Beyond what the data don’t “prove” (both guilt and innocence), there is a tougher intermediate question: Are Clemens’ career statistics better thought of as evidence for the prosecution, or evidence for the defense? We see enough unusual patterns in his career trajectory that we think of them as being more persuasive for the prosecution than for the defense. Different approaches yield slightly different conclusions, but enough of them look somewhat odd that it is hard to see an honest presentation of the data helping Clemens’ case.

Here is an excerpt from me.

I agree that the statistics cannot exonerate Roger Clemens nor any other baseball player accused of using steroids. I also think they cannot convict…. In Clemens’s case, especially considering the specificity of Brian McNamee’s allegations, I don’t think swings in the data support the current allegations…. So, to put it in Justin’s terms, I think the evidence supports the defense.

Justin has also added another post at Freakonomics (here is the initial post), where he walks through the data analysis process. I think this analysis puts too much weight on the WHIP statistic. WHIP suffers from the same malady that ERA does: it is highly variable because it includes fielder contributions from hits on balls in play.

Generally, one way we can look at metrics to see if they measure skill or if they are just reflecting random fluctuations is to see how individuals perform over time. If the skill is real, then pitchers ought to perform similarly from season to season. Here are the year-to-year correlations for pitchers throwing back-to-back 100+ innings seasons from 1980–2006.

Metric		Correlation
Strikeout Rate	0.79
Walk Rate	0.64
WHIP		0.42
ERA		0.37

All measures are correlated, but the correlation is lower for the metrics that include fielder contributions. The season-to-season correlation between individuals pitchers’ WHIP and ERA are quite similar. Also, both metrics vary similarly: the average coefficient of variation (mean/standard deviation) for the pitchers in the sample is 2.46 for WHIP and 1.99 for ERA.

Here is a graph of ERA and WHIP by age for Roger Clemens on that using connected scatter plots and quadratic fit curves.

The metrics tend to move in concert (correlation = 0.9), and the small difference in quadratic fit seems to be explained by a few more-extreme deviations in WHIP.

Thus, if WHIP has any advantage over ERA, it is slight; and I prefer to concentrate on the individual metrics. I think using WHIP to examine Clemens’s career is especially problematic because the reduction in walks was largely responsible for his late-career success, and it is his walks that cause his career WHIP to be upside down. I don’t view walks as a good marker for steroid use. Thus, I interpret the same data to support rather than damage the case for Clemens’s performance being natural.

Thanks to Dave for setting this up, and thanks to Justin for participating. It is a pleasure to discuss a disagreement cordially—a rarity on the internet.

If you haven’t followed the debate, here are some relevant links.
Bradbury: What Do the Statistics Say about Roger Clemens’s Steroid Use?
Wolfers, et. al.: Report Backing Clemens Chooses Its Facts Carefully
Bradbury: A Critique of the Clemens Report
Wolfers: Breaking Down the Clemens Report: A Guest Post
Hendricks: Official Clemens Response to the NY Times Article
Wolfers: Analyzing Roger Clemens: A Step-by-Step Guide