I’m pissed, and I cannot contain my anger any longer. Because remaining silent about this ingrained attitude of what constitutes a good argument within the sabermetric community sets a broad precedent for how we as an online community are going to determine what is truth in the game of baseball, this issue must be addressed before we go any further. While I noticed this bizarre clubish backlash against reasonable yet unpopular arguments in the response to Steven Levitt’s critique of sabermetric dogma, I really didn’t realize the full extent of the problem until I became the target. I am being asked to defend an undefendable position, which is not undefendable because my argument is wrong. I’m being asked to produce deductive truth where the the only logical tools we have are induction or abduction. And this standard is wholly unacceptable and must not be tolerated if the quest for knowledge about baseball is to continue.
I must say that the most recent Baseball Primer thread discussing my THT article on DIPS has been the low-light of my association with the online sabermetric community. The general approach of the posters involving sheer ignorance of statistics or the role of the scientific method in determining truth is embarrassing. I cannot prove that there are no flaws with my study anymore that I can prove that God did not create the world in 6 days a little over 5,000 year ago. (And just so I don’t offend anyone, I am a Christian and serious about my faith. I don’t believe the several conflicting accounts of God’s creation of the world should be read literally.) All I can do is offer up the best argument from the data that exists as I wrote in the Primer thread.
Here are some limitations any study of DIPS must face. Take your best shot at working around them. I will gladly applaud your efforts.
Predictions involve a BEFORE and an AFTER. You have to gage the efficiency of predictions from some unit of measurement in the past and in the future. It could be a season, half-season, a moving 3-year average, batter-to-batter, etc. We also have to have a cut-off to allow entry into the study to minimize noise. A pitcher who faces one batter is certainly too small. Facing 1000 innings is too high. There is a tradeoff that must be made when choosing the cutoff. If you don’t like my choice, which is the lowest cutoff of any study on DIPS that I am aware of, pick a new one and go to it. The data is available at www.baseball1.com. Have a field day.
It’s time for my critics to put up or shut up. I have made my case, put some alternative empirical evidence on the table if you wish to claim that there is something biased in my findings and the findings that many others have made. I am trained as an empirical researcher, and I am of an open mind. I only wish to find truth. In fact, one of the reasons I set out to do this study was to verify or reject some previous work that I thought was not statistically rigorous. For example, I have no knowledge that any previous person studying this issue had identified serial correlation in the data. Well, it exists and must be addressed. This is something I did in the study, and it turns out that the past conclusions of those studying DIPS still stand up to this. It would have been wrong for me to simply state “there could be autocorrelation” and end my critique there. Had the results of my reevaluation of DIPS gone differently, I would have been the first person to say, DIPS has to go.
If you find fault with what I have done, demonstrate the errors of my study with empirical data. I will be the first person to link to and praise any study that does so. As Bill James once said about new offensive metrics, “What we really need… is for the amateurs to clear the floor.” I’m throwing down the gauntlet. Get serious or get a life. No more nit picking, no more pretending to understand statistics by merely tossing around relevant statistical terms. I want real hard-core statistical evidence if you would care for me to change my mind. And maybe changing my mind isn’t anything you’re interested in. If not, then don’t criticize my work.
And if you are offended and think I’m being rude? Good! Now I’ve got your attention. I asked politely several times only to be ignored.