The other day, the Wall Street Journal posted a note about an interesting research finding regarding baseball attendance and winning. According to research published in the Journal of Quantitative Analysis in Sports increasing attendance increases the home team’s chance of winning. The finding makes some intuitive sense. If players are motivated and umpires are influenced by a large boisterous crowd, then teams might want to make more of an effort to get fans to the ballpark. I’ve had a few people ask for my opinion of this study, and since I am quite familiar with the paper I will offer a very blunt negative assessment.
When I first saw a draft of this paper as a referee for another journal, I was intrigued by the finding. I reviewed the paper positively, but I still had a few questions that I thought the authors needed to address before the results could be accepted and published according to the quality standards of the journal. A few months later a revised paper was submitted to me, but I was not pleased by the revisions.
In my initial referee report, I suggested using a method that I had used for investigating how umpires are influenced by outside pressure using QuesTec. The authors bizarrely interpreted my suggestion and did something that made little sense. But more importantly, they credited a source that they did not use, and that source happened to be me. Using the citation that I had provided, they stated that in my work I had found that QuesTec monitoring limited racial bias by umpires. I was shocked to read this, because I have done no such research. The authors simply fabricated this. I can only guess their motives: I suspect laziness in an attempt to placate an annoying referee.
If the authors had lied about something so simple and easy to verify, what had they done behind the scene, where numbers can easily be manipulated? I looked at the results more closely, things looked fishy, and the explanations in the text didn’t make much sense. I wrote up my report, in which I stressed the severity of the academic integrity violation and expressed my other concerns about the research. I recommended that the editor reject the paper, and he agreed with my recommendation. In his letter to the authors, he also noted the false citation.
Jump ahead to earlier this year. I stumbled across the paper at JQAS. To my surprise, the paper had been published with the offending text that I had identified in my report. I contacted JQAS editor Ben Alamar to tell him the saga of the paper. I was most upset by the academic dishonesty, but I was also concerned that my work was being cited as finding results that I didn’t find. Dr. Alamar responded promptly and stated that he would discuss the matter with the editorial board.
Soon after I initiated my complaint, I received an e-mail from one of the study’s authors Erin Smith. She apologized to me and stated that the incorrect text should not have referred to my book but to some commentary that I provided on another study of racial bias among umpires in The New York Times. I replied to Ms. Smith that I appreciated her apology; however, this did not explain why the error made it to the JQAS. Ms. Smith was unaware that I was an anonymous referee on her paper, that I was someone who had previously pointed out the error to her, and that I was aware that another journal editor had also pointed out this error to her. Yet, the offending text remained in the paper. Ms. Smith was lying again, and I never received another response from her.
A few weeks later, I received an e-mail from JQAS editor Ben Alamar in which he stated, “I just wanted to let you know that we have finished our review and have rejected the paper based on the incorrect citation of your work.” So, you can imagine my surprise when I read an article on the study in the mainstream media. When I went to the JQAS’s website, I found the paper still published with the following appended.
Please note that the following statement has been retracted from Page 4:
“Bradbury (2007) shows racial discrimination is less likely to occur when the umpires are monitored by an electronic pitch tracking system called QuesTec.”
There is no argument in the paper by Bradbury that such discrimination is likely to occur.
The offending passage was removed with explanation; however, this is not what I had been told would be done to rectify the situation. I contacted Dr. Alamar to request an explanation, but he has not yet replied to me.
As you might imagine, I am not particularly happy about this affair. I don’t like academic dishonesty, I don’t like being lied to, and I’d rather spend my time doing other things. The thing that annoys me the most is that none of this should have happened. Ms. Smith could have removed the offending passage, or Dr. Alamar could have just told me that the journal would publish the paper with an erratum. I still wouldn’t think much of the paper or the decision to publish it, but that would be the end of it.
I would also like to note that this is the second time that I have identified serious errors published in JQAS articles (I identified coding irregularities in a paper that claimed to find performance spikes among players included in the Mitchell Report) and nothing was done about it.