On the JQAS Study of Attendance and Winning

The other day, the Wall Street Journal posted a note about an interesting research finding regarding baseball attendance and winning. According to research published in the Journal of Quantitative Analysis in Sports increasing attendance increases the home team’s chance of winning. The finding makes some intuitive sense. If players are motivated and umpires are influenced by a large boisterous crowd, then teams might want to make more of an effort to get fans to the ballpark. I’ve had a few people ask for my opinion of this study, and since I am quite familiar with the paper I will offer a very blunt negative assessment.

When I first saw a draft of this paper as a referee for another journal, I was intrigued by the finding. I reviewed the paper positively, but I still had a few questions that I thought the authors needed to address before the results could be accepted and published according to the quality standards of the journal. A few months later a revised paper was submitted to me, but I was not pleased by the revisions.

In my initial referee report, I suggested using a method that I had used for investigating how umpires are influenced by outside pressure using QuesTec. The authors bizarrely interpreted my suggestion and did something that made little sense. But more importantly, they credited a source that they did not use, and that source happened to be me. Using the citation that I had provided, they stated that in my work I had found that QuesTec monitoring limited racial bias by umpires. I was shocked to read this, because I have done no such research. The authors simply fabricated this. I can only guess their motives: I suspect laziness in an attempt to placate an annoying referee.

If the authors had lied about something so simple and easy to verify, what had they done behind the scene, where numbers can easily be manipulated? I looked at the results more closely, things looked fishy, and the explanations in the text didn’t make much sense. I wrote up my report, in which I stressed the severity of the academic integrity violation and expressed my other concerns about the research. I recommended that the editor reject the paper, and he agreed with my recommendation. In his letter to the authors, he also noted the false citation.

Jump ahead to earlier this year. I stumbled across the paper at JQAS. To my surprise, the paper had been published with the offending text that I had identified in my report. I contacted JQAS editor Ben Alamar to tell him the saga of the paper. I was most upset by the academic dishonesty, but I was also concerned that my work was being cited as finding results that I didn’t find. Dr. Alamar responded promptly and stated that he would discuss the matter with the editorial board.

Soon after I initiated my complaint, I received an e-mail from one of the study’s authors Erin Smith. She apologized to me and stated that the incorrect text should not have referred to my book but to some commentary that I provided on another study of racial bias among umpires in The New York Times. I replied to Ms. Smith that I appreciated her apology; however, this did not explain why the error made it to the JQAS. Ms. Smith was unaware that I was an anonymous referee on her paper, that I was someone who had previously pointed out the error to her, and that I was aware that another journal editor had also pointed out this error to her. Yet, the offending text remained in the paper. Ms. Smith was lying again, and I never received another response from her.

A few weeks later, I received an e-mail from JQAS editor Ben Alamar in which he stated, “I just wanted to let you know that we have finished our review and have rejected the paper based on the incorrect citation of your work.” So, you can imagine my surprise when I read an article on the study in the mainstream media. When I went to the JQAS’s website, I found the paper still published with the following appended.

Erratum
Please note that the following statement has been retracted from Page 4:

“Bradbury (2007) shows racial discrimination is less likely to occur when the umpires are monitored by an electronic pitch tracking system called QuesTec.”

There is no argument in the paper by Bradbury that such discrimination is likely to occur.

The offending passage was removed with explanation; however, this is not what I had been told would be done to rectify the situation. I contacted Dr. Alamar to request an explanation, but he has not yet replied to me.

As you might imagine, I am not particularly happy about this affair. I don’t like academic dishonesty, I don’t like being lied to, and I’d rather spend my time doing other things. The thing that annoys me the most is that none of this should have happened. Ms. Smith could have removed the offending passage, or Dr. Alamar could have just told me that the journal would publish the paper with an erratum. I still wouldn’t think much of the paper or the decision to publish it, but that would be the end of it.

I would also like to note that this is the second time that I have identified serious errors published in JQAS articles (I identified coding irregularities in a paper that claimed to find performance spikes among players included in the Mitchell Report) and nothing was done about it.

5 Responses “On the JQAS Study of Attendance and Winning”

  1. Greg says:

    I can see why you’re upset, and rightly so, but it’s still super lame when anonymous referees force you to cite their own work or the work of their friends. Pad out your resume some other way …

  2. Arthur Kamble says:

    Mr. Bradbury, this post makes you look childish and petty. This seems to be a minor mis-cite; the sort that happens all the time in academia. It is not central to the paper, and the authors to action to correct it.

    The fact that you would write an entire blog post “exposing” the transgression is excessive to an absurd degree. The fact that you would take advantage of your position as an anonymous referee is highly dubious. And the fact that you were using that post to promote your own work is downright shameful.

  3. JC says:

    Greg,

    Citing oneself for vanity is not appropriate. I assure you, I would not have pointed the authors to my work if I did not think it would help them in this case. If you think it is inappropriate for a referee to ever cite his/her own work, I must disagree. Referees are normally selected for their knowledge in a specific area, which is another reason why referees sometimes refer to their own work. In some cases, referees past experiences are helpful to authors. I know that I have benefited from excellent research pointers that may or may not have pointed to the referee’s own work. I try to avoid myself unless in is necessary, and in this case I thought the authors would benefit from an approach that I had developed.

    Arthur,

    Fabricating research, which includes investigating the applicable literature, is not minor in academia. This was not a “minor mis-cite.” The authors claimed to do research that they did not do. If I did this, I would at the minimum be suspended and likely fired from my job. As I pointed out above, the fabricated research indicated to me that the authors were not taking the care necessary to produce good research. Upon investigation, I found the research presentation confusing and unpersuasive, and even without the citation malfeasance I would have recommended rejecting the paper. The attempted deception served to make me more skeptical of the reported findings, as I would suspect it would make most readers wary. But the degree of the violation was ratcheted up when the authors ignored explicit warnings from two separate sources that their deception had been detected. Furthermore—again, as I stated above—the JQAS editor Ben Alamar did not have to reject the paper. He e-mailed me that the paper had been rejected. The fact that a journal editor told me something that turned out not to be true is bothersome. Do you like being lied to? I don’t. Given that other people read this journal, I think it is important for others to know the degree of care taken by the journal to maintain quality. In fact, I have received several e-mails thanking me for reporting this episode. I really don’t like writing such posts, and I considered just blowing this off, but given the public interest in the paper I felt that I ought to comment. If you feel that the violation is minor, and you like the paper, you are free to embrace it.

  4. Marc Schneider says:

    First, JC, condolences for the loss of your father.

    Second, what’s the big deal here? The JQSA article is what’s lame. I don’t see the issue with what JC did.

  5. levi in alaska says:

    I didn’t think JC was being arrogant in this post. If the post is self-promoting, (which I don’t really think it is) who cares? It is HIS blog after all.

    Academic Dishonesty…on a large or small scale…should be identified as such, and the creators of dishonest work are due whatever public shaming they receive.