After writing my article Another Look at DIPS back in May, several readers contacted me to say that I needed to see a few studies at Baseball Prospectus on the subject. I agreed. I was willing to see any study on the subject. However, I’m not a BPro subscriber. It’s nothing personal, I’m just cheap. And though I could have acquired the studies via friends I decided that violated my ethical code regarding intellectual property rights. (And I’m not trying elevate myself to some moral high-ground here, I just have a nasty guilty conscience that I try to avoid.) So, I asked the authors of the studies to send me copies. One did not respond, the other responded as though I had not requested to see the study. This is perfectly acceptable, and I have no problem with it. Why give me something free that others have paid for? I’m all for capitalism and making money. As libertarian and an economist, I can’t complain too much. But, in the interest of seeking truth I want other people to know that I was not just ignoring these studies.
Thankfully Baseball Prospectus is currently running a nice promotion to generate subscriptions by letting potential subscribers see what they are missing. (Maybe they’re just being altruistic…nah!) It just so happens that this has a nice side benefit in allowing me to see these studies. So, just yesterday, I was able to read three studies by Clay Davenport (here) and Nate Silver (here and here).
Both of these studies suffer from the same problem: failure to employ the ceteris paribus assumption. The Latin phrase, which means all other factors remaining constant, is an important caveat in understanding DIPS. DIPS theory is about pitcher control over hits on balls in play as a skill separate from defense-independent statistics. It’s very easy to interpret DIPS as saying just that pitchers have little control over balls in play. That’s almost correct, but not quite. DIPS theory tells us that after controlling for a pitcher’s defense-independent pitching statistics (the holy trinity being: strikeouts, walks, and home runs) knowing a pitchers BABIP tells us very little about a pitcher’s ability to prevent hits. In fact, I find in my study (so did Voros when we last heard from him on the matter) that pitchers do have control over balls in play. It is just that this control is captured in the DIPS metrics. In particular strikeouts are very important predictors of a pitcher’s ability to prevent hits on balls in play. As I wrote,
While pitchers may have some ability to prevent hits on balls in play, the effect is small. And any effect a pitcher does have is reflected within DIPS metrics.
Ceteris paribus, pitchers don’t seem to have any control over balls in play. DIPS is useful not just because it tells us what pitchers do on balls not put into play but because it says something additional about what happens when the ball is in play. So, how does this apply to these other studies.
Davenport looks at the difference in hits on balls in play in the minor leagues. He attempts to see if pitchers who will one day pitch in the majors have control over balls in play that non-major leaguers lack. If so, then maybe having control over hits on balls in play is a very important skill that major league pitchers have. It’s just that once they reach the majors this skill is not observable since it maxes out at the highest level of competition. It’s quite an excellent idea for a study. However, what Davenport concludes from the data he presents does not mean that major league pitchers have more control over balls in play than minor leaguers separate from their DIPS. It is true that at every level of the minors, the major league pitchers seem to have a superior batting average on balls in play by a small amount. However, it’s also true that this same group has more strikeouts, and strikeouts are associated with fewer hits on balls in play. We need multiple regression analysis to see if major league pitchers have more control over BABIP while holding DIPS constant. Otherwise, it is likely the case that the observed differences in BABIP are the product of DIPS not despite them. I don’t have the data, otherwise I’d look into it.
Silver also looks at differences in BABIP, but he focuses on difference types of major league pitchers. He looks at lists of pitcher with excellent change-ups, cureveballs, and fastballs. Surprisingly, all of these types of pitchers have a tendency to have lower BABIPs than expected. Well, again that’s really no surprise. I suspect all of the these guys had good DIPS numbers too.
The moral of the story here is ceteris paribus: holding other factors constant. When trying to determine if control over hits on balls in play is a distinct skill, we much control for the impact of the DIPS metrics that we know do influence this ability. This critique does not render these past studies wrong, but opens the door to further research. I encourage others to continue the work.