Pitcher Control Over BABIP, Ceteris Paribus

After writing my article Another Look at DIPS back in May, several readers contacted me to say that I needed to see a few studies at Baseball Prospectus on the subject. I agreed. I was willing to see any study on the subject. However, I’m not a BPro subscriber. It’s nothing personal, I’m just cheap. And though I could have acquired the studies via friends I decided that violated my ethical code regarding intellectual property rights. (And I’m not trying elevate myself to some moral high-ground here, I just have a nasty guilty conscience that I try to avoid.) So, I asked the authors of the studies to send me copies. One did not respond, the other responded as though I had not requested to see the study. This is perfectly acceptable, and I have no problem with it. Why give me something free that others have paid for? I’m all for capitalism and making money. As libertarian and an economist, I can’t complain too much. But, in the interest of seeking truth I want other people to know that I was not just ignoring these studies.

Thankfully Baseball Prospectus is currently running a nice promotion to generate subscriptions by letting potential subscribers see what they are missing. (Maybe they’re just being altruistic…nah!) It just so happens that this has a nice side benefit in allowing me to see these studies. So, just yesterday, I was able to read three studies by Clay Davenport (here) and Nate Silver (here and here).

Both of these studies suffer from the same problem: failure to employ the ceteris paribus assumption. The Latin phrase, which means all other factors remaining constant, is an important caveat in understanding DIPS. DIPS theory is about pitcher control over hits on balls in play as a skill separate from defense-independent statistics. It’s very easy to interpret DIPS as saying just that pitchers have little control over balls in play. That’s almost correct, but not quite. DIPS theory tells us that after controlling for a pitcher’s defense-independent pitching statistics (the holy trinity being: strikeouts, walks, and home runs) knowing a pitchers BABIP tells us very little about a pitcher’s ability to prevent hits. In fact, I find in my study (so did Voros when we last heard from him on the matter) that pitchers do have control over balls in play. It is just that this control is captured in the DIPS metrics. In particular strikeouts are very important predictors of a pitcher’s ability to prevent hits on balls in play. As I wrote,

While pitchers may have some ability to prevent hits on balls in play, the effect is small. And any effect a pitcher does have is reflected within DIPS metrics.

Ceteris paribus, pitchers don’t seem to have any control over balls in play. DIPS is useful not just because it tells us what pitchers do on balls not put into play but because it says something additional about what happens when the ball is in play. So, how does this apply to these other studies.

Davenport looks at the difference in hits on balls in play in the minor leagues. He attempts to see if pitchers who will one day pitch in the majors have control over balls in play that non-major leaguers lack. If so, then maybe having control over hits on balls in play is a very important skill that major league pitchers have. It’s just that once they reach the majors this skill is not observable since it maxes out at the highest level of competition. It’s quite an excellent idea for a study. However, what Davenport concludes from the data he presents does not mean that major league pitchers have more control over balls in play than minor leaguers separate from their DIPS. It is true that at every level of the minors, the major league pitchers seem to have a superior batting average on balls in play by a small amount. However, it’s also true that this same group has more strikeouts, and strikeouts are associated with fewer hits on balls in play. We need multiple regression analysis to see if major league pitchers have more control over BABIP while holding DIPS constant. Otherwise, it is likely the case that the observed differences in BABIP are the product of DIPS not despite them. I don’t have the data, otherwise I’d look into it.

Silver also looks at differences in BABIP, but he focuses on difference types of major league pitchers. He looks at lists of pitcher with excellent change-ups, cureveballs, and fastballs. Surprisingly, all of these types of pitchers have a tendency to have lower BABIPs than expected. Well, again that’s really no surprise. I suspect all of the these guys had good DIPS numbers too.

The moral of the story here is ceteris paribus: holding other factors constant. When trying to determine if control over hits on balls in play is a distinct skill, we much control for the impact of the DIPS metrics that we know do influence this ability. This critique does not render these past studies wrong, but opens the door to further research. I encourage others to continue the work.

7 Responses “Pitcher Control Over BABIP, Ceteris Paribus

  1. Cyril Morong says:

    Maybe this little study I did fits ceteris paribus. I am curious to know what you think.

    I ran a regression where the dependent variable was the difference between a pitcher’s AVG on BIP and the AVG on BIP for the rest of the team while the difference between SO, BB, and HR per BFP were the independent variables. The r-squared was only .05. I looked at the top 50 pitchers in BFP in the NL from 2004. I thought the r-squared might be
    higher, thinking that if SO, BB and HR measure pitching quality. If you do better on that than the rest of your team, I thought you would do
    better on BIP AVG. So that seems to support McCracken. Maybe I need to look at more years.

  2. lisa gray says:

    several things –
    you said – “In particular strikeouts are very important predictors of a pitcher’s ability to prevent hits on balls in play.”

    i am not clear on what you mean here – IF the batter strikes out, THEN the ball can not possibly be in play.

    - i would guess ( sorry – i have NO freaking idea how to get a formula for something like that) that ML pitchers have lower BABIP for 2 reasons
    1) better control of their pitches – so that batted balls get popped up more, or GO more
    2) the fielding abilty of ML fielders (incredibly better) compared to minor leaguers

  3. Vinay Kumar says:

    Lisa, what he means is that the pitchers that strike out more batters also are better at getting batters out on balls in play.

  4. GuyM says:

    I see two big problems with JC’s argument. First, while there is an association between the DIPS stats and BABIP, his own work shows it isn’t that strong. It isn’t the case that a high K rate invariably means a low BABIP; in fact, many low-K pitchers survive in the majors precisely because they had low BABIP. So DIPS will actually yield a less accurate assessment in some cases. When JC says “…pitchers do have control over balls in play. It is just that this control is captured in the DIPS metrics” he is overstating his case. The association btwn K/9 and BABIP is a useful first step, but should not stop us from trying to find better ways to determine a pitcher’s ability to prevent hits on BIP.

    And when JC takes his analysis one step further and makes his case for ceteris paribus, he really gets into trouble. Davenport shows that good minor lg pitchers (those who later make the majors) post better BABIP than bad minor lg pitchers. JC suggests we can’t accept that finding w/o controlling for the 3 DIPS variables: “When trying to determine if control over hits on balls in play is a distinct skill, we must control for the impact of the DIPS metrics that we know do influence this ability.” This is the classic mistake of confusing association with causality. We have no evidence at all that DIPS metrics “influence” the ability. In fact, by definition that cannot be the case (Ks, BBs and HRs cannot become balls in play). DIPS metrics are associated w/ low BABIP, they don’t CAUSE it. To the extent there is an association, we might just as well say that low-BABIP “causes” low K rates.

    If Davenport had, say, attributed all of the ERA advantage enjoyed by MLB-bound minor leaguers to their lower BABIP, w/o accounting for their superiority on Ks, BB, and HRs, then JC’s point would apply. Clearly, all four components contribute to the superior ERAs. But Davenport would only need to control for DIPS variables if they had a causal impact on BABIP, which they do not.

    The fact these variables are associated w/ BABIP (and somewhat w/ each other) tells us that there is some general (though weak) tendency for athletes who are good at one to be good at the others. It probably reflects some underlying skill that impacts them all — i.e. the ability to throw balls with a mix of velocity, location, and movement that prevents hitters from hitting the ball hard. To control for these factors in Davenport’s study is to say: Let’s see if good pitchers have lower BABIP than weak pitchers, but first let’s “control” for the fact that some of these pitchers are better than others.

    Let’s move beyond predicing next year’s ERA and ask the bigger questions: What is pitching talent? What makes a pitcher successful? The evidence provided by Davenport and others shows that preventing hits on BIP is just as much a skill as the other three factors, and the variance in this skill is just as important as the other three in determining how many runs a pitcher allows. JC says that Davenport finds only “small” differences in BABIP among minor leaguers, but neglects to tell us that these differences are just as large as with Ks, HRs, or BBs in terms of the impact on runs allowed, which is what matters.

    DIPS was right about the difficultly in predicting BABIP in any given year, but wrong about the role of preventing hits in pitchers’ success. JC’s argument here serves to further obscure rather than illuminate this point.

  5. JC says:

    I’m sorry, Guy, but you don’t understand the argument. I can’t think of any other way to respond.

  6. josh says:

    We don’t know if all of the information for determining BABIP is captured in the DIPS metrics, we just know that some of it is.

  7. opie says:

    Aaron Chalfin has a study relating to this:

    http://www.geocities.com/ajc730/ERAmodel.htm