UPDATE: There have been a lot of visits to this blog entry because of Leo Mazzone’s recent negotiations with the Yankees and Orioles. If you are interested in this study, I urge you to read the follow-up article at The Baseball Analysts. And thanks for visiting!
This is a question I’ve been thinking on long and hard for quite some time, but the recent discussion over Jaret Wright’s imminent departure from the Braves caused me to finally tackle the question. Wright seems to be a classic example of Mazzone’s ability to improve a pitching career. Wright was picked up off the waiver wire from San Diego during the 2003 season. As I recall, though I could be mistaken, I think the Braves were the last team with the chance to claim him; 29 other teams thought he wasn’t worth his measly contract. While he pitched admirably for the Braves in 2003, it was only for a few innings, in 2004 Wright became the ace of the staff for $850K.
Jaret’s turnaround is something Braves fans have come to expect. And since Leo has been with the Braves — halfway through the 1990 season — he’s gotten much of the credit for resurrecting pitchers who seemed to be waning. (He was also the Braves pitching coach briefly in the mid-1980s, but I don’t think that counts for what we consider the era of Leo.) John Burkett, Steve Karsay, Darren Holmes, and the list goes on. And Leo fits the stereotype of the quiet genius, rocking like a metronome when his pitchers are in a tight spot. But that’s not enough for me to prove that Leo lives up to his reputation. What about Albie Lopez, Jason Schmidt, and Jason Marquis? My point is not that Leo is not a good coach, but I want to see what impact he has on the pitchers he coaches. I’m sure not everyone will be better, but I want to see what the overall trend is. To answer this question I decided to quantify Mazzone’s impact on every pitcher he has coached for the Braves in the big leagues — I’m going to ignore his preceding time in the Braves farm system.
So, here’s what I did. I gathered yearly stats for every pitcher Leo ever coached in Atlanta when they were on and off the Braves. Thanks to the new Lahman 5.2 Baseball Archive, this was somewhat easy to do. Using multiple regression techniques, I estimated the impact of Leo’s presence on the pitcher ERA by season for seasons when pitchers had thrown at least 30 innings. To make sure I was not picking up some other things I included several other variables for which the regression techniques can isolate separate impacts:
League ERA: How many runs were pitchers giving up in a particular year? This should account for fluctuations in ERA that have to do with the change in run scoring across both leagues.
Career ERA: How good was the pitcher over his entire career? I want to see what Leo had to work with. Maybe, it’s Schuerholz, not Mazzone, who is identifying good talent.
Age: The aging process is quadratic, meaning pitcher ERA is U-shaped with age, so I included the pitchers age and age squared.
Defense: What if the Braves have had great defenses over the years? This is hard to measure, but I think including the Team’s Defense Efficiency Record (DER) should be sufficient to proxy the quality of the men who play behind the pitcher — DER = (1 – batting average on balls in play). If DIPS/FIP theory is right — I am convinced — that pitchers have close to zero control over balls in play, then variances in DER across leagues should reflect only defense and park factors. Since, correct all of the ERAs in the data for park effects, this is going to be picking up defense.
The Mazzone Effect: I use two methods for capturing Leo’s impact on his pitchers. First, I use an indicator variable =1 when the pitcher has Leo as his coach and 0 otherwise. Second, I use two indicator variables, one for seasons prior to being coached by Leo and one for seasons after. This way, I can see if the effect is the result of new knowledge that Mazzone passes along or the result of day-to-day coaching. Day-to-day coaching may involved identifying and correcting new problems or situational knowledge in when to use or stop using a pitcher.
(All of data are corrected for park effects using the 3-year park factors in The Lahman.)
I present the results below.
[If you want to skip ahead now scroll down below the table. This next paragraph contains some technical details.]
There is a really easy way to interpret the results. The numbers not in parentheses, but next to the variable names represent the effect on ERA. The first two columns report the results using a “random effects” estimator, while the latter two report “fixed effects” results. qxzygdsbuyofasbasdbn < -- Did that last sentence mean as much as this collection of letters? Don't fret. Fixed effects is a technique that attempts to isolate impacts unique to each pitcher. When I do this I must exlcude Career ERA, so that is why the numbers are missing for these results. And to add one last bit of complexity, the data suffer from serial correlation; therefore, I took the appropriate steps to correct for it.
|Number of Players||103||103||93||93|
Absolute value of z statistics in parentheses. * significant at 5%; ** significant at 1%
So, what’s the veridict? Leo Mazzone is a damn good coach! Working with Leo is shaves off between .55 and .85 points of a pitcher’s ERA. And I promise you, the results are not some artifact of some manipulation of the numbers to prove a point. In fact, my bias when I started this project was that Leo was a bit overrated. To put this in perspective, the standard deviation of ERA for pitchers in the sample was 1.36. Leo’s boys gain about half of a standard deviation on their ERA. I think Schuerholz ought to take this number into arbitration hearings with pitchers. Also interesting is the fact that the effect seems to go away when pitchers leave. This may be because Mazzone imparts useful everyday help, not just new knowledge to fix an old problem, or maybe the Braves know when to dump guys. In any event I think it’s clear Rob Neyer was not exaggerating when he suggested Leo Mazzone ought be in the Hall of Fame.
Addendum: Repoz kindly posted a link to this study on Baseball Primer. The discussion is quite interesting and includes not just suggested modifications, but individuals actually doing their own studies to replicate my results and look at other pitching coaches. If my study interests you, I encourage you to read the entire Primer thread. If you were one of those who commented, I offer you a special thank you and congratulations on a job well done.