Overhauling Umpires

Between the bad calls made during the MLB’s playoff and SEC football games, officiating has been a popular subject among my friends these days. This has gotten me thinking about umpires. Baseball cannot survive without human umpires; the game requires that snap judgments be made in real time. While replay could be used in spots, I feel that replay won’t fix the human problems that remain. I think fans would prefer that humans got the calls right in the first place so we wouldn’t have to wait for replay corrections.

In general, I think the umpires do a fine job, as I often find my initial disagreements to calls to be wrong after seeing the replay. Training and experience are going to make umpires better referees of play than fans, but that doesn’t mean that umpiring couldn’t be better. So, I decided to do a little exploratory data analysis to see if there is anything we can learn about the state of umpiring in baseball.

Using the Retrosheet game logs and umpire databases, I constructed a file of umpire careers. I found birthdays for only those umpires who worked for a minimum of 10 seasons from 1954 until the present. I would have preferred not to use the career-length cutoff, but the ages had to be entered by hand. Here are some basic stats.

Starting Age: 32
Retirement Age: 52

The data shows that umpires reach the majors in their early-to-mid thirties. I really don’t worry too much about MLB not selecting the best umpires to fill vacancies. Among those who rise to the top, MLB can evaluate and hire the best from that group. It’s who makes up the population from which MLB selects umpires that may be problematic. This means it’s important to understand how umpires make their way to the top.

Umpires begin their careers the minor leagues, where they earn extremely low wages. Minor league umpire salaries range from $1,800 to $3,400 a month, which translates to about $11,000 to $20,000 for six-months of games. I could hardly believe these numbers when I read them, because this is an extremely important age for acquiring skills that will be used for life.

The starting salary for major-league umpires is around $90,000 with top salaries maxing out around $350,000. While MLB umpires are certainly respectable, the probability of earning a big-league promotion are long. At low levels there are umpires covering games for over 200 teams, all vying for slots in the 30-team major league. Not only must you suffer through many years of low pay, but there exists the possibility that even if you excel and do everything right, there won’t be a spot for you when you rise to the top. Thus, the major-league salary probably isn’t high enough for most people to justify enduring the low-wage employment. This means there are two types of people who pursue umpiring as a career: those who love baseball so much that being in the game compensates for the low pay and people with low opportunity costs because they have few other valuable skills.

I suspect that the former group make up a larger pool of the labor supply, but the fact that most umpires don’t see the rewards until their mid-thirties means that the most competent umpires will be tempted to do other things. Maybe that’s not bad for society, because in the grand scheme of things baseball isn’t all that important, but it’s not good for encouraging the best umpires to stick around. Good umpires will have more opportunities outside of umpiring than bad umpires, and will be more likely to leave the profession.

But what happens when umpires make it to the majors? The histogram below plots the frequency of umpire career lengths since 1954.

Umpire Career Length

The mean career length is 20 years, the median is 21 years. The histogram shows that about half of umpires work between 20 and 30 years, and that only 13% work less than 10 years. Basically, once you make it to the big leagues, you stay. This is where the dilemma of evaluating umpires comes in.

One of the things that may keep developing umpires in the minors to suffer low wages is the fact that they expect a 20-year paycheck that averages over $100,000 per year. While strict monitoring at the high levels will improve current umpiring, it also likely discourages future umpires from sticking it out in the minors. If MLB started whacking bad umpires, the expected value of the job will plummet. Certainly, umpires who stay in the game will be more careful in their umpiring, but the impact on the lower level cannot be ignored. Plus, if age takes away some of an umpire’s ability, no matter how hard an umpire works, his performance may decline when he is older to force an early retirement. This may explain why MLB has been reluctant to discipline bad umpires. While I’ve heard commentators blame the umpires union, MLB had no problem breaking it in 1999; which was no surprise given the dearth of available replacements.

Stable employment has value, especially in a field where there are few other options for your skills. If you are fired from your umpiring job at 40, what can you do? You’ve likely spent the past two decades doing nothing but umpiring when you could have been acquiring human capital in other areas. Going back to school or starting at the bottom of the employment ladder are difficult to do at that age. The promise of job security is a key motivator for developing umpires.

This relationship is similar to tenure for college professors. You spend years in school studying tangentially relevant topics in order to put young adults through mental gymnastics for a few years. After many years school, lecturing, and publishing research that is normally read only by three people (the author, editor, and a referee), losing your employment would be devastating. When you apply for jobs, you’re competing with young crop of new professors, who are much more attractive to employers because they are fresh out of top graduate schools where they have been working with the best minds in the field. Tenure is a cheap way to offset the risk of losing a job that has very few alternatives. It allows schools to pay professors relatively low wages, by offsetting the risk of termination.

But we all know the downside of tenure. The young energetic professor slowly transforms into a grumpy apathetic asshole who is late to class but early to receptions. It doesn’t mean that there are not good tenured professors—at least, I hope there are, because I’m in the cohort—but there exists an opportunity for shirking. On the margin, even the best professors shirk a little more than they did before tenure. Do we see this effect among baseball umpires?

Measuring umpire performance over time is more difficult than it is for players, whose performances can be measured objectively from widely-available data. While new technology makes it easier to evaluate umpires than it used to be, the data available to track umpires over a long time-period is more subjective—I can’t tell whether the performances are good or bad, only that they are different from their colleagues. I decided to compare umpires’ strikeout-to-walk ratios to the league average and see how they changed with age. If umpires shirk or become less competent with age, then we should see their K/BB deviations from the average increase over time. The reason I use the deviation from the average is to control for changes in league rules, policies, or informal changes that affect umpire ball-strike calls over time. From 1954–1999, umpires are compared to their own league; from 2000-2008, after the crews merged, they are compared to the major-league average.

Using a fixed-effects least-squares estimation technique that controls for individual umpire tendencies, I estimated the impact of age on the absolute percentage deviation of umpire strikeout-to-walk ratios from the league average over time.* It turns out that every year an umpire ages he increases his deviation from the league average by about 0.8%. That doesn’t seem like a lot—and it really isn’t a huge effect—but over a period of 12 years that pushes the umpire a full standard deviation (9.5%) above/below the average deviation. Thus, by the end of an umpire’s career, his calls are about two standard deviations from the typical deviation. This is evidence of a tenure effect or a loss of competency.

It’s also important to look at how consistent umpire calls are across individuals. The spread above takes into account that umpires have their own tendencies to call balls and strikes (due to the use of fixed effects). The age effect exacerbates any consistent deviation that an umpire has. Below, I list the average K/BB ratios for umpires with more than 300 games behind the plate from 1998–2008, ordered from highest to lowest.

Umpire			K/BB
Doug Eddings		2.82
Bill Miller		2.36
John Hirschbeck		2.29
Phil Cuzzi		2.28
Bruce Dreckman		2.18
Brian Gorman		2.17
Gary Darling		2.16
Angel Hernandez		2.14
Brian O'Nora		2.14
Wally Bell		2.13
Paul Nauert		2.12
Laz Diaz		2.12
Eric Cooper		2.12
Jeff Nelson		2.11
Brian Runge		2.11
Mike Everitt		2.10
Bill Welke		2.09
Larry Vanover		2.08
Bob Davidson		2.07
Terry Craft		2.07
Tony Randazzo		2.07
Ron Kulpa		2.06
Dan Iassogna		2.06
Ted Barrett		2.06
Mike DiMuro		2.05
Tim Welke		2.02
Mike Winters		2.01
Andy Fletcher		2.00
Tim Timmons		2.00
Fieldin Culbreth	2.00
Ed Rapuano		2.00
Paul Emmel		1.99
Marty Foster		1.99
Rob Drake		1.99
Jim Wolf		1.98
CB Bucknor		1.98
Chris Guccione		1.98
Jim Joyce		1.97
Mark Carlson		1.97
Charlie Reliford	1.97
Dale Scott		1.96
Joe Brinkman		1.94
Jim Reynolds		1.94
Bill Hohn		1.94
Larry Young		1.93
Hunter Wendelste	1.93
Marvin Hudson		1.92
Kerwin Danley		1.90
Jerry Meals		1.90
Tim Tschida		1.89
Joe West		1.89
Sam Holbrook		1.88
Jeff Kellogg		1.87
Lance Barksdale		1.87
Larry Poncino		1.87
Bruce Froemming		1.85
Dana DeMuth		1.84
Mark Wegner		1.84
Rick Reed		1.83
Gary Cederstrom		1.83
Mike Reilly		1.83
Ed Montague		1.82
Greg Gibson		1.79
Alfonso Marquez		1.78
Chuck Meriwether	1.78
Jerry Crawford		1.73
Jerry Layne		1.73
Gerry Davis		1.68
Tim McClelland		1.67
Paul Schrieber		1.67
Randy Marsh		1.63
Derryl Cousins		1.60
----------------	----------
Total			1.97

How does the league allow Doug Eddings and Derryl Cousins to keep their jobs? Maybe one of them is enforcing the true strike zone (though, I doubt it), but both of them cannot be doing so—they are seven standard deviations apart. It’s clear that strike zones differ quite a bit by umpire, and this should not be the case. No wonder the umpires are worried about monitoring; they should be, because some of them are not properly enforcing the strike zone. The disparity across umpires also might help explain why veteran pitchers disliked Questec: they had amassed knowledge of umpire tendencies that gave them an advantage over hitters and younger pitchers.

Former commissioner Fay Vincent, who isn’t at all bitter that he was sacked by the owners, recently suggested that MLB should take over the training of umpires to improve umpiring. I don’t think this gets to the root of the problem. Private clinics that produce bad umpires will fail, good ones will succeed. Training is not the root of the problem and isn’t in need of an MLB subsidy.

Rather than spend more money training umpires, I’d urge MLB devote more resources to umpire salaries at all levels. This should be combined with increased monitoring that includes swift termination for bad umpiring. These policies should improve the talent pool from which MLB selects umpires and give existing umpires incentives to properly call the game.

I believe MLB could strengthen its objective rating of umpires through technology and human evaluation. MLB sets the rules of the game and is responsible for making sure that the rules are properly enforced. There is no excuse for the disparity in ball-strike calls across umpires. Doug Eddings should be told to get his calls in line with the rules—that is, unless he’s the lone umpire properly calling balls and strikes—or be fired.

But tighter monitoring cannot be the only step. Monitoring with stiff penalties generates a development problem unless the payoffs for becoming an umpire change. Umpires in the minors may give up working if they see that their expected tenures in MLB will be cut short. There is a simple fix to this problem: pay umpires more at all levels. Maybe even pay some of the bad ones to go away, just as universities do with shirking tenured professors.

At low levels, better compensation will mean more good talent will be attracted to umpiring as a profession, and a good umpire may be willing to stick around longer when his part-time winter employer offers him a raise to go full-time. Higher pay at the MLB level will keep good low-level umpires in the minors in hopes of a long-term payoff as well as increase the opportunity cost of being a bad umpire. Umpires will have an incentive work on improving their skills in order to earn their big paychecks. Overall, the effect of higher pay will permit MLB to fire bad umpires while doing less harm to the talent pool of potential umpires.

Addendum: I just noticed that the umpiring goats of this post-season, Phil Cuzzi and Tim McClelland, are near the top and bottom of the K/BB list.

*The estimate is statistically significant at the 1% level. The model included indicator variables to control for year effects, and I only looked at umpires in seasons when they covered at least 15 games behind the plate. The estimation includes a correction for detected first order serial correlation.

7 Responses “Overhauling Umpires”

  1. Millsy says:

    Really interesting stuff, JC. Great points about the opportunity cost here for umps. From what I know, a lot of them are even college graduates who turned down other job prospects for their chance at baseball.

    I have a quickly rising umpire friend who is currently at the AA level and expects to be paid pretty much dirt for the next 5 years at least (he’s 27). He’s a college graduate and has to deliver pizzas in the off season, missing out on gaining other skills as you say. When he told me even what the best umpires at the MLB level get paid, I was surprised as well. It seems to be a strange way of handling things for such a performance-oriented industry.

    I’m really curious how umpires (who seem to be in a very tight circle with one another) would react to the idea of removing the ‘tenure’ aspect, and rewarding performance. Maybe I’ll shoot my friend this link and see what he thinks. I know from talking with him that in the umpire circle, ‘change’ and ‘innovation’ isn’t exactly a welcome construct.

    It seems like there are a lot of egos–similar to the scouting vs. sabermetric problems–that feel that they really don’t need to be monitored or Questecked, and that adjusting their calls to the attitudes or playing style of the players is good enough reason to not follow the rules exactly.

    Perhaps there is a learning curve with self-adjustment to the rule book and styles of play, rather than performance problems. The longer the tenure, the more willing the umpire is to ‘give a call’ for framing a pitch a certain way or ‘slapping the tag’ correctly? Is there a reward they get from making calls in that way either from the team or the fans? That may not be an excuse, but it could be a factor in your model there.

  2. Jason Graham says:

    Does MLB have any incentive to change the status quo thanks to their antitrust exemption? Yes, the umpiring has been less than ideal during the playoffs but I suspect if there aren’t any egregious errors in the World Series the public outcry will lessen dramatically and the only pressure on MLB will evaporate.

  3. David says:

    Interesting post, J.C. One question – you state, “If MLB started whacking bad umpires, the expected value of the job will plummet.” Might that only be true for bad umpires? A good umpire could see his expected value rise through more opportunities because they are getting rid of bad umpires. Also, getting rid of bad umpires, there would likely be quicker promotions up through the ranks, so it would be easier for umpires to see quickly where they stand. Because of this, there would be less risk – if it took 2 years instead of 10 to see you were not going to make it to the majors, you have lost less. Because of that, the pool of applicants could increase. What do you think?

  4. Marc Schneider says:

    I don’t really understand what the antitrust exemption has to do with it, but I suspect that MLB will increae its use of instant replay and then forget about it. They certainly aren’t going to rush out and pay minor league umpires more money. And I’m not sure MLB has any more (or perhaps less) of a problem with officiating than the NBA, where we have a referee serving time for gambling-related offenses and many people think the refs fix games anyway.

    No doubt the umpiring has become increasingly worse, although the advent of instant replay, especially on balls and strikes, might make it appear worse than it actually is. It’s a bit unreasonable, after all, to expect umps to call all borderline strike/balls correctly. But there is little excuse for the mistakes on the bases.

  5. Ken Houghton says:

    While I agree that Eddings is an outlier (next umpire is more than 0.4 behind him), Derryl Cousins isn’t all that far from the next few people (he’s only 0.03 from Randy Marsh, 0.07 from a couple of human beings).

    There is a better case that Bill Miller (#2 on the hit list) is an outlier (0.07 above #3 Hirshbeck) than that Cousins is.

    That I recognize all of the names except Miller despite not having paid regular attention to a full baseball season in this decade probably demonstrates that you have a solid point. Just questions the idea that both ends are extremes.

  6. toad says:

    I wonder whether it would be good to increase somewhat the number of major league umpires, so as to be able to give more time off during the season. It’s a demanding job, especially at home plate, and weariness could easily affect quality.

    An increased number of major league openings might also create a bit more incentive for minor league umps as there would be more job openings.

    Also, I wonder whether it would be helpful to require umps to spend some off-season time on training, review of games and mistakes, etc. If you raised the salaries you could certainly impose this sort of thing.

Trackbacks/Pingbacks

  1. [...] This post was mentioned on Twitter by J.C. Bradbury, Mick Hunt. Mick Hunt said: Overhauling Umpires http://bit.ly/17VZgK [...]