I am very sympathetic with people who want to develop objective statistics to judge fielders.  While I think much progress has been made, I am still not satisfied with what is available.  One valiant attempt at improving the quantification of fielding is the “zone rating” (ZR).  ZR is a statistic that is like a batting average for fielding.  Every fielder is assigned a zone of responsibility on the field.  The zone is malleable based on the trajectory of the ball.  Fielders have less territory of responsibility for line drives and more responsibility for fly balls, due to the different reaction times needed to get to the ball.  Every ball that passes in that zone is either a putout (PO) or a hit.  We then divide the PO by the Balls in Zone (BIZ).  A balls caught out of the zone (BOZ) are counted as a PO and added to BIZ.  Thus, the formula for ZR is:

ZR = PO/(BIZ+BOZ)

I believe there are a few major problems with this metric that dampen its value.  First, like errors, ZR requires the subjective evaluation of the scorekeeper.  While the standards for ZR are much more objective than for errors, I still worry about he borderline cases.  At what point does a line drive become a flyball?  How can an observer identify the edge of the zone precisely?  Do scorekeepers vary in how they make these judgment calls?  I think that the people at STATS Inc. that calculate the ZR do a pretty good job at solving these problems, but pobody is nerfect.   But, even if we could design an unbiased system, say using satellites tracking a computer chip in the ball, I still think ZR is fundamentally flawed.

  
ZR suffers from two serious problems that are intertwined. First, it cannot handle defensive shifts.  In 1946, Cleveland Indians player-manager Lou Boudreaux designed a unique defense to combat the left-handed pull hitter Ted Williams.  He shaded his fielders to the extreme right of the field.  I am not sure he was the first person to adjust his defense to reflect the batting styles of individual hitters, but it is the most famous case of such a shift being employed.  The shift was used against Williams through much of his career, and since that time managers have adopted strategies of moving fielders to according to the tendencies of batters. For modern day examples see Barry Bonds and Jim Thome.  With the invention of computers, these tendencies have become very easy to measure.  Today, any fan can purchase a scouting guide with the placement of every hit ball for every batter in the big leagues. In terms of ZR, such shifts constitute players moving out of their pre-defined zones.  Even if a player is still standing within the zone, certain balls within the zone become harder to reach, while others outside the zone become outs.

This is where the second problem arises.  Because of the ZR formula treats all PO as BIZ, players can be punished for catching balls out of their zones.  How can fielders be punished when they are credited for POs out of the zone?  Well, because defensive positioning that allows balls to be caught outside of the zone necessitates a tradeoff of other balls falling for hits in the zone.  The POs outside of the zone count for both the numerator and the denominator, which has the effect of expanding the zone of responsibility of the player.  Here is an example of the bias using Andruw Jones’s 2002 fielding statistics.  During the 2002 season Andruw Jones led all NL center fielders with 404 PO.  His ZR was .876, which means that the sum of BIZ plus his BOZ outs was 461.  Let’s see how a few out of zone POs due to positioning may bias the zone rating.  I am going to assume that positioning is going to allow Andruw Jones to catch 10 BOZ; however, the tradeoff for these 10 BOZ is the 10 balls he could have caught had he been in the middle of the zone.  To the Atlanta Braves, the tradeoff means absolutely nothing.  The shift translates into 10 outs gained and 10 outs lost.  But, Andruw’s ZR is going to make Andruw look like a much worse fielder. First, let’s take away his BOZ POs from both the numerator and denominator.

ZR- BOZ  = 394/451 = .874

His ZR is virtually identical, but here is the problem. Andruw converted 10 balls into outs that were not in his zone.  Remember, he is still being punished for not catching the 10 balls that fell in for hits in his zone.  If Andruw played at the center of his zone his ZR would be:

ZR* = 404/451 = .896

That may not seem like much, but that would put him in first place in the NL ZR, rather than sixth place.  However, my assumption about an equal tradeoff between BOZ and BIZ POs is not realistic.  Why would the Braves make such a shift if they gained nothing? In fact, this assumption makes Andruw Jones look even worse.  Let’s assume Andruw is so good that he catches 15 balls out of the zone, while giving up only 10 in the zone. Then, his ZR would have been:

ZR** = 404/446 = .906

My point is not that Andruw Jones was robbed of the ZR crown.  Plenty other center fielders suffer the same fate as Andruw, and it would be unfair for me to target any player as a winner or loser in this system without a detailed analysis of all players.  I simply want to show why ZR are not very reliable, because of all the shifting that goes on in major league baseball.  These shifts are not just the mammoth “Williams Shift,” where all players move to one side of the field.  A few steps forward or back for every hitter is all it takes. And if you watch baseball regularly, you know that shifts are made for almost every player.

 
Although I use an outfielder as an example, I think the problem is just as bad, if not worse, for infielders.  The criteria used for infielders are slightly different (only grounders matter and there are adjustments for double plays), but the general idea is the same.  Infielders adjust their position not just to the tendency of the hitters, but also in response to the runner configuration and game situation.  A runner on first with less than two out sets the infielders at “double-play” depth, which increases the chances that a fielded ball will result in two outs but lowers the probability that a ball will be fielded.  This is just one example.

For these reasons, I don’t pay much attention to ZR or ZR-type systems. There are several other measures of defense used, such as base runs allowed and ultimate zone rating (UZR), that offer some slight improvements, but I am still not sold on these systems.  Unfortunately, measuring defensive ability is hard, especially from a historical perspective.  I am sure that general managers, mangers, and scouts have some methods for measuring fielding quality using drills and other off-field tools, but I don’t think we will ever get a system that can precisely measure fielding ability as well as we can measure batting ability.  I think it is clear that the very good players will have better ZRs than very bad players, but differentiating players of similar ability is much more difficult.