A New Level of Relief

Baseball statistics are usually used in one of two ways: analysis of past performance or predicting future performance. In either case, we often use them to analyze the decisions of the manager, the general manager, the owner, and anyone else related to the team. Along those lines, my motivation for tracking pitcher WPA stats was not only to know more about pitching, but also to know which pitchers should be used in certain situations. To that end, I have come up with a statistic that I call Usage Score. While the stat corresponds to specific players, it is actually a reflection of the manager’s tendencies with regard to those players.

(If you haven’t read my introductory article on WPA, you may want to now. Otherwise, this will be difficult to follow.)

The calculation for Usage Score (USG) is relatively simple, although getting to the right numbers to use can be quite a task. First, I take a player’s average total P and WPA per appearance. Next, I find the team’s averages in those categories (not including the player’s stats) during the time period that the player was on the team. With that data, I also have the player’s and team’s WPTP over that time period.

Player Team
Boyer 0.052 0.004 0.068 0.058 -0.011 -0.192
Brower 0.059 -0.007 -0.115 0.074 0.004 0.055
Farnsworth 0.041 -0.002 -0.054 0.020 0.013 0.649
Foster 0.069 0.003 0.047 0.073 -0.010 -0.136
Kolb 0.083 -0.040 -0.477 0.062 0.012 0.192
McBride 0.107 0.094 0.882 0.117 -0.010 -0.084
Reitsma 0.094 0.036 0.388 0.072 -0.001 -0.019

By taking the differences between the player and team averages in these stats, you can learn a few things. First, you know if a player is being used in higher-leverage situations than his teammates by comparing the P values. You also know if a player is doing a better overall job by comparing WPTP rates. Take a look at the Braves’ leaders in average total P and WPTP vs. team averages:

Reitsma 0.021 McBride 0.966
Farnsworth 0.021 Reitsma 0.408
Kolb 0.021 Boyer 0.261
Foster -0.004 Foster 0.183
Boyer -0.006 Brower -0.170
McBride -0.010 Kolb -0.668
Brower -0.016 Farnsworth -0.703

From this table, we might already be able to tell which players are not used at the right times, and we’re now getting close to the USG. Using the player’s WPTP and difference from team average P (WPTP +/- and P +/-), we can find out what the player’s WPA would be if he had been used in the average situation his team faced. Just multiply the player’s WPTP +/- by the team’s average P to find the player’s theoretical WPA +/-.

Now we have the two building blocks for USG: The player’s theoretical WPA +/- and his average total P +/-. The difference between the two is the usage score (I subtract P +/- from WPA +/- so that underused players are at the top). Here is the Braves’ USG leaderboard:

Pitcher USG
McBride 0.1226
Boyer 0.0211
Foster 0.0174
Reitsma 0.0082
Brower 0.0031
Farnsworth -0.035
Kolb -0.062

Theoretically, a manager should use his middle-of-the-pack reliever in situations close to his team’s average P. More crucial situations should go to above-team-average relievers, while garbage time goes to the team’s worst players. In the above leaderboard, you’ll notice that Macay McBride, Blaine Boyer, and John Foster are the top three most “under-used” current relievers, while Kolb and Farnsworth are “over-used.” Sample sizes are a problem with the current staff, since only Reitsma and Kolb have seen more than a handful of appearances, but you get the idea. It appears that Bobby Cox has done a decent job, aside from the anomaly of Dan Kolb, who was clearly not expected to perform so poorly.

I think USG can be very helpful, but there are a few problems with using this as a catch-all manager performance stat, beyond sample size issues. First, I only have data for the Braves, which means that I really have no idea how Cox is doing compared to other managers. All I can say is that he probably should have used John Foster a few times when he actually used Kolb.

Second, as JF said in his comment on Kyle’s article, you just can’t predict the leverage of future situations. You have to make decisions based on the current situation of your staff (things such as fatigue and perhaps hot and cold players) and only the current state of the game. If Cox brings in Chris Reitsma for a 3-2 game in the eighth, who’s to say that he won’t need someone to get Kyle Farnsworth out of a bases-loaded jam in a 3-3 ninth? Of course, the reverse could be true, with the Braves scoring 4 runs to put the game out of reach after Reitsma’s appearance. I don’t know if those scenarios average out, so we’d likely need a guinea pig manager to start bringing in the closer in these early-but-tight situations to see what ends up happening.

All of the stats I’m tracking are still in the early stages of their development. WPA analysis is not widespread, by any means, so it’s very difficult to say anything conclusively when looking from a broad perspective. Any and all comments and suggestions about my work are always welcome. You can visit my site by following JC’s link to “Advanced Pitching Stats for Relievers” on the left.

Comments are closed