Yeah, I know it’s been done to death, even I’ve already taken one stab at it, but I’d like to look at Buster Olney’s Productive Out Percentage (POP) one more time. Olney first proposed using POP to guage good and bad teams in April of this year. Larry Mahnken has been the chief debuker of the Olney hyothesis. In his analyses (here and here) he has found no support for Olney’s idea, which Tigers hitting coach Bruce Field explains: “That’s how games are won and lost — productive outs, advancing baserunners and getting guys in from third with less than two out.” To measure team’s ability to play “small ball” Olney developed the POP metric, which is defined as to capture this ability. POP is simply the percent of productive outs in productive out situations. According to Olney a productive occurs when:
- A baserunner advances with the first out of an inning
- A pitcher sacrifices with one out
- A baserunner is driven home with the second out of an inning
I admit, that I am a little sympathetic to Olney’s idea. I mean, compared to an out when a runner advances versus one where a runner does not advance, I prefer the former. However, using this one statistic to evaluate the offense of a team all by itself is a bad idea. Olney has tried to sell the statistic as some kind of alternative to the Moneyball strategy of winning baseball games. Since I don’t wish to argue exactly what a Moneyball strategy might be, I’ll just say that Olney takes issue with the belief that OBP and SLG (sometimes united as OPS) are the best way to measure offensive prowess. Well, that is just silly. Anyone with an Excel spreadsheet can run a simple regression of OPS on runs per game by team. Depending which year you choose, OPS will explain between 90-95% of variance in runs scored across teams. Replacing OPS with POP explains a whopping 3%, and it has a negative but insignificant impact on runs.
If there is anything of use in POP it must be in addition to the impact of OBP and SLG, not an alternative measure. Olney’s argument ought to be: all else being equal, teams that have a higher percentage of productive outs will score more runs than those that do not. This means that when two teams have identical OPSs the one with a higher POP will score more runs. So, what happens when I run a regression including both OPS and POP, which allows me to control for the run-scoring abilities of teams due to OBP and SLG, to capture any additional POP effect? Well, not much. Using the 2004 team data provided by ESPN.com I find that POP has no effect on run-scoring. Though the coefficient is negative it is not statistically significant.
OPS 14.168 14.138 (21.84)** (20.76)** POP -2.734 -0.229 (0.78) (0.35) Cons. -5.991 5.66 -5.897 (12.11)**(5.17)**(9.40)** Obs. 30 30 30 R-sq. 0.94 0.03 0.94 Robust t-statistics in parentheses; * significant at 5%; ** significant at 1%
So, why doesn’t it have an effect? I mean, clearly logic dictates that productive outs are preferred to non-productive outs. The problems lies in the fact that productive out situations are also productive at-bat situations. While productive outs are preferred to non-productive outs, non-outs are even better. A team that is producing productive outs is still producing outs. While the time to put away POP has passed, now it’s really time. Let’s hope ESPN decides not to waste resources paying Elias to calculate this statistic for 2005.