PrOPS Questions

Over the weekend, Rich Lederer of The Baseball Analysts pointed me to an ESPN story by David Srinivasan (Insider $) about a statistic I developed a few years ago, PrOPS. This led to few comments on the site that I wanted to address. I’ve had numerous conversations about PrOPS since its invention, so I wanted to write a post to bring people up to speed on its development.

First, let me offer a brief introduction. PrOPS (which stands for predicted OPS) is a measure that generates an OPS—on-base percentage(OBP) plus slugging percentage(SLG):mdash;for a player based one a few things that players do. Rather than focus on outcomes on balls in play (hits, outs, etc.) that generate OBP and SLG, PrOPS uses batted-ball types (line drive rate, groundball-to-flyball ratio) and a few other things to generate the typical outcome for a player who hits the ball in this manner.

Now, PrOPS has its origins in my wanting to use batted ball types recently made available by The Hardball Times. In the introductory article on the subject, I used PrOPS to predict which slumping and hot hitters were due for a rise and fall in the 2005 season. The initial numbers were based on one season of data. A few people responded that many of the under-performers were speedy while the over-performers tended to be big and slow. So, I made a minor adjustment to the formula to account for speed. However, the adjustment did very little.

At the end of the season, I wrote a chapter for The Hardball Times Baseball Annual 2006, refining PrOPS using several seasons of data. When including several seasons, I found no relationship between any existing measure of speed and over/under-performance. This doesn’t mean that speed has no impact, but it doesn’t seem to be very important. A few months ago, I posted a summary of the findings.

———

There is a highly statistically significant relationship…between a player’s over/under performance and his decline/improvement. And the greater the the deviation between PrOPS and OPS, the larger the reversion is the following season. For every 0.01 increase/decrease in a player’s over/under performance, his OPS is likely to fall/rise by 0.008 the following season. For example, a player with an OPS 10 “points” above his PrOPS, can expect his OPS to fall by eight points in the following season. That is quite a reversion.

I also generated lists of the top-25 over and under performing season from 2002-2004. And what happened to them?

Of the top 25 over performers, 20 players had lower OPS in the following season.

Of the top 25 under performers, 21 improved their OPS in the following season.

The article also lists the top-25 over and under performers for 2005. What happened to those players in 2006?

Of the over performers, 12 players declined, 7 improved, and 6 did not deviate more than 20 OPS-points from the previous season. Of the under performers, 11 players improved, 7 declined, 3 had no change, and 5 didn’t garner serious playing time. It’s not an air-tight projection system, but there seems to be some information there.

OPS explained approximately 43% of the variance in OPS in the following year, while PrOPS explained about 46%.

PrOPS is not a stand-alone projection tool. You should not look only at a player’s PrOPS and assume it’s exactly what the player should be doing. When I look at it, I also consider the player’s recent hitting history, injuries, aging, and all that other stuff we sometimes use to evaluate hitters. But when I see a player have a career year, and his PrOPS don’t show it, I start to get suspicious.

If you’re curious about the over/under performers of 2006, see The Hardball Times.

NL over performers
NL under performers
AL over performers
AL under performers
——–

Over the weekend I also ran across a new system for measuring luck by Protrade (also see here). It looks promising.

7 Responses “PrOPS Questions”

  1. Kyle S says:

    One of the issues I’ve always had with PrOPS is that the over-performers and under-performers tend to fall into the same categories (and to be the same people in succeeding years).

    The over-performers list is filled with fast line-drive hitters like Ichiro, Hanley Ramirez, and Jose Reyes. The under-performers list, by contrast, is filled with slow sluggers like Andruw Jones, Adam Dunn, and Carlos Delgado. I’m pretty certain that Andruw was one of the top 5 under-performers the last two years, and I suspect that Dunn might have been as well.

    (as an aside, I didn’t look at the AL under-performers list until writing that paragraph, and who should appear on it? Frank Thomas and Jason Giambi are #1 and #2).

    I think PrOPS is a very useful tool, but I believe it is somewhat biased. I don’t know if you’ll look at trying to fix it in the future (I’d understand if you didn’t bother) but it would be neat if ProPS got even better than it is now.

  2. JC says:

    While it might look like there is a pattern here, the numbers don’t show it. There is not a positive correlation between over/under-performance from year to year. If speed, or some other omitted factor was responsible, these players ought to consistently over/under-perform. Over/under-performers tend to decline/improve the following season. Furthermore, while there are some fast guys on the extreme-over list and slow guys on the extreme-under list, both lists include all types of players. And appearing at the extreme more than once is not outside the realm of random chance. Marcus Giles had two amazing over years, but was under in 2006.

  3. Ron says:

    Interesting that the Mets have a number of players high up on the overperformers while the Braves have several on the underperformers.

  4. tangotiger says:

    I am putting all the forecasters up against each other and the fans in a project I’m running on my site. If you want to contribute your PrOPS, we can see how it stacks up to Pecota, Bill James, THT, Zips, Marcel, et al.

  5. JC says:

    PrOPS isn’t a forecasting system. It’s backwards-looking tool. It might have some use in projecting, but I haven’t put much effort in to applying it in this way.

  6. WTE says:

    How do you calculate PrOPS? Is the formula accessible to end-users? Is there a web-site or resource around now that computes PrOPS as a reference?

  7. JC says:

    It is a proprietary system. PrOPS is made available to the public by The Hardball Times (follow the links above).