Phil Birnbaum has a new theory as to why I’m wrong (I suspect it won’t be his last), and he links to others who think I’ve made the same mistake.
This time, my sample is the problem. By choosing a sample of players from 24 — 35 with a minimum of 10 seasons played and 5,000 plate appearances, this “biases” the estimates because I’ve chosen a sample that excludes people with short careers. To demonstrate this, Birnbaum simulates a new world to show that sample choice can affect estimated average peaks. This is irrelevant to what I have done, and shows a serious lack of understanding of the technique I employed. I’m not taking mean of the sample to calculate a peak, I’m estimating an aging function using a common-yet-sophisticated technique designed to see how changing factors of many units over time affect an outcome. Because of the way the technique works, the sample won’t bias a peak estimate as suggested.
This fact is easily seen in the graphs presented in the paper. Below, I post the aging functions for strikeouts per nine innings and walks per nine innings for pitchers on a single graph. The functions have their peaks (denoted by vertical dotted lines) at the opposite ends of the sample—9 years apart. This is a curious finding that I discuss in the paper.
Why didn’t they center around the middle of the sample, or why weren’t strikeouts biased upwards due to the career requirements? Just because the sample is pared down doesn’t mean that the technique will be biased one way or the other. The people in the sample still age like normal human beings. The technique captures how these individuals age in accordance with the aging process by looking at how players’ performances change. Strikeouts are shown to peak early because the players in the sample strike out fewer batters as they age—the function actually peaks outside the sample range at 23.56. All of this confusion could be cleared up with an introductory econometrics course.
In any case, it’s always possible that estimates might be sensitive to sample selection. These critiques tend to focus on what could be rather than what is. In the paper, I explain the reasoning for my sample choice by highlighting potential selection bias problems. As I said in the comments: “Including a list of players who played 10 years or more allows for the smoothing of random fluctuations over time, because we don’t have to worry about players being dropped in and out of the sample. More importantly, it allows for identifying a career baseline for each player from which we can observe how he progresses. It certainly shouldn’t perform worse than the average-yearly-change method.” I did not make this choice lightly. My cutoffs were chosen because they fit with cutoffs used by other researchers and I tested the models for sensitivity to cutoff decisions.
If you’re not convinced, why don’t we move from the hypothetical to reality. The graph below maps the aging function estimated on the low threshold of 1000 career plate appearance and 300 plate appearances in a season. No more age range, no career-length limit, and a vastly-reduced history of performance. And guess what? Peak age is 29.