Methods for Predicting Individual Performance
5.1 The Aging Process: A Multilevel Modeling Approach
( )]
( [ ) ( )
(t 0 1AGE t 2 AGE t 2 U t
OBPi =α +α i +α i + i
i = 1,2,…, N; t = 1,2,…, T
where OBPi(t) is the on-base percentage of player i in year t; AGEi(t) is the age of
player i in year t, and Ui(t) is the random disturbance with an unconditional mean of zero.
A Model of Career OBP Profiles
Age
OBP
Figure 5.1: A player's peak age (apex) can be defined as the point where the effect of additional experience is exactly offset by a loss of physical ability. The perpendicular distance of the curve to the x-axis can be interpreted as a function of the individual player's current offensive ability.
5.1 The Aging Process: A Multilevel Modeling Approach
One of the main purposes of this section is to address the question of how much the performance variation among players in general is due to variation within an age year
and how much is accounted for by variation among ages. A player’s ability is thought to generally start at a relatively low level, increase until a particular peak age, and then deteriorate gradually until retirement. The question concerns whether ages “homogenize”
players over their careers with respect to the performance measure in question.
The problem can be restated in terms of whether players of the same age are more alike when compared to players across ages, and whether there is some type of age effect that accounts for that resemblance. In the context of statistical modeling, should
appreciably large portions of variance lie at the level of individual differences within age, then interest would focus on player characteristics that might account for that variance.
Similarly, should appreciably large portions of performance variance remain among ages, then interest would focus on explanatory variables at this level.
This section will address the decomposition of variance in player performance via the method of multilevel modeling. One can think of multilevel models as ordinary regression models that have additional variance terms for handling group membership effects. The key to understanding MLM in this context is understanding how group membership (age) can lead to additional sources of variance in ones model.
The first variance term that distinguishes a multilevel model from a regression model is a term that allows groups to differ in their mean values (intercepts) on the dependent variable. When this variance term,τ00, is non-zero, it suggests that groups differ on the dependent variable. One predicts group-mean differences with group-level variables. These are variables that differ across groups, but do not differ within-groups.
Group-level variables are often called “level-2” variables.
The second variance term that distinguishes a multilevel model from a typical regression model is the term that allows slopes between independent and dependent variables to differ across groups,τ11. Single-level regression models generally assume that the relationship between the independent and dependent variable is constant across groups. If slopes randomly vary, one can attempt to explain this slope variation as a function of group differences within multilevel modeling.
A third variance term is common to both multilevel models and regression models. This variance term,σ2, reflects the degree to which an individual score differs from its predicted value within a specific group. One can think of σ2 as an estimate of within-group variance.
5.1.1 Steps in multilevel modeling: Variance Decomposition
Because multilevel modeling involves predicting variance at different levels, one typically begins a multilevel analysis by determining the levels at which significant variation exists. In the case of the two-level model, one generally assumes that there is significant variation in σ2 – that is, one assumes that within-group variation is present.
One does not necessarily assume, however, that there will be significant intercept variation,τ00, or between-group slope variation, τ11. Therefore, one typically begins by examining intercept variability. If τ00 does not differ by more than chance levels, there may be little reason to use random coefficient modeling since simple OLS modeling will suffice. Note that if slopes randomly vary even if intercepts do not, there may still be
In step 1 of a multilevel analysis, one explores the group-level properties of the outcome variable to determine three things: first, how much of the variance in the outcome can be explained by group membership. Second, whether the group means of the outcome variable are reliable. By convention, one would like the group mean reliability to be around .70 because this indicates that groups can be reliably
differentiated (Bliese, 2000). Third, one wants to know whether the variance of the intercept τ00 is significantly larger than zero.
The initial model that is used to decompose the individual-level variance into
“within-age” and “among-ages” components is equivalent to an unconditional model equivalent to the one-way random effects analysis-of-variance (ANOVA) model. The
“Level-1” model describes the performance for player i who is nested in age j and can be written as
Yij =β0j +rij (5.1) where, for example, Yij could be Derek Jeter’s current OBP at his current age of 33, β0j can be interpreted as the unadjusted true OBP mean for all players 33 years old, and rij is the level-1 random error term reflecting variation among Jeter and all other players 33 years old. The error term, rij is assumed to be normally distributed and homogenous within ages, rij ~ N(0,σ2). The component that accounts for the age effect (the so-called
“Level-2” model) can be written as
β0j =γ00 +µ0j (5.2)
where γ00 is the average performance across all of the ages, andµ0jis the level-2 random component reflecting variation in average OBP around the grand mean of all ages. It is also assumed µ0j~ N(0,τ2) with τ2being the variance of the true means β0jaround the
grand meanγ00. (Raudenbush and Bryk, 2002).
Inserting equation (5.1) into equation (5.2) yields the one-way random effects ANOVA model
Yij =γ00 +µ0j+rij (5.3)
so that the performance of player i of age j is a function of a grand mean performance across all ages (γ00), a component related to the effect of being a particular age (µ0j) and a component related to that player’s individual ability (r ). ij
5.1.2 The Data
The data for this analysis consist of both batters and pitchers. The top 387 players in at-bats from the 2006 season were used which resulted in 2800 player-year
observations. Meaning, for each player, every year of his career is treated as a separate observation with current OBP, career average OBP to date, and age serving as the variables that were recorded. For pitchers, data from the 425 most used pitchers from 2006 were recorded for a total of 2239 player-years. For both sets of players, current
OBP (OBPA for pitchers) will serve as the outcome variable, with the previous season OBP (OBPA) and age as potential explanatory variables at the individual level.
5.1.3 The Intra-Class Correlation
The statistic that we will use to obtain the variance in the individual performance accounted for by differences among ages is the intraclass correlation, written as
2 00
00
σ τ ρ τ
= +
IC
where σ2= var(r ) represents the component of variation among players and ij
τ00= var(µ0j) represents the component of variation accounted for by ages. Offensive players had aρIC= 0.05741239 and pitchers had a ρIC= 0.02652585. These results indicate that there is little variation at the age level for both groups of players, but a large amount of variation at the individual level.
5.1.4 Group-Mean Reliability
When exploring the properties of the outcome variable, it can also be of interest to examine the reliability of the group mean. The reliability of group means often affects one’s ability to detect emergent phenomena. In other words, a prerequisite for detecting changes at the aggregate level is to have reliable group means. By convention, one strives
to have group mean reliability estimates around .70. Group mean reliability estimates are a function of the Intra-Class Correlation and group size (Bliese, 2000).
18 21 24 27 30 33 36 39 42
0.20.40.60.8
Mean Reliability vs. Age Batters
Age Group
Reliability
18 21 24 27 30 33 36 39 42
050100150200250
Number of Observations per Age Group Batters
Age Group
Group Size
Figure 5.2: Group Mean Reliability and Group Size Measures for Batter data.
19 22 25 28 31 34 37 40 43
0.00.20.40.60.8
Mean Reliability vs. Age Pitchers
Age Group
Reliability
19 22 25 28 31 34 37 40 43
050100150200
Number of Observations per Age Group Pitchers
Age Group
Group Size
Figure 5.3: Group Mean Reliability and Group Size Measures for Pitcher data.
As can be seen from the above histograms, the group-mean reliability scores for both sets of players vary primarily according to sample size. Both batters and pitchers have
acceptable scores beginning in the early 20’s age range but their reliability weakens in the early 30’s for pitchers and the mid 30’s for batters. Although certain levels of age lack significant power to detect change at the aggregate level, the overall weighted group-mean reliability scores are 0.8996683 and 0.775374 for batters and pitchers respectively.
These results are above the acceptable level of .70 and indicate that overall trends in the outcome variable can be viewed with a fair amount of confidence.
5.1.5 Determining whether
τ
00 is significantBefore continuing further in this analysis, it would be interesting to know whether the intercept variance (τ00) estimates of .0000464923 and 0.0001727730 for pitchers and batters respectively are significantly different from zero. To do this, we can compare the -2 log likelihood values between a model with a random intercept, and a model without a random intercept. For both groups, the results suggest (p < .0001) that there is significant intercept variation.
In summary, it can be concluded that there is significant variation in terms of OBP across ages for both pitchers and batters. We also estimate that 5.74% and 2.65% of the variation in batters and pitchers OBP (OBPA) is a function of a player’s age. Thus, a model that allows for random variation in OBP among ages is better than a model that does not allow for this random variation.
5.1.6 Adding Predictors
The next logical step in the analysis would be to add predictor variables into the equation. Since it is assumed that past performance is the best predictor of future performance, each batter’s previous year (pitchers) OBP (OBPA) will be used as a predictor of current OBP (OBPA). The form of such a model is
10 1
0 00 0
1
0 ( )
γ β
µ γ β
β β
= +
=
+ +
=
j
j j
ij ij j j
ij OBP r
Y
The first row states that current OBP is a function of each age group’s intercept plus a component that reflects the linear effect of previous year OBP plus some random error.
The second line states that each groups’ intercept is a function of some common intercept and random between-group error. The third line states that the slope between current individual OBP and previous year OBP is fixed – it is not allowed to randomly vary across groups. Stated another way, we assume that the relationship between previous year OBP and current OBP is identical in each group.
When we combine the three rows into a single equation, we get an equation that looks like a common regression equation with an extra error term (µ0j). This error term indicates that the estimated current OBP intercepts (i.e., means) can randomly differ across groups. The combined model is
ij j ij
ij OBP r
Y =γ00 +γ10( )+µ0 +
This constant slope model with previous year OBP performs poorly with respect to its ability to explain away added within-group varianceσ2. This should come as no surprise as the relationship between current OBP and previous OBP changes over the course of a player’s career. Figure 5.4 illustrates this progression. The color changes represent the sample size for each age with lighter colors indicating more observations. For batters, a typical career appears to increase in performance until a peak year around age 27 or 28 and then decrease steadily. For pitchers (where lower OBPA indicates better
performance), players seem to peak at a slightly lower age of 26 or 27 and then decrease in performance, but with a much greater variability than the average batter.
22 24 26 28 30 32 34 36 38 40
0.51.01.52.0
OBP(t)/OBP(t-1) Batters
Age
Relative Change
22 24 26 28 30 32 34 36 38 40
0.60.81.01.21.41.61.8
OBPA(t)/OBPA(t-1) Pitchers
Age
Relative Change
Figure 5.4: Player Performance Progression as a function of age
To account for this variability among age groups with respect to predicting current OBP, another term can be added to the model which allows for variation in slopes.
j j
j j
ij ij j j
ij OBP r
Y
1 10 1
0 00 0
1
0 ( )
µ γ β
µ γ β
β β
+
= +
=
+ +
=
The last line of the model includes the error termµ1jand indicates that the relationship between current OBP and previous year OBP is permitted to randomly vary across age.
The variance term associated withµ1jisτ11and the significance ofτ11is verified after using a log likelihood ratio test between the two models (constant vs. variable slope). As shown in Table 5.1, a greater portion of the variability among batters and pitchers is now
explained by age.
Batters Pitchers
Random Intercept ρIC= 0.0574 ρIC= 0.0265
Random Intercept + Constant Slope ρIC= 0.0597 ρIC= 0.0324 Random Intercept + Random Slope ρIC= 0.1204 ρIC=0.05858
Table 5.1: Model Performance Summaries for Batters and Pitchers
It appears, however, that age does not have the same effect on batters as it does for pitchers. These numbers suggest that age plays a greater role in determining the
performance of a batter than of a pitcher. This claim can also be seen from the boxplots
where both the IQR and range of values across age for pitchers are more varied than for batters.
The idea that age affects player ability is nothing new. Because of various physical processes in the human body, an athlete’s body cannot perform at a peak level for an extended amount of time. The multilevel model used in this analysis confirms the claim that ability varies as a function of age. A drawback to this analysis, however, is it assumes all players within a particular age group, age at the same rate. Because the validity of such an assumption seems questionable, the search for a more flexible, player based approach to the performance versus age relationship should also be explored.