Defining Mallows’ Cp

What Mallows’ Cp Measures and What Goes Into Its Definition

When several least-squares regression models are being compared, each model has a residual sum of squares and a number of fitted coefficients. Mallows’ Cp combines these quantities with an estimate of the error variance. Its definition is useful only when the inputs are identified consistently: in particular, the error-variance estimate comes from the full model, not separately from each candidate.

This tutorial defines Cp and shows how to calculate it. It does not interpret Cp as an estimate of prediction error; that requires a separate argument. The calculations below assume the models are fit to the same response observations.

Definition: Suppose $n$ responses are modeled using least squares. Let a candidate model have residual sum of squares $\operatorname{RSS}_k$ and $p_k$ fitted regression coefficients. Let a designated full model have residual sum of squares $\operatorname{RSS}_F$ and $p_F$ fitted regression coefficients. Estimate the error variance using the full model: $$ \hat{\sigma}^2_F=\frac{\operatorname{RSS}_F}{n-p_F}. $$ Mallows’ Cp for the candidate model is $$ C_p=\frac{\operatorname{RSS}_k}{\hat{\sigma}^2_F}-n+2p_k. $$

Here $p_k$ and $p_F$ count regression coefficients, including the intercept when one is fitted. For a categorical predictor represented by indicator variables, count the indicators actually included in the model. The same $n$ must be used for the candidate and full-model residual sums of squares, and $n>p_F$ is required so that the full model has positive residual degrees of freedom.

The full model is the designated larger regression model used to estimate the common error variance. In standard use, candidate models are subsets of its predictors. The least-squares fits should use the same response observations; comparing residual sums of squares from different samples does not produce a like-for-like Cp calculation.

The Regression Setting and the Variance Estimate

A conventional setting for Mallows’ Cp is the fixed-design linear model. For the full model, write $Y=X_F\beta_F+\varepsilon$, where $Y$ is the vector of $n$ responses, $X_F$ is the full-model design matrix, and $\beta_F$ is its coefficient vector. Assume the full design matrix has rank $p_F$, and that the errors have mean zero and common variance with no correlation:

E[\varepsilon]=0,\qquad \operatorname{Cov}(\varepsilon)=\sigma^2 I.

For the usual Gaussian linear model, errors are also normally distributed. Normality is not needed for the unbiasedness calculation below, but it is part of the usual distributional model for exact normal-theory regression inference. If the full model is misspecified, or the errors are heteroscedastic or correlated, the standard variance estimate and the customary Cp framework may not be appropriate without adjustments.

Result: Under the stated mean and covariance assumptions, with a full-rank design and a correctly specified full-model mean, $\operatorname{RSS}_F/(n-p_F)$ is an unbiased estimator of $\sigma^2$. The denominator is the full model’s residual degrees of freedom.

A short derivation explains the denominator. Let $H_F$ be the full-model projection matrix onto the columns of $X_F$, and let $M_F=I-H_F$ be the residual-maker matrix. For a correctly specified full model, the fitted mean is in the column space of $X_F$, so the residual vector is $M_F\varepsilon$. The residual sum of squares is therefore $\varepsilon'M_F\varepsilon$. For a symmetric matrix $M_F$, the expected quadratic form is $\operatorname{tr}(M_F\operatorname{Cov}(\varepsilon))$ when $E[\varepsilon]=0$. Thus:

E[\operatorname{RSS}_F] =E[\varepsilon'M_F\varepsilon] =\operatorname{tr}(M_F\sigma^2 I) =\sigma^2\operatorname{tr}(M_F) =\sigma^2(n-p_F).

The final equality holds because $H_F$ has rank $p_F$, so $M_F=I-H_F$ has rank and trace $n-p_F$. Dividing the expected residual sum of squares by $n-p_F$ gives $E[\hat{\sigma}^2_F]=\sigma^2$. This justifies the full-model variance estimate in the definition; it is not a derivation of Cp’s prediction-error interpretation.

A Reliable Calculation Sequence

Keep the variance estimate, candidate fit, and coefficient count distinct. A useful calculation sequence is:

Confirm the common sample.
Identify $n$ and check that the candidate and full model use the same response observations.

Count full-model coefficients.
Determine $p_F$, including the intercept if present, then calculate $n-p_F$.

Estimate the error variance.
Calculate $\hat{\sigma}^2_F=\operatorname{RSS}_F/(n-p_F)$ using the full model.

Count candidate coefficients and calculate Cp.
Use $p_k$ for the candidate and substitute its own $\operatorname{RSS}_k$ into the Cp formula.

Worked Example: A Candidate With Two Slopes

A hypothetical portfolio study fits a full regression model to $n=18$ policy-level responses. The full model has $p_F=5$ coefficients and $\operatorname{RSS}_F=26$. A candidate model has an intercept and two slopes, with $\operatorname{RSS}_k=34$. Calculate its Cp. Method: estimate variance from the full model, then use the candidate’s coefficient count and RSS. This avoids using the candidate residual mean square as the variance estimate.

Step 1: Full-model residual degrees of freedom. $n-p_F=18-5=13$.

Step 2: Full-model variance estimate. $\hat{\sigma}^2_F=26/13=2$.

Step 3: Candidate coefficient count. The intercept and two slopes give $p_k=1+2=3$.

Step 4: Calculate Cp.

C_p=\frac{34}{2}-18+2(3)=17-18+6=5.

Check: The candidate contains fewer coefficients than the full model, $3<5$, and its RSS is larger, $34>26$, as expected for a least-squares subset fit on the same data. The calculated value is $C_p=5$.

Worked Example: Counting a Categorical Predictor

A hypothetical claims-severity regression uses $n=24$ observations. Its full model has $p_F=6$ coefficients and $\operatorname{RSS}_F=54$. A candidate contains an intercept, one continuous predictor, and a four-level factor represented using three indicator coefficients. The candidate RSS is 69. Find its coefficient count and Cp. Method: count the fitted coefficients explicitly before substituting. This avoids counting a four-level factor as four additional coefficients when it is represented by three indicators.

Step 1: Full-model residual degrees of freedom and variance estimate. $24-6=18$, so $\hat{\sigma}^2_F=54/18=3$.

Step 2: Count candidate coefficients. There is one intercept, one continuous-predictor coefficient, and three factor-indicator coefficients: $p_k=1+1+3=5$.

Step 3: Calculate Cp.

C_p=\frac{69}{3}-24+2(5)=23-24+10=9.

Check: The candidate’s five coefficients do not exceed the full model’s six, and its RSS of 69 exceeds the full-model RSS of 54. The candidate coefficient count is $p_k=5$, and $C_p=9$.

Worked Example: A Smaller Candidate With a Different RSS

For a hypothetical renewal-pricing analysis, a full model uses $n=20$ observations, has $p_F=4$, and has $\operatorname{RSS}_F=32$. A candidate with $p_k=2$ coefficients has $\operatorname{RSS}_k=38$. Find Cp. Method: use the full-model residual mean square for the variance estimate. This example also checks that the candidate RSS is not substituted for the full-model RSS in that estimate.

Step 1: The full-model residual degrees of freedom are $20-4=16$.

Step 2: The full-model variance estimate is $\hat{\sigma}^2_F=32/16=2$.

Step 3: Substitute $n=20$, $p_k=2$, $\operatorname{RSS}_k=38$, and $\hat{\sigma}^2_F=2$:

C_p=\frac{38}{2}-20+2(2)=19-20+4=3.

Check: The candidate has fewer coefficients than the full model, and its RSS is greater than the full-model RSS. The calculation gives $C_p=3$.

Common Trap: Estimating Variance From the Candidate

The most consequential substitution error is to replace $\hat{\sigma}^2_F$ with the candidate model’s residual mean square, $\operatorname{RSS}_k/(n-p_k)$. That is not the definition given here. The candidate RSS belongs in the numerator of the fit term; the full-model RSS and degrees of freedom determine the variance estimate.

Worked Example: Correcting a Candidate-Variance Substitution

A hypothetical model comparison has $n=16$, a full model with $p_F=4$ and $\operatorname{RSS}_F=24$, and a candidate with $p_k=3$ and $\operatorname{RSS}_k=33$. Calculate Cp correctly, then calculate the value produced by incorrectly using the candidate residual mean square. Method: keep the two variance quantities separate. This avoids mistaking an incorrect calculation for the definition merely because the result may sometimes coincide.

Correct calculation. The full-model residual degrees of freedom are $16-4=12$, so $\hat{\sigma}^2_F=24/12=2$. Therefore:

C_p=\frac{33}{2}-16+2(3)=16.5-16+6=6.5.

Incorrect calculation. The candidate residual mean square is $33/(16-3)=33/13$. Substituting it in place of the full-model variance estimate gives:

\frac{33}{33/13}-16+2(3)=13-16+6=3.

This value of 3 is not the Cp defined above; the correct value is 6.5. The correction is to use the full-model estimate, $24/12=2$, rather than the candidate mean square, $33/13$.

What to Verify Before Reporting Cp

A Cp calculation is only as consistent as its model bookkeeping. Before reporting a value, verify that the residual sums of squares come from the same response sample; that the full model has more residual degrees of freedom than zero; and that the counts include all fitted regression coefficients. In particular, an intercept counts when fitted, and a factor contributes the number of indicator coefficients actually used.

Some presentations define a model-size symbol as the number of predictors and then write a separate adjustment for the intercept. Here, $p_k$ and $p_F$ are defined as the numbers of fitted regression coefficients, so the intercept is already included. Do not add another intercept penalty after counting it in $p_k$.

Key takeaway: For this definition, calculate $\hat{\sigma}^2_F=\operatorname{RSS}_F/(n-p_F)$ from the full model, then calculate $C_p=\operatorname{RSS}_k/\hat{\sigma}^2_F-n+2p_k$ for each candidate. Keep the full-model and candidate-model quantities separate.

Check Your Understanding

For each problem, count fitted coefficients carefully and use the full-model variance estimate.

A full model uses $n=14$, has $p_F=4$, and has $\operatorname{RSS}_F=20$. A candidate has $p_k=2$ and $\operatorname{RSS}_k=26$. Find its Cp. (Answer: $C_p=3$)
A full model has $n=30$, $p_F=6$, and $\operatorname{RSS}_F=48$. A candidate with $p_k=4$ has $\operatorname{RSS}_k=62$. Find its Cp. (Answer: $C_p=9$)
A full model has $n=21$, $p_F=5$, and $\operatorname{RSS}_F=32$. A candidate has an intercept, two continuous-predictor coefficients, and a three-level factor represented by two indicators. Its RSS is 46. Find $p_k$ and Cp. (Answer: $p_k=5,\ C_p=12$)
A full model has $n=18$, $p_F=4$, and $\operatorname{RSS}_F=28$. A candidate has $p_k=3$ and $\operatorname{RSS}_k=40$. Find the correct Cp and the value obtained if the candidate residual mean square is incorrectly used as the variance estimate. (Answer: correct $C_p=8$; incorrect value $=3$)
A candidate and a designated full model have different numbers of observations because missing responses were excluded from one fit. Can their residual sums of squares be inserted directly into the same Cp calculation? State the requirement. (Answer: No; the candidate and full model must be fit to the same $n$ response observations for the stated calculation.)

What You'll Learn