Why Compare Nested Models?
A multiple regression model can include several predictors, but sometimes the question is not whether the model as a whole is useful. Instead, we may want to know whether a particular set of added predictors improves a model that already contains other variables. The partial F test answers this joint question by comparing two models fitted to the same response values.
For example, a model might already account for baseline severity and age when predicting a later health score. We could ask whether adding sleep duration and attendance information improves the model after those existing predictors are included. The partial F test evaluates the additions together; it does not identify which added predictor, if any, is responsible for the improvement.
The Partial F Statistic
Let \(SSE_R\) be the residual sum of squares from the reduced model and \(SSE_F\) the residual sum of squares from the full model. Each is the sum of the squared differences between observed responses and their fitted values. Let \(K_F\) be the number of coefficients estimated in the full model, including its intercept, and let \(q=K_F-K_R\) be the number of restrictions tested. Equivalently, \(q\) is the number of coefficients added when moving from the reduced to the full model, provided the model is full rank.
The null hypothesis says that all \(q\) added coefficients are zero. The alternative says that at least one is nonzero. The test compares the reduction in residual sum of squares per restriction with the full model's residual mean square.
The numerator degrees of freedom are \(q\), the number of restrictions. The denominator degrees of freedom are \(n-K_F\), the residual degrees of freedom in the full model. Under the null hypothesis and the model assumptions below, \(F\) follows an \(F\) distribution with \(q\) and \(n-K_F\) degrees of freedom.
Because the full model includes every predictor in the reduced model, ordinary least squares cannot give it a larger residual sum of squares: \(SSE_F\leq SSE_R\). The numerator therefore measures how much the fit improves when the \(q\) restrictions are released, on average per added coefficient. The denominator estimates the error variance using the full model. A large \(F\) means the improvement is large relative to the remaining variation.
Model Conditions and Test Procedure
For the usual exact partial F test, assume the response follows a linear regression model with independent errors having mean zero, common variance \(\sigma^2\), and a normal distribution conditional on the predictor values. The models must be correctly specified for the comparison, nested, and full rank; the predictor values are treated as fixed, or the inference is conditional on them. These assumptions support the stated reference distribution. The test is not an exact F test if the models use different observations or if the restrictions do not define a nested comparison.
The hypotheses can be written as \(H_0\): all \(q\) added coefficients equal zero, versus \(H_a\): at least one of those coefficients is nonzero. The p-value is the probability, assuming \(H_0\) and the model conditions, of obtaining an F statistic at least as large as the observed value. The test is an upper-tail test.
Name the reduced and full models, verify that they use the same response and observations, and state which coefficients are restricted under \(H_0\).
Set \(q\) equal to the number of added coefficients and calculate the full model's residual degrees of freedom as \(n-K_F\).
Substitute both residual sums of squares into the partial F formula, keeping the numerator and denominator mean squares separate.
Use the upper-tail \(F(q,n-K_F)\) distribution to find the p-value or compare with a chosen significance level. State what the result says about the added predictors jointly, in context.
Worked Comparisons
Worked Example: Testing Two Added Predictors Together
Suppose a hypothetical data set contains \(n=30\) patients. The response is a symptom score measured after 12 weeks. The reduced model includes an intercept, baseline symptom score, and age. The full model also includes sleep duration and number of clinic visits attended. Assume the usual regression conditions hold for this illustrative comparison. The reduced model has \(SSE_R=180\), and the full model has \(SSE_F=120\).
The full model estimates five coefficients: its intercept and four predictor coefficients. Thus \(K_F=5\), \(q=2\), and the denominator degrees of freedom are \(30-5=25\). The hypotheses are that both added coefficients are zero versus the alternative that at least one is nonzero.
Substituting the values gives
The reference distribution is \(F(2,25)\). Its upper-tail p-value is approximately \(0.0063\). At the \(0.05\) significance level, this is evidence against the joint null hypothesis. In this hypothetical setting, sleep duration and clinic visits together improve the model's fit beyond baseline symptom score and age, under the stated model conditions. The result alone does not establish that both added coefficients differ from zero or that either predictor causes a change in symptoms.
Worked Example: A Small Improvement After One Addition
In another hypothetical analysis, a reduced model predicts a laboratory measurement from two existing predictors. A full model adds one predictor. There are \(n=24\) observations; the full model estimates four coefficients, including its intercept. Suppose \(SSE_R=96\) and \(SSE_F=88\). The test concerns the single added coefficient, so \(q=1\), and the full-model residual degrees of freedom are \(24-4=20\).
The calculation is
For \(F(1,20)\), the upper-tail p-value is approximately \(0.192\). At a \(0.05\) significance level, this comparison does not provide strong evidence that the added predictor improves the model, conditional on the two predictors already present. This is not proof that the coefficient is exactly zero; it means the observed reduction in residual sum of squares is not unusually large under the null model.
Worked Example: Testing Three Added Predictors
A hypothetical model predicts a response score for \(n=40\) participants. The reduced model has an intercept and three predictors. The full model adds three more predictors and estimates seven coefficients in total. Suppose \(SSE_R=240\) and \(SSE_F=180\). Assume the same observations are used in each model and the stated regression conditions are appropriate.
Here \(q=3\), and the denominator degrees of freedom are \(40-7=33\). The hypotheses set all three added coefficients to zero jointly. The statistic is
The upper-tail p-value from \(F(3,33)\) is approximately \(0.022\). Since this is below \(0.05\), the data provide evidence that the three added predictors are useful as a set, conditional on the predictors in the reduced model. The test does not reveal which of the three has a nonzero coefficient; additional analyses would be needed to address individual coefficients.
Why the F Reference Distribution Applies
The test's reference distribution follows from how least squares divides the variation in the response between the added model component and the remaining residuals. Let \(P_F\) and \(P_R\) be the projection matrices onto the spaces spanned by the full and reduced model columns. Nesting means the reduced model space is contained in the full model space. Under \(H_0\), the conditional mean response lies in the reduced model space, so the part of the full-model fit added beyond the reduced model reflects error variation.
With normal errors of variance \(\sigma^2\), the extra sum of squares \((SSE_R-SSE_F)\), divided by \(\sigma^2\), has a chi-square distribution with \(q\) degrees of freedom. The full-model residual sum of squares \(SSE_F\), divided by \(\sigma^2\), has a chi-square distribution with \(n-K_F\) degrees of freedom. These two quantities are independent because they come from orthogonal components of the response. Dividing each by its degrees of freedom and taking their ratio cancels \(\sigma^2\), giving the stated F distribution:
This also explains why the statistic is evaluated in the upper tail: unusually large extra variation explained by the full model, relative to its remaining residual variation, is evidence against the restrictions.
Common Mistake and Interpretation
A frequent error is to compare the two raw residual sums of squares and conclude that the full model is better whenever its \(SSE\) is lower. Adding predictors cannot increase the least-squares \(SSE\), even when those predictors add little useful information. The partial F test accounts for both the number of added coefficients and the full model's residual variation; it asks whether the reduction is large enough to count as evidence against the joint null hypothesis.
A significant result supports the claim that at least one added coefficient is nonzero under the model. It does not show that every added predictor matters, establish causation, or guarantee useful predictions in new data. A non-significant result is likewise not proof that all added coefficients are zero. Interpret the test as evidence about the specified model comparison, not as a blanket verdict on the predictors.
Check Your Understanding
Try these questions before reviewing the worked calculations.
- A reduced model estimates four coefficients and a nested full model estimates seven. What is \(q\), and what are the numerator and denominator degrees of freedom if \(n=35\)?
- Why must the reduced and full models use the same observations for the usual partial F comparison?
- If \(SSE_R=150\), \(SSE_F=126\), \(q=2\), \(n=32\), and \(K_F=6\), calculate the partial F statistic.
- A test of four added predictors is significant. What can you conclude about their coefficients jointly, and what can you not conclude about each coefficient individually?
- Why does a lower full-model residual sum of squares alone not establish that the added predictors improve the model meaningfully?