Partial F Tests for Adding Predictors

Why Compare Nested Models?

A regression model with more predictors cannot have a larger residual sum of squares than a model that contains the same predictors plus additional ones: the larger model has at least as much flexibility to fit the observed responses. The key question is whether the reduction is large relative to the unexplained variation remaining in the larger model. A partial F test makes this comparison for two nested regression models, testing whether the added predictor or predictors improve the fit beyond what would be expected from random error.

This test is useful when a model-building procedure considers adding a predictor or a group of predictors. It differs from comparing unrelated models: the reduced model must be obtainable from the full model by setting a specified group of coefficients to zero. The test assesses that particular addition, not whether the full model is best among every possible candidate model.

Models, Hypotheses, and Assumptions

For example, a full model that adds one quantitative predictor has \(q=1\). If it adds a categorical predictor represented by three indicator variables, it has \(q=3\). The relevant number is the number of additional coefficients, not simply the number of variable names.

The usual exact partial F test relies on the classical linear regression model for the full model. Conditional on the predictor values, the response is a linear mean plus errors that are independent, normally distributed with mean zero, and have a common variance \(\sigma^2\). The full model must have full column rank, so its coefficients can be estimated uniquely. The reduced model must be nested in the full model, and both models must use the same response observations. These conditions give the test statistic its stated F distribution under the null hypothesis.

The null hypothesis says that all the added coefficients are zero; the alternative says that at least one is nonzero. If the added coefficients are \(\beta_{k_R+1},\ldots,\beta_{k_F}\), then \(H_0:\beta_{k_R+1}=\cdots=\beta_{k_F}=0\), against \(H_a:\) at least one added coefficient is nonzero. The test is conditional on the predictors already in the reduced model: it asks whether the added terms explain response variation beyond those existing predictors.

The Partial F Statistic

The numerator measures the average decrease in residual sum of squares per added coefficient. The denominator is the full model’s estimate of the error variance, \(RSS_F/(n-k_F-1)\). A large positive value means the reduction per added coefficient is large relative to the remaining residual variation. The p-value is the probability, assuming \(H_0\) and the model assumptions, of obtaining an F statistic at least as large as the observed one. At significance level \(\alpha\), reject \(H_0\) when this p-value is at most \(\alpha\).

Why the Statistic Has an F Distribution

A short derivation explains both degrees of freedom and why the decrease in \(RSS\) is divided by the full model’s residual variance estimate. Let \(P_R\) and \(P_F\) be the projection matrices onto the fitted-value spaces of the reduced and full models. Nesting means the reduced fitted-value space lies inside the full one. Under the null, the true mean lies in the reduced space. For normally distributed errors, the difference in residual sums of squares is the squared length of the component explained by the full model but not by the reduced one:

The matrix \(P_F-P_R\) projects onto the additional part of the full-model space. Its rank is \(q\), the number of added coefficients. Under \(H_0\), therefore, \((RSS_R-RSS_F)/\sigma^2\) has a chi-square distribution with \(q\) degrees of freedom. The full-model residuals occupy a space of dimension \(n-k_F-1\), so \(RSS_F/\sigma^2\) has a chi-square distribution with \(n-k_F-1\) degrees of freedom. The two components are independent because they are projections onto orthogonal spaces, and the errors are normal. Dividing each chi-square quantity by its degrees of freedom and taking their ratio gives the stated F distribution.

Worked Comparisons

Interpreting and Using the Test

A partial F test is a model comparison with a precise scope: it evaluates one specified set of added terms while holding the reduced-model terms in place. A rejection indicates that the added coefficients are not all zero under the model and test assumptions. It does not say which coefficient in a multi-coefficient block is responsible, nor does it establish practical importance. The size and units of the response, the estimated effects, and the purpose of the analysis matter alongside statistical evidence.

For one added coefficient, the partial F test is equivalent to the usual two-sided t test for that coefficient in the full model: the F statistic equals the squared t statistic, with the same p-value. For a block of two or more coefficients, the F test assesses them jointly. It is not obtained by adding up separate t statistics or by deciding from whether every individual coefficient test is significant.

The test is not appropriate for comparing non-nested models, models fit to different response observations, or models with different response variables. Its \(RSS\) comparison relies on nesting and a common response scale. When those conditions do not hold, the difference in RSS does not have the interpretation used in the partial F derivation.

Common Mistakes

• Counting predictor names instead of coefficients. A categorical variable can require multiple coefficients. Use the change in the number of estimated coefficients, \(q\), for the numerator degrees of freedom.
• Using the reduced model’s residual degrees of freedom in the denominator. The denominator estimates error variance from the full model, so its degrees of freedom are \(n-k_F-1\).
• Assuming any RSS decrease is significant. In a nested comparison, the full model’s RSS cannot be larger, but the F statistic evaluates the reduction relative to residual variation and the number of added coefficients.
• Treating failure to reject as proof of no effect. A large p-value is not proof that all added coefficients are exactly zero; it indicates insufficient evidence against their joint null at the chosen level.
• Comparing models that are not nested or do not use the same observations. The partial F formula and its reference distribution depend on the models being nested and fit to the same response data.