The Logic of the Partial F-Test

Why a Reduction in Residual Error Needs a Benchmark

Adding predictors to a regression model cannot increase its residual sum of squares when the models are fitted to the same outcome and observations and the smaller model is nested within the larger one. The larger model has more freedom to fit the data, so some reduction in residual error can occur even when the added predictors do not capture a meaningful signal in the population. The question is whether the improvement is large relative to the residual variation that remains.

A partial F-test addresses that question for a specified set of added predictors. Its logic has two parts: it averages the improvement in fit across the restrictions being tested, then compares that average with the full model’s residual variation per residual degree of freedom. This makes the test sensitive not just to how much the residual sum of squares falls, but also to how many added coefficient restrictions are being assessed.

The numerator is the reduction in residual sum of squares per restriction. The denominator is the full model’s residual mean square, an estimate of the error variance under the full model. The statistic therefore compares an average improvement in fit with the scale of the residual noise left after fitting the full model. A larger \(F\) means the average improvement is large relative to that residual scale.

What Is Being Tested?

Suppose the full model contains all the terms in the reduced model plus a specified group of additional terms. The null hypothesis says that the added terms satisfy the restrictions defining the reduced model. For example, if the full model adds two predictors with coefficients \(\beta_5\) and \(\beta_6\), the null hypothesis may be \(H_0:\beta_5=0\) and \(\beta_6=0\). The alternative is that at least one of those restrictions does not hold.

The number \(q\) counts independent restrictions, not necessarily the number of words in the hypothesis or the number of variables in a data table. In the example above, \(q=2\). A categorical predictor represented by several indicator coefficients may contribute multiple restrictions, as can a group of polynomial terms. The comparison must be genuinely nested: every mean pattern allowed by the reduced model must also be allowed by the full model.

The division by \(q\) matters. A reduction of 20 units from adding one coefficient and a reduction of 20 units from adding five coefficients do not represent the same average improvement per restriction. The division by \(df_F\) matters as well: a residual sum of squares has no direct scale-free interpretation without accounting for how many residual degrees of freedom remain.

Conditions for the Usual F Reference Distribution

Before interpreting a p-value from the usual partial F-test, state the assumptions that support its reference distribution. The reduced and full models must use the same outcome and the same observations, and the reduced model must be nested in the full model. The restrictions must be independent, so that \(q\) correctly counts their number. The full model’s estimated coefficients must be identifiable; in particular, its design matrix must have full column rank.

For the exact classical F reference distribution, the regression model is assumed to have a correct conditional mean, with independent normally distributed errors of mean zero and constant variance \(\sigma^2\), conditional on the predictor values. Equivalently, the error terms have the same variance and are normal, with independence across observations. In applications, the sampling or study design and residual diagnostics can inform whether these conditions are plausible. If the assumptions are seriously questionable, the usual p-value may not be reliable.

Under the null hypothesis, the reduced model’s restrictions are true. The reduction in fit then reflects random variation rather than a real contribution from the added terms. In that setting, both mean squares in the ratio estimate the same error-variance scale, which explains why their ratio is a natural benchmark. Under the alternative, the added terms can account for systematic variation, increasing the numerator relative to the full model’s residual variation. This is the reasoning behind the test; the exact distributional result requires the stated assumptions.

Worked Examples

A Short Justification of the Mean-Square Comparison

The form of the statistic follows from asking what quantities should be compared. The raw reduction \(RSS_R-RSS_F\) tends to be larger when more restrictions are released, so divide it by \(q\) to express improvement per restriction. The residual sum of squares in the full model also tends to scale with the number of residual degrees of freedom, so divide it by \(df_F\) to obtain the full-model residual mean square. Taking their ratio gives $$ \frac{(RSS_R-RSS_F)/q}{RSS_F/df_F}. $$

Under \(H_0\), neither mean square is expected to reflect a systematic contribution from the added terms; both are on the error-variance scale. If the added terms explain real systematic variation, the numerator tends to be larger relative to the denominator. This reasoning motivates the statistic and its upper-tail test. It does not mean the ratio itself is the probability that \(H_0\) is true; the p-value is calculated from the null distribution of the statistic.

Common Pitfalls and Why the Test Matters

• Using only the raw reduction. A decrease in residual sum of squares is expected when predictors are added. Compare the decrease per restriction with the full model’s residual mean square instead.
• Forgetting \(q\). A block of several added coefficients is not treated as one restriction unless it truly represents one independent restriction. Use the number of independent restrictions in the hypothesis.
• Comparing models fitted to different observations. The residual-sum-of-squares reduction has the intended interpretation only when both models use the same response and sample.
• Calling a nonsignificant result proof of no effect. Failure to reject means the observed statistic is not sufficiently unusual under the null at the selected \(\alpha\); it does not establish that the null is true.
• Reporting a p-value without its context. State which added terms were tested, the degrees of freedom, and the conclusion in the outcome’s context. Statistical evidence of improved fit is not, by itself, evidence of practical importance or causation.