A Worked Partial F-Test

Why Work Through a Partial F-Test?

A regression model with more predictors can fit the observed data at least as well as a simpler model, because it has more flexibility. The important question is whether the improvement is large relative to the residual variation that remains. A partial F-test compares two nested models to assess whether the reduction in residual sum of squares is more than would reasonably be expected from random error alone.

The calculation depends on both models’ residual sums of squares (RSS), the number of additional fitted coefficients, and the residual degrees of freedom for the larger model. The p-value then summarizes how unusual an F statistic at least this large would be if the smaller model were adequate. The examples below use hypothetical regression output; they are not reports of real studies.

Models, Assumptions, and the Test Statistic

A reduced model is the smaller model; a full model includes all the terms in the reduced model and may include additional terms. The models are nested when the reduced model can be obtained from the full model by setting its additional coefficient or coefficients to zero. For a meaningful RSS comparison, both models must use the same outcome and the same observations.

Definition: Let $RSS_R$ and $RSS_F$ be the residual sums of squares for the reduced and full models, respectively. Let $q$ be the number of additional coefficients in the full model, and let $df_F$ be its residual degrees of freedom. The partial F statistic is $$ F=\frac{(RSS_R-RSS_F)/q}{RSS_F/df_F}. $$ The numerator is the average reduction in RSS per additional coefficient; the denominator is the full model’s residual mean square. Under the null hypothesis that the added terms do not improve the population regression beyond random error, this statistic has an $F$ distribution with $q$ and $df_F$ degrees of freedom, under the regression assumptions stated below.

For ordinary least squares regression with $n$ observations and $k_F$ estimated coefficients in the full model, including the intercept when present, $df_F=n-k_F$. The numerator degrees of freedom are $q=k_F-k_R$, where $k_R$ is the number of estimated coefficients in the reduced model. Since the models are nested, $RSS_R$ cannot be smaller than $RSS_F$, apart from possible rounding in reported output.

The test relies on the linear regression error assumptions: conditional on the predictors, errors have mean zero, are independent, have a common variance, and are normally distributed. Normality gives the exact finite-sample F reference distribution; independence and constant variance are also essential to the usual variance calculations. In practice, the study design should support independence, and residual plots and other diagnostics should be checked for marked departures from the assumptions. The calculation alone does not verify them.

How the Comparison Works

The numerator measures the improvement in fit after adding terms, scaled by how many coefficients were added. The denominator estimates the error variance using the full model’s residuals. Under the null hypothesis, both quantities estimate the same error variance, so their ratio should generally be near 1. A much larger ratio indicates that the RSS reduction is large relative to the remaining residual variation.

The degrees of freedom follow directly from the calculation: $q$ belongs to the numerator because the RSS reduction is divided by the number of added coefficients, and $df_F$ belongs to the denominator because the full model’s RSS is divided by its residual degrees of freedom.

\text{Under }H_0,\qquad \frac{(RSS_R-RSS_F)/q}{\sigma^2} \text{ and } \frac{RSS_F/df_F}{\sigma^2} \text{ are scaled chi-square quantities,} \qquad F\sim F_{q,df_F}.

This gives a short justification for the test: under the null, the added terms do not account for genuine improvement, so the RSS reduction and the full model’s residual variation both reflect the same error variance. Their scaled ratio has the stated F distribution. The p-value is the upper-tail probability $P(F_{q,df_F}\geq F_{\text{observed}})$. It is not the probability that the null hypothesis is true.

Worked Example: Two Added Predictors

Suppose a hypothetical analysis of 30 clinics models mean waiting time in minutes. The reduced model includes an intercept and weekly patient volume. The full model adds staffing ratio and electronic triage use. The reported RSS values are 52 for the reduced model and 39 for the full model. Assume both models use the same 30 clinics. For this hypothetical example, suppose the study design makes independence plausible and the residual diagnostics show no marked nonlinearity, unequal spread, or severe nonnormality.

Count the added coefficients and residual degrees of freedom. The reduced model estimates two coefficients (intercept and patient volume). The full model estimates four (intercept and three predictors), so $q=4-2=2$. Thus $df_F=30-4=26$.

Calculate the reduction and the full-model residual mean square.

RSS_R-RSS_F=52-39=13, \qquad MSE_F=\frac{RSS_F}{df_F}=\frac{39}{26}=1.5.

The average reduction per added coefficient is $13/2=6.5$. Therefore,

F=\frac{(52-39)/2}{39/26} =\frac{6.5}{1.5} =\frac{13}{3} \approx 4.333.

The reference distribution is $F_{2,26}$. For an F distribution with 2 numerator degrees of freedom and $v$ denominator degrees of freedom, the upper-tail probability at $f$ is $\left(v/(v+2f)\right)^{v/2}$. Here,

P\left(F_{2,26}\geq\frac{13}{3}\right) = \left(\frac{26}{26+2(13/3)}\right)^{13} = (0.75)^{13} \approx 0.0238.

At the 0.05 significance level, the p-value is smaller than $\alpha=0.05$, so we reject the null hypothesis that the added predictors provide no improvement beyond the reduced model. In this hypothetical comparison, the staffing-ratio and electronic-triage terms jointly correspond to a statistically significant reduction in RSS, conditional on the model assumptions. This does not by itself establish that either term is individually significant or that adding them is practically worthwhile.

Worked Example: A Reduction That Is Not Significant at 0.05

A hypothetical model of neighborhood energy use is fitted to the same 24 neighborhoods in both versions. The reduced model estimates five coefficients, and the full model adds one coefficient, giving six estimated coefficients in total. Suppose $RSS_R=66$ and $RSS_F=54$. Assume the observations are independent by design and the residual checks show no serious departure from the linear-model assumptions.

Here $q=6-5=1$, and the full model has $df_F=24-6=18$. The RSS reduction is $66-54=12$, while the full-model residual mean square is $54/18=3$. Substitution gives

F=\frac{(66-54)/1}{54/18} =\frac{12}{3} =4.00.

The reference distribution is $F_{1,18}$. Its upper-tail probability at 4.00 is approximately 0.060. Since $0.060>0.05$, we do not reject the null hypothesis at the 0.05 level. The observed fit improvement is not sufficiently unusual under the null to meet this chosen significance criterion. This conclusion does not show that the added term has exactly no effect; it means the comparison does not provide strong enough evidence against the null at this level.

Worked Example: Two Added Terms With a Different Denominator

Suppose a hypothetical analysis of 23 farms compares a reduced model with an intercept and one predictor against a full model with an intercept and three predictors. Both models use the same farms and outcome. Their RSS values are 64 and 48, respectively. For this example, assume the design and residual diagnostics make the independence, constant-variance, and normal-error assumptions plausible.

The full model estimates four coefficients, so $df_F=23-4=19$—not 18. To make the denominator degree of freedom 18, the full model must instead estimate five coefficients. Suppose the full model includes four predictors plus an intercept, and the reduced model includes an intercept and two of those predictors. The model comparison then has $q=5-3=2$ and $df_F=23-5=18$. We use these latter model sizes for the calculation.

The RSS reduction is $64-48=16$, and the full-model residual mean square is $48/18=2.6667$. Thus

F=\frac{(64-48)/2}{48/18} =\frac{8}{2.6667} =3.00.

For $F_{2,18}$, the upper-tail probability at 3.00 is

P(F_{2,18}\geq 3) = \left(\frac{18}{18+2(3)}\right)^9 = (0.75)^9 \approx 0.0751.

At $\alpha=0.05$, this is not enough evidence to reject the null hypothesis. Note the bookkeeping: $q$ is determined by the difference in the models’ coefficient counts, whereas the denominator degrees of freedom are determined by the full model and sample size. They are not interchangeable.

Interpreting and Reporting the Result

A clear report gives the observed statistic, both degrees of freedom, the p-value, and the conclusion in context. For the clinic example, one suitable report is: “Adding staffing ratio and electronic triage use reduced the RSS from 52 to 39; the partial F-test gave $F(2,26)=4.333$, $p=0.0238$. At the 0.05 level, the comparison provides evidence that the added terms improve fit beyond the reduced model, assuming the regression conditions hold.”

The p-value measures how often a statistic at least as large as the observed value would occur if the null model were adequate and the assumptions held. It is not the probability that the full model is true, nor does it measure the size or practical importance of the improvement. A small p-value can accompany a modest change in prediction, and a larger p-value can arise when the data provide limited precision.

Common Mistakes

Using the wrong denominator degrees of freedom. The denominator uses the full model’s residual degrees of freedom, $n-k_F$, not the reduced model’s degrees of freedom.
Dividing by the wrong number of added terms. The numerator is divided by $q$, the number of additional estimated coefficients, not the total number in the full model.
Comparing different samples. The RSS difference is meaningful for this test when both models describe the same outcome for the same observations.
Using a lower-tail p-value. Large F values indicate a large improvement relative to residual variation, so the p-value is an upper-tail probability.
Claiming the larger model is automatically better. Its RSS cannot increase, but the partial F-test evaluates whether the reduction is large relative to residual variation.
Treating a nonsignificant result as proof of no effect. Failure to reject means the comparison did not provide sufficient evidence at the chosen level; it does not prove the null hypothesis.

Key takeaway: For nested models fitted to the same outcome and observations, calculate the RSS reduction, divide it by the number of added coefficients, and compare that quantity with the full model’s residual mean square. Use $F(q,df_F)$ for the upper-tail p-value, and interpret the result with the regression assumptions and context in view.

Check Your Understanding

Use the definitions and calculation steps above to answer these questions.

A reduced model has RSS 90 and a full model has RSS 72. If the full model adds three coefficients and has 24 residual degrees of freedom, what are the RSS reduction and numerator degrees of freedom?
In a comparison with $n=40$ observations, the full model estimates seven coefficients, including the intercept. What are its residual degrees of freedom?
Why must the reduced and full models use the same outcome and observations for the usual partial F comparison?
What does the p-value $P(F_{q,df_F}\geq F_{\text{observed}})$ describe under the null hypothesis?
If a test has $p=0.08$ and $\alpha=0.05$, state the decision and explain why it does not prove the added terms have no effect.

What You'll Learn