Deriving The Regression F Statistic Under The Null

Why the Null F Distribution Has Two Degrees of Freedom Counts

In simple linear regression, the overall significance test asks whether the slope is zero. Its statistic compares variation explained by fitting a line with variation left in the residuals. Under a suitable normal-error model and the null hypothesis, these two sources of variation come from independent chi-square quantities. Their degrees of freedom determine the exact F reference distribution.

The numerator has one degree of freedom because the full model adds one slope parameter beyond the intercept-only model. The denominator has $n-2$ degrees of freedom because fitting the intercept and slope uses two degrees of freedom. The derivation below makes both counts precise and shows why independence matters.

Model and assumptions: For fixed predictor values $x_1,\ldots,x_n$, assume $$ Y_i=\beta_0+\beta_1x_i+\varepsilon_i,\qquad \varepsilon_1,\ldots,\varepsilon_n\ \text{independent }N(0,\sigma^2), $$ where $\sigma^2>0$, $n\geq 3$, and the predictor values are not all equal. The significance null hypothesis is $H_0:\beta_1=0$. The exact finite-sample F result requires the normal, independent, constant-variance errors in this model. If these assumptions fail, the result need not be an exact F distribution.

Regression Sums of Squares and the F Statistic

Let $\bar{x}$ and $\bar{Y}$ be the sample means, and define $$ S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2,\qquad S_{xy}=\sum_{i=1}^n(x_i-\bar{x})(Y_i-\bar{Y}). $$ Because the $x_i$ are not all equal, $S_{xx}>0$. With an intercept in the fitted model, the least-squares slope is $\hat{\beta}_1=S_{xy}/S_{xx}$. The regression sum of squares, residual sum of squares, and overall F statistic are $$ \mathrm{SSR}=\frac{S_{xy}^2}{S_{xx}},\qquad \mathrm{SSE}=\sum_{i=1}^n\bigl(Y_i-\hat{\beta}_0-\hat{\beta}_1x_i\bigr)^2,\qquad F=\frac{\mathrm{SSR}/1}{\mathrm{SSE}/(n-2)}. $$

Here, $\mathrm{SSR}$ measures the additional fitted variation attributed to the slope, beyond fitting an intercept alone. The residual mean square $\mathrm{SSE}/(n-2)$ estimates $\sigma^2$. Under the null, the ratio compares a one-degree-of-freedom estimate of variation with an independent residual estimate of variation.

Theorem: Under the stated model and $H_0:\beta_1=0$, $$ \frac{\mathrm{SSR}}{\sigma^2}\sim\chi^2_1,\qquad \frac{\mathrm{SSE}}{\sigma^2}\sim\chi^2_{n-2}, \qquad \frac{\mathrm{SSR}}{\sigma^2}\ \text{and}\ \frac{\mathrm{SSE}}{\sigma^2}\ \text{are independent}. $$ Consequently, $$ F=\frac{\mathrm{SSR}/1}{\mathrm{SSE}/(n-2)}\sim F_{1,n-2}. $$

Deriving the Independent Chi-Square Quantities

The geometry of least squares gives a direct justification. Regard the observations as a vector $Y=(Y_1,\ldots,Y_n)^\mathsf{T}$. The intercept-only model fits vectors in the direction of the all-ones vector. Adding the slope adds one further direction: the centered predictor vector $x_c=(x_1-\bar{x},\ldots,x_n-\bar{x})^\mathsf{T}$. These two directions are orthogonal because the entries of $x_c$ sum to zero.

The fitted increment from adding the slope is the projection of the centered response onto $x_c$. The residual vector is the part orthogonal to both the intercept direction and $x_c$. Those projection spaces are orthogonal, and their dimensions are one and $n-2$, respectively.

Reduce the response under the null.
When $\beta_1=0$, $Y_i=\beta_0+\varepsilon_i$. Centering removes the intercept, so the centered response has no systematic component in the slope direction.

Standardize the slope-direction projection.
The random variable $S_{xy}$ is a linear combination of independent normal errors. Under $H_0$, it has mean zero and variance $\sigma^2S_{xx}$. Thus $S_{xy}/(\sigma\sqrt{S_{xx}})\sim N(0,1)$, and squaring gives $\mathrm{SSR}/\sigma^2\sim\chi^2_1$.

Count the residual directions.
Removing the intercept and slope directions leaves a residual subspace of dimension $n-2$. The squared length of the normal-error projection onto this space, divided by $\sigma^2$, has a chi-square distribution with $n-2$ degrees of freedom. That squared length is $\mathrm{SSE}/\sigma^2$.

Use orthogonality to obtain independence.
Projections of a multivariate normal error vector onto orthogonal subspaces are independent. Therefore the one-dimensional slope projection and the $n-2$-dimensional residual projection are independent.

Form the F ratio.
For independent $U\sim\chi^2_{\nu_1}$ and $V\sim\chi^2_{\nu_2}$, $(U/\nu_1)/(V/\nu_2)\sim F_{\nu_1,\nu_2}$. Set $U=\mathrm{SSR}/\sigma^2$, $V=\mathrm{SSE}/\sigma^2$, $\nu_1=1$, and $\nu_2=n-2$. The common factor $\sigma^2$ cancels, giving the stated statistic.

This derivation explains why the unknown error variance does not have to be supplied to calculate the test statistic: it appears in both chi-square quantities and cancels in their ratio. It also explains why simply knowing that two sums of squares are chi-square is not enough. The F result depends on their independence as well.

Worked Example: Calculating the F Statistic From Regression Summaries

A hypothetical analyst models the monthly calibration error of a sensor as a function of its operating temperature, using $n=6$ observations and an intercept. The least-squares summaries are $S_{xx}=10$, $S_{xy}=6$, and $\mathrm{SSE}=8$. Find the overall F statistic and its null reference distribution. Assume the fixed-predictor normal-error model stated above.

Method. Calculate the regression sum of squares from $S_{xy}^2/S_{xx}$, then divide its mean square by the residual mean square.

First, $$ \mathrm{SSR}=\frac{S_{xy}^2}{S_{xx}}=\frac{6^2}{10}=\frac{36}{10}=3.6. $$ There is one numerator degree of freedom. The residual degrees of freedom are $$ n-2=6-2=4, $$ so $$ \mathrm{MSE}=\frac{\mathrm{SSE}}{n-2}=\frac{8}{4}=2. $$ Therefore, $$ F=\frac{3.6/1}{2}=1.8. $$ Under $H_0:\beta_1=0$, the reference distribution is $F_{1,4}$. As a consistency check, the total corrected sum of squares is $\mathrm{SSR}+\mathrm{SSE}=3.6+8=11.6$, so the regression and residual components add to the total.

Common mistake avoided: Using $n-1=5$ as the residual degrees of freedom would ignore that the fitted regression estimates both an intercept and a slope. The correct residual count is $6-2=4$.

Worked Applications of the Degrees of Freedom

Worked Example: Checking the Null Chi-Square Scaling

For a hypothetical regression of weekly filter pressure on operating hours, suppose $n=10$, $\mathrm{SSR}=18$, and $\mathrm{SSE}=64$. Find the observed F statistic and state the two chi-square degrees of freedom under the null. The error variance is unknown, and the model assumptions hold.

Method. Use the sums of squares directly for $F$, and use the projection dimensions for the null chi-square variables.

The numerator mean square is $18/1=18$. The residual degrees of freedom are $10-2=8$, so the denominator mean square is $64/8=8$. Hence $$ F=\frac{18}{8}=2.25. $$ Under the null, the scaled regression sum of squares has a $\chi^2_1$ distribution, while the scaled residual sum of squares has a $\chi^2_8$ distribution; these variables are independent. Thus the F reference distribution is $F_{1,8}$.

The sum-of-squares check gives total corrected variation $18+64=82$. The variance $\sigma^2$ is not needed to compute $F$, because the same scale would divide both sums of squares and cancel.

Common mistake avoided: The sample size does not become the numerator degrees of freedom. Only one slope direction is added to the intercept-only model, so the numerator degrees of freedom are one.

Worked Example: Interpreting Orthogonal Components

A hypothetical study relates the fill time of a packaging machine to a fixed pressure setting. There are $n=7$ settings, the model includes an intercept, and a valid fit has $\mathrm{SSR}=12$ and $\mathrm{SSE}=20$. For illustration, suppose the model's error variance is $\sigma^2=4$. Find the two scaled sums of squares, verify the F ratio, and identify its null distribution.

Method. Divide each sum of squares by $\sigma^2$ to display the chi-square quantities, then form their degrees-of-freedom-adjusted ratio.

The scaled regression component is $$ \frac{\mathrm{SSR}}{\sigma^2}=\frac{12}{4}=3, $$ and the scaled residual component is $$ \frac{\mathrm{SSE}}{\sigma^2}=\frac{20}{4}=5. $$ Their null distributions are $\chi^2_1$ and $\chi^2_{7-2}=\chi^2_5$, independently. The F ratio from these scaled components is $$ \frac{3/1}{5/5}=3. $$ Direct calculation agrees: the regression mean square is $12/1=12$, the residual mean square is $20/5=4$, and $12/4=3$. Thus the null reference distribution is $F_{1,5}$. The variance value was supplied only to illustrate the scaling; it is not required for the observed F calculation.

Common mistake avoided: The denominator of the F ratio is not the unadjusted scaled residual sum of squares $5$. It is that quantity divided by its five degrees of freedom, $5/5$.

Assumptions, Interpretation, and an Exam Trap

The exact result is conditional on the predictor values and depends on independent normal errors with the same variance. Under these conditions, the null makes the slope-direction projection central, so its square has a central $\chi^2_1$ distribution. If the errors are not normal, the projection squares are not generally exact chi-square variables in a finite sample. If the errors have unequal variances or are dependent, the stated independence and scaling argument may also fail.

A frequent degrees-of-freedom error is to count the intercept as part of the numerator. The test compares the intercept-only model with the model that adds a slope. Only one parameter is added, so the numerator degrees of freedom are one. The full fitted model estimates two coefficients, giving $n-2$ residual degrees of freedom.

Key takeaway: For the simple linear regression significance test with an intercept, under $H_0:\beta_1=0$ and independent $N(0,\sigma^2)$ errors, the regression and residual sums of squares divided by $\sigma^2$ are independent $\chi^2_1$ and $\chi^2_{n-2}$ variables. Their mean-square ratio has an $F_{1,n-2}$ distribution.

Check Your Understanding

For each problem, assume a simple linear regression with an intercept and the fixed-predictor normal-error assumptions unless the problem says otherwise.

A calibration regression has $n=12$, $\mathrm{SSR}=10$, and $\mathrm{SSE}=50$. Find the F statistic and its null reference distribution. (Answer: $F=2.00$, with reference distribution $F_{1,10}$.)
In a study with $n=9$, what are the degrees of freedom of the independent chi-square quantities $\mathrm{SSR}/\sigma^2$ and $\mathrm{SSE}/\sigma^2$ under a zero-slope null? (Answer: $1$ and $7$, respectively; the reference distribution is $F_{1,7}$.)
A regression has $n=8$, $\mathrm{SSR}=9$, and $\mathrm{SSE}=30$. Calculate its overall F statistic. (Answer: $9/(30/6)=1.80$, with reference distribution $F_{1,6}$.)
For $n=11$, a candidate uses $10$ as the denominator degrees of freedom because the response has 11 observations and an intercept was fitted. What is the correct denominator count, and which additional fitted parameter explains the difference? (Answer: $9=11-2$; estimating the slope as well as the intercept uses two degrees of freedom.)
For $n=6$, suppose the model has independent normal errors but the predictor values are all identical. Can the usual simple-regression slope F test be formed from $S_{xy}^2/S_{xx}$? Explain. (Answer: No. Then $S_{xx}=0$, so the slope is not identifiable and the stated regression sum of squares and F statistic are undefined.)

What You'll Learn