Recovering F Tests from an ANOVA Table

Why Recover the F Test From the Table?

A regression ANOVA table summarizes how variation in a response is divided between the fitted regression and its residuals. The overall F test uses that summary to assess whether the predictors, taken together, provide evidence of a linear relationship with the response. In exam problems, the table may omit a mean square, a sum of squares, or a degree of freedom; recovering the test requires knowing how those entries fit together.

This tutorial focuses on calculating and interpreting the overall regression F statistic from an ANOVA table. It is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus. The test is not a test of whether every individual slope is nonzero, and a nonsignificant result does not prove that all slopes equal zero.

Definition: For a multiple linear regression with an intercept and \(p\) predictors, the overall F test evaluates \(H_0:\beta_1=\cdots=\beta_p=0\) against \(H_a:\) at least one slope coefficient is nonzero. The test statistic is the regression mean square divided by the residual mean square.

What the ANOVA Entries Represent

Consider the usual linear regression model with \(n\) observations:

Y_i=\beta_0+\beta_1x_{i1}+\cdots+\beta_px_{ip}+\varepsilon_i,\qquad i=1,\ldots,n.

For the standard exact F test, assume the errors are independent and normally distributed with mean zero and common variance \(\sigma^2\): \(\varepsilon_i\sim N(0,\sigma^2)\). The model includes an intercept, and the design matrix has full column rank. The usual ANOVA partition is based on corrected total variation about the response mean.

Let \(SST\) be the corrected total sum of squares, \(SSR\) the regression sum of squares, and \(SSE\) the residual sum of squares. With an intercept, \(SST=SSR+SSE\). The regression uses \(p\) degrees of freedom, the residual uses \(n-p-1\), and the corrected total uses \(n-1\).

MSR=\frac{SSR}{p},\qquad MSE=\frac{SSE}{n-p-1},\qquad F=\frac{MSR}{MSE}.

Here \(MSR\) and \(MSE\) are the regression and residual mean squares. Under \(H_0\), and under the stated model assumptions, \(F\) has an \(F\) distribution with \(p\) numerator degrees of freedom and \(n-p-1\) denominator degrees of freedom. Evidence against \(H_0\) is in the upper tail: a large regression mean square relative to the residual mean square indicates that the fitted predictors explain substantial variation compared with the remaining variation.

Why the Mean-Square Ratio Has an F Distribution

The ratio is not simply a convenient convention. Under the null hypothesis, the regression terms do not explain systematic variation beyond the intercept-only model. For a full-rank design and normally distributed errors, the fitted regression and residual components correspond to orthogonal projections of the error vector. Their squared lengths, when divided by \(\sigma^2\), have independent chi-square distributions: the regression component has \(p\) degrees of freedom and the residual component has \(n-p-1\).

Scale the sums of squares.
Under \(H_0\), \(SSR/\sigma^2\) has a chi-square distribution with \(p\) degrees of freedom, while \(SSE/\sigma^2\) has a chi-square distribution with \(n-p-1\) degrees of freedom. The two quantities are independent.

Divide each sum of squares by its degrees of freedom.
\(MSR/\sigma^2\) and \(MSE/\sigma^2\) are the corresponding chi-square quantities divided by their degrees of freedom.

Form the ratio.
The common \(\sigma^2\) cancels, leaving the ratio of two independent, degrees-of-freedom-scaled chi-square quantities. By definition, this ratio has an \(F_{p,n-p-1}\) distribution under \(H_0\).

This exact reference distribution depends on the normal-error model and the other stated assumptions. If those assumptions do not hold, the usual finite-sample F test need not have its stated exact distribution. The calculation of the table ratio is still possible, but its conventional significance assessment may not be justified.

A Reliable Calculation Sequence

Read the degrees of freedom before calculating the test statistic. The regression degrees of freedom identify the number of predictors in the overall test; the residual degrees of freedom determine the denominator mean square. If the table gives \(n\) and \(p\), calculate residual degrees of freedom as \(n-p-1\). If a table supplies an incomplete entry, recover it using the definitions of mean square and the ANOVA degrees of freedom.

Key result: For the conventional overall regression test with an intercept, use \(F=MSR/MSE\), with numerator degrees of freedom \(p\) and denominator degrees of freedom \(n-p-1\). At significance level \(\alpha\), reject \(H_0\) when the observed statistic exceeds the upper-tail critical value \(F_{\alpha;p,n-p-1}\).

The notation \(F_{\alpha;p,n-p-1}\) here means the critical value with upper-tail probability \(\alpha\). If a problem supplies a critical value, use the degrees of freedom specified for that value and compare it directly with the observed statistic. If software or a table gives a p-value instead, compare that p-value with \(\alpha\). Keep the degrees of freedom in the stated order: predictor degrees of freedom first, residual degrees of freedom second.

Worked Example: Completing a Table From Sums of Squares

A hypothetical pricing analyst fits a regression with three predictors to 24 observations of a response. The ANOVA table reports \(SSR=180\) and \(SSE=120\). At the 5% significance level, use a supplied critical value of 3.10 to test the overall regression hypothesis.

Method and assumptions. Use the standard intercept-included regression ANOVA test, assuming independent normal errors with mean zero and common variance and a full-rank design. The supplied critical value is for \(3\) numerator and \(20\) denominator degrees of freedom.

Step 1: Find the degrees of freedom. There are \(p=3\) predictors, so the regression degrees of freedom are \(3\). The residual degrees of freedom are \(n-p-1=24-3-1=20\). The corrected total degrees of freedom are \(n-1=23\), which agrees with \(3+20=23\).

Step 2: Calculate the mean squares. The regression mean square is \(MSR=180/3=60\). The residual mean square is \(MSE=120/20=6\).

Step 3: Calculate and assess \(F\). The observed statistic is \(F=60/6=10\). Since \(10>3.10\), reject \(H_0:\beta_1=\beta_2=\beta_3=0\) at the 5% level. The data provide evidence that at least one slope differs from zero.

Check and common mistake avoided. The degrees of freedom add to the corrected total, and \(SSR+SSE=180+120=300\). Dividing \(SSR\) directly by \(SSE\) would give \(180/120=1.5\), which is not the F statistic: both sums of squares must first be divided by their own degrees of freedom.

Worked Example: Recovering Sums of Squares From Mean Squares

A hypothetical benefit study has 30 observations and two regression predictors. Its ANOVA table gives \(MSR=14.4\) and \(MSE=4.8\). At the 5% level, use a supplied upper-tail critical value of 3.35 for the appropriate degrees of freedom. Determine the F statistic and the test conclusion.

Method and assumptions. Apply the overall F test for a full-rank regression with an intercept, under independent normal errors with mean zero and common variance. The stated critical value is for the degrees of freedom obtained below.

Step 1: Find degrees of freedom. The regression degrees of freedom are \(p=2\). The residual degrees of freedom are \(30-2-1=27\), and the corrected total degrees of freedom are \(29\). The check \(2+27=29\) is satisfied.

Step 2: Recover the sums of squares. Since \(MSR=SSR/p\), \(SSR=14.4(2)=28.8\). Since \(MSE=SSE/27\), \(SSE=4.8(27)=129.6\). The table entries are consistent with \(SSR+SSE=158.4\).

Step 3: Calculate and assess \(F\). The ratio is \(F=14.4/4.8=3.00\). Because \(3.00<3.35\), do not reject \(H_0:\beta_1=\beta_2=0\) at the 5% level. The result does not provide sufficient evidence that at least one slope is nonzero; it does not establish that both slopes equal zero.

Check and common mistake avoided. The F statistic is the ratio of mean squares, not a comparison of a mean square with zero and not a comparison of the recovered sums of squares alone. Using the supplied degrees of freedom also avoids looking up a critical value for the wrong reference distribution.

Worked Example: Assessing Significance With a Small Residual Degree of Freedom

A hypothetical risk team fits four predictors to 18 observations. The table gives \(SSR=96.0\) and \(SSE=52.5\). At the 5% level, use a supplied critical value of 3.18 for the appropriate F distribution. Calculate the mean squares and test the overall regression.

Method and assumptions. Use the conventional overall regression test, assuming an intercept, full rank, independent normal errors with mean zero, and common variance. The supplied critical value corresponds to the degrees of freedom determined from the sample and predictor count.

Step 1: Determine degrees of freedom. The regression degrees of freedom are \(4\). The residual degrees of freedom are \(18-4-1=13\). The corrected total degrees of freedom are \(17\), and \(4+13=17\).

Step 2: Calculate the mean squares. \(MSR=96.0/4=24.0\). The residual mean square is \(MSE=52.5/13=4.0384615\ldots\).

Step 3: Calculate and compare the statistic. Thus \(F=24.0/4.0384615\ldots=5.942857\ldots\). Rounded to three decimal places, \(F=5.943\). Since \(5.943>3.18\), reject the overall null hypothesis at the 5% level. There is evidence that at least one of the four slopes is nonzero.

Check and common mistake avoided. The sums of squares add to \(96.0+52.5=148.5\), and the degrees of freedom add to \(17\). The denominator is \(MSE=52.5/13\), not \(52.5/17\): the residual sum of squares must be divided by residual degrees of freedom, not total degrees of freedom.

Reading the Conclusion Carefully

The overall test addresses the predictors jointly. Rejecting \(H_0\) supports the conclusion that at least one slope is nonzero, but it does not identify which predictor is responsible. Conversely, failing to reject means the observed evidence is not strong enough against the joint null at the chosen significance level; it is not proof of no relationship.

A large \(F\) statistic means that the regression mean square is large relative to the residual mean square. It is not, by itself, a measure of practical importance. Statistical significance also depends on the degrees of freedom and the selected significance level. When table entries have been rounded, a statistic calculated from them is approximate; if it is very close to the critical value, avoid implying more precision than the reported table supports.

A frequent exam trap is to use the wrong denominator degrees of freedom, often by forgetting that estimating the intercept uses one degree of freedom. For a model with an intercept, \(n-p-1\) is the residual degrees of freedom. A second trap is to switch the F degrees of freedom: the regression degrees of freedom belong in the numerator, and the residual degrees of freedom belong in the denominator. Always state both before making a significance decision.

Check Your Understanding

For each problem, calculate or recover the required table quantities, then use the supplied critical value to state the conclusion.

A regression with one predictor uses \(n=16\) observations and has \(SSR=25\), \(SSE=70\). At the 5% level, the supplied critical value is 4.60. Find the degrees of freedom, mean squares, F statistic, and conclusion. (Answer: df \(=(1,14)\); \(MSR=25\), \(MSE=5\), \(F=5.00\); reject the overall null hypothesis.)
A model has three predictors and \(n=22\). Its table reports \(SSR=90\) and \(SSE=180\). At the 5% level, use the supplied critical value 3.16. Calculate \(F\) and state the conclusion. (Answer: df \(=(3,18)\); \(MSR=30\), \(MSE=10\), \(F=3.00\); do not reject the overall null hypothesis.)
A regression has two predictors and \(n=25\). The table reports \(MSR=8.75\) and \(SSE=140\). The 5% critical value is 3.44. Find the residual mean square and F statistic, then decide. (Answer: residual df \(=22\); \(MSE=140/22=6.3636\ldots\), \(F=1.375\); do not reject the overall null hypothesis.)
A model with four predictors uses \(n=31\) observations and has \(SSR=200\), \(SSE=260\). At the 5% level, the supplied critical value is 2.74. Reconstruct both mean squares and assess the test. (Answer: df \(=(4,26)\); \(MSR=50\), \(MSE=10\), \(F=5.00\); reject the overall null hypothesis.)
A regression with five predictors has \(n=40\), \(SSR=150\), and \(SSE=340\). At the 1% level, use the supplied critical value 3.41. Give the degrees of freedom, F statistic, and conclusion. (Answer: df \(=(5,34)\); \(MSR=30\), \(MSE=10\), \(F=3.00\); do not reject the overall null hypothesis.)

What You'll Learn