How to Read a Regression ANOVA Table
A regression ANOVA table organizes the variation in a response into variation explained by the fitted regression and variation left in the residuals. In a calculation problem, some entries may be provided while others must be recovered. The reliable approach is to identify the model and its degrees of freedom first, then use the sum-of-squares identities, and only then calculate mean squares and F statistics.
Assume a regression model with an intercept, \(p\) predictors, and \(n\) observations. The design matrix has full column rank. For inference, also assume errors are independent with mean zero and common variance \(\sigma^2\); exact F-test distributions additionally require normally distributed errors. The identities used to complete the table are OLS identities for a model with an intercept. The normality assumption is not needed for those arithmetic identities, but it is needed for the usual exact finite-sample F tests.
A mean square is its sum of squares divided by its degrees of freedom. Thus \(MSR=SSR/p\) and \(MSE=SSE/(n-p-1)\). The overall regression F statistic is \(F=MSR/MSE\), for testing the joint null hypothesis that all \(p\) slope coefficients are zero. The coefficient of determination is \(R^2=SSR/SST=1-SSE/SST\), provided \(SST>0\).
Why the Sums of Squares Add
The identity \(SST=SSR+SSE\) follows from the orthogonality of OLS residuals to fitted values in a model with an intercept. For each observation, write \(y_i-\bar{y}=(\hat{y}_i-\bar{y})+(y_i-\hat{y}_i)\). Squaring and summing gives a cross-product term as well as the two sums of squares:
The cross-product is zero. In OLS, the residual vector is orthogonal to every column of the design matrix, including the intercept column and the fitted-value vector. Since the fitted values sum to \(n\bar{y}\), the centered fitted values also have zero inner product with the residuals. Therefore the cross-product vanishes, leaving \(SST=SSR+SSE\). This is why the identity applies to ordinary least squares with an intercept; without an intercept, the usual centered total sum-of-squares decomposition need not hold.
Worked Example: Completing an Overall Regression Table
Worked Example: Completing an Overall Regression Table
A hypothetical insurer models a claim-processing time using \(p=3\) predictors and \(n=24\) observations. The fitted model has \(SSR=180\) and \(SSE=120\). Complete the sums-of-squares and degrees-of-freedom entries, then calculate the mean squares, \(R^2\), and overall F statistic. Use the usual regression model assumptions stated above for the F-test interpretation.
Method. First determine degrees of freedom from \(n\) and \(p\). Then calculate total variation, mean squares, and the requested ratios.
Degrees of freedom. Regression degrees of freedom are \(p=3\). Error degrees of freedom are \(n-p-1=24-3-1=20\). Total degrees of freedom are \(n-1=23\). The check \(3+20=23\) confirms that the degrees of freedom add correctly.
Total sum of squares. By the decomposition, \(SST=SSR+SSE=180+120=300\).
Mean squares. \(MSR=180/3=60\). Also, \(MSE=120/20=6\).
Explained proportion. \(R^2=SSR/SST=180/300=0.6000\), or 60.00%.
Overall F statistic. \(F=MSR/MSE=60/6=10\). Under the joint null hypothesis, the reference distribution is \(F_{3,20}\).
Completed table and check. The regression, error, and total sums of squares are \(180\), \(120\), and \(300\); their degrees of freedom are \(3\), \(20\), and \(23\); and the regression and error mean squares are \(60\) and \(6\). The checks \(180+120=300\) and \(60/6=10\) agree. The common mistake avoided here is dividing the error sum of squares by \(n-1\); \(SSE\) must be divided by the error degrees of freedom, \(n-p-1\).
Recovering Entries from R-Squared
When \(SST\) and \(R^2\) are known, the sums of squares can be found directly:
These formulas are equivalent to the definition of \(R^2\) and the decomposition \(SST=SSR+SSE\). They are useful when a table gives a proportion explained but omits both component sums of squares. Treat a reported rounded \(R^2\) cautiously: calculations based on a rounded value may not reproduce the original sums of squares exactly.
Worked Example: Recovering Sums of Squares from R-Squared
A hypothetical regional risk team fits a model of annual claim counts using \(p=4\) predictors for \(n=30\) observations. The table reports \(SST=500\) and \(R^2=0.72\). Complete the sums of squares, degrees of freedom, mean squares, and overall F statistic.
Degrees of freedom. Regression degrees of freedom are \(4\), error degrees of freedom are \(30-4-1=25\), and total degrees of freedom are \(30-1=29\). They check because \(4+25=29\).
Regression sum of squares. \(SSR=R^2SST=0.72(500)=360\).
Error sum of squares. \(SSE=(1-R^2)SST=(1-0.72)(500)=0.28(500)=140\). The decomposition checks: \(360+140=500\).
Mean squares. \(MSR=360/4=90\). \(MSE=140/25=5.6\).
Overall F statistic. \(F=90/5.6=16.071428\ldots\). Rounded to four decimal places, \(F=16.0714\), with reference distribution \(F_{4,25}\) under the joint null.
Check and trap. The recovered \(R^2\) is \(360/500=0.72\), as reported. A common error is to calculate \(SSE=R^2SST=360\). That assigns the explained proportion to the residual variation. The correction is to use \(SSE=(1-R^2)SST=140\).
Nested-Model Comparisons in an ANOVA Table
A partial F test compares a reduced model with a full model fitted to the same response observations. The reduced model is nested in the full model, meaning its predictors or restrictions form a subset of those in the full model. Let the models have \(p_R\) and \(p_F\) predictors, where \(p_F>p_R\), and let their error sums of squares be \(SSE_R\) and \(SSE_F\). The number of added predictors is \(m=p_F-p_R\). For nested OLS models, \(SSE_R\geq SSE_F\).
The reduction in error sum of squares is the additional regression sum of squares attributable to the added terms. Its mean square is compared with the full model's error mean square:
The numerator degrees of freedom are \(m\); the denominator degrees of freedom are the full model's error degrees of freedom, \(n-p_F-1\). Under the usual normal linear-model assumptions and the null hypothesis that the added terms have zero coefficients, this statistic has an \(F_{m,n-p_F-1}\) distribution.
Worked Example: Reconstructing a Nested-Model Comparison
A hypothetical benefits administrator compares a reduced model with \(p_R=2\) predictors against a full model with \(p_F=5\) predictors, using the same \(n=40\) observations. The reduced model has \(SSE_R=150\), the full model has \(SSE_F=96\), and the total sum of squares is \(SST=600\). Calculate the added-term partial F statistic and the overall F statistic for the full model. Assume the usual conditions for exact F inference.
Degrees of freedom. The full model has error degrees of freedom \(40-5-1=34\), and total degrees of freedom \(39\). Its regression degrees of freedom are \(5\). For the nested comparison, \(m=5-2=3\) terms are added. The reduced model has error degrees of freedom \(40-2-1=37\).
Partial F statistic. The reduction in error sum of squares is \(150-96=54\). The numerator mean square is \(54/3=18\). The full-model error mean square is \(96/34=48/17=2.823529\ldots\). Therefore,
The partial test uses \(3\) numerator and \(34\) denominator degrees of freedom, so its reference distribution is \(F_{3,34}\).
Full-model overall F statistic. The full-model regression sum of squares is \(SSR_F=SST-SSE_F=600-96=504\). Thus \(MSR_F=504/5=100.8\), while the full-model \(MSE\) is \(96/34=48/17\). The overall statistic is \(100.8/(48/17)=100.8(17)/48=35.7\), with reference distribution \(F_{5,34}\) under the overall null.
R-squared checks. For the full model, \(R_F^2=504/600=0.84\). For the reduced model, \(SSR_R=600-150=450\) and \(R_R^2=450/600=0.75\). The difference in regression sums of squares is \(504-450=54\), matching \(SSE_R-SSE_F=54\). This confirms the partial-test numerator. The common mistake avoided is using the reduced model's error degrees of freedom in the denominator; the partial F statistic uses the full model's \(MSE\) and its \(34\) error degrees of freedom.
A Reliable Table-Completion Routine
When a regression ANOVA table is incomplete, calculate each entry in an order that allows independent checks. Do not start by filling cells in the order they appear. In particular, keep a nested-model comparison distinct from the overall F test: the overall test uses \(MSR/MSE\), while a partial test uses the reduction in \(SSE\) per added degree of freedom divided by the full model's \(MSE\).
Record \(n\) and the number of predictors. For a model with an intercept, calculate regression degrees of freedom \(p\), error degrees of freedom \(n-p-1\), and total degrees of freedom \(n-1\).
Use \(SST=SSR+SSE\). If \(R^2\) is supplied, use \(SSR=R^2SST\) and \(SSE=(1-R^2)SST\).
Divide each sum of squares by its own degrees of freedom. For an overall test, divide \(MSR\) by \(MSE\); for a nested comparison, use the reduction in \(SSE\) divided by the number of added terms, then divide by the full-model \(MSE\).
Verify that sums of squares add, degrees of freedom add, and the reported \(R^2\) and F statistic can be reproduced from the entries.
These checks are more than cosmetic. An F statistic can look numerically plausible even when a sum of squares has been assigned to the wrong row or the wrong degrees of freedom have been used. Recomputing the identities catches such mistakes before a test result is interpreted.
Check Your Understanding
Complete each calculation, then compare your results with the final answer.
- A model with an intercept has \(n=18\), \(p=2\), \(SST=240\), and \(SSE=96\). Find \(SSR\), all three degrees of freedom, \(MSR\), \(MSE\), \(R^2\), and the overall F statistic. (Answer: \(SSR=144\); degrees of freedom \(2,15,17\); \(MSR=72\); \(MSE=6.4\); \(R^2=0.6000\); \(F=11.25\), with \(F_{2,15}\) reference distribution.)
- A regression with \(n=26\) and \(p=3\) has \(SST=400\) and \(R^2=0.65\). Recover \(SSR\) and \(SSE\), then calculate \(MSR\), \(MSE\), and the overall F statistic. (Answer: \(SSR=260\); \(SSE=140\); degrees of freedom \(3,22,25\); \(MSR=86.6667\); \(MSE=6.3636\); \(F=13.6190\), with \(F_{3,22}\) reference distribution.)
- For \(n=32\), a reduced model has \(p_R=1\) and \(SSE_R=117\). A nested full model has \(p_F=4\), \(SSE_F=81\), and \(SST=300\). Find the partial F statistic and the full-model \(R^2\). (Answer: \(m=3\); full-model error degrees of freedom \(27\); numerator mean square \(12\); \(MSE_F=3\); partial \(F=4\), with \(F_{3,27}\) reference distribution; full-model \(R^2=0.73\).)
- A model with \(n=20\) and \(p=2\) has \(MSR=45\) and \(MSE=5\). Recover \(SSR\), \(SSE\), \(SST\), \(R^2\), and the overall F statistic. (Answer: degrees of freedom \(2,17,19\); \(SSR=90\); \(SSE=85\); \(SST=175\); \(R^2=18/35\approx0.5143\); \(F=9\), with \(F_{2,17}\) reference distribution.)
- A nested comparison adds \(m=2\) predictors to a full model with \(n=25\) and \(p_F=4\). If \(SSE_R=110\) and \(SSE_F=80\), calculate the partial F statistic and its degrees of freedom. (Answer: numerator mean square \(30/2=15\); full-model \(MSE=80/(25-4-1)=4\); \(F=3.75\), with \(F_{2,20}\) reference distribution.)