Why Reconstruct a Regression ANOVA Table?
A regression output may report only part of an analysis-of-variance table. You may be given the sample size, the number of predictors, and two sums of squares, then asked to fill in the remaining entries. The arithmetic is direct, but a single degrees-of-freedom error changes the mean square and therefore the overall F-statistic.
The key is to keep three quantities distinct: variation explained by the fitted regression, variation left in the residuals, and total variation around the response mean. For a regression model with an intercept, these quantities have an exact relationship. The predictor count means the number of explanatory variables, not the number of coefficients including the intercept.
Reconstructing the Table
A regression ANOVA table arranges the sums of squares, degrees of freedom, and mean squares by source of variation. The regression mean square is \(MSR=SSR/p\), and the error mean square is \(MSE=SSE/(n-p-1)\). The total row has no mean square in the standard regression table.
| Source | Sum of squares | Degrees of freedom | Mean square |
|---|---|---|---|
| Regression | \(SSR\) | \(p\) | \(SSR/p\) |
| Error (residual) | \(SSE\) | \(n-p-1\) | \(SSE/(n-p-1)\) |
| Total (corrected) | \(SST=SSR+SSE\) | \(n-1\) | Not usually reported |
These formulas require a full-rank fitted design matrix: the intercept and predictors must not be exactly linearly dependent. The residual degrees of freedom must be positive, so \(n>p+1\). The table uses the corrected total sum of squares, which measures variation around \(\bar{y}\); it is not the uncorrected sum \(\sum_i y_i^2\).
Worked Example: Completing a Table From Two Sums of Squares
A hypothetical insurer models annual claim-processing time using three predictors and an intercept. There are \(n=24\) observations. The regression sum of squares is \(SSR=180\), and the error sum of squares is \(SSE=120\). Reconstruct the ANOVA table, calculate the overall F-statistic, and find \(R^2\). For the F-test, assume the errors, conditional on the predictors, are independent \(N(0,\sigma^2)\) with constant variance, and the stated linear model is correctly specified.
Method. First count the predictors excluding the intercept: \(p=3\). The regression degrees of freedom are \(3\), the error degrees of freedom are \(24-3-1=20\), and the corrected total degrees of freedom are \(24-1=23\).
The total sum of squares is \(SST=SSR+SSE=180+120=300\). The mean squares are \(MSR=180/3=60\) and \(MSE=120/20=6\). Therefore, the overall F-statistic is \(F=MSR/MSE=60/6=10\). The coefficient of determination is \(R^2=SSR/SST=180/300=0.60\).
Reconstructed table. The regression row is \(SSR=180\), df \(=3\), and \(MSR=60\). The error row is \(SSE=120\), df \(=20\), and \(MSE=6\). The corrected total row is \(SST=300\), df \(=23\).
Check. The sums of squares satisfy \(180+120=300\), and the degrees of freedom satisfy \(3+20=23\). The calculated \(R^2=0.60\) also agrees with \(1-SSE/SST=1-120/300=0.60\). A common mistake would be to count the intercept as a fourth predictor, giving regression df \(4\); that is incorrect because the regression df here is the number of non-intercept predictors, \(p=3\).
Why the Sum-of-Squares and F Formulas Hold
The decomposition \(SST=SSR+SSE\) follows from the fitted values and residuals being orthogonal in ordinary least squares with an intercept. For each observation, write \(y_i-\bar{y}=(\hat{y}_i-\bar{y})+e_i\), square both sides, and sum. This gives
The cross-product sum is zero: the residual vector is orthogonal to the fitted-value vector, and the residuals sum to zero when the model includes an intercept. Thus the final term vanishes, leaving \(SST=SSR+SSE\). This identity is not generally available in this form for a model fitted without an intercept.
The overall regression F-statistic compares the amount of variation explained per regression degree of freedom with the residual variation per error degree of freedom:
For the model assumptions in the example, this statistic tests the joint null hypothesis \(H_0:\beta_1=\cdots=\beta_p=0\), where the \(\beta_j\) are the non-intercept regression coefficients. Under this null, \(SSR/\sigma^2\) and \(SSE/\sigma^2\) have independent chi-squared distributions with \(p\) and \(n-p-1\) degrees of freedom, respectively. Dividing each by its degrees of freedom gives scaled quantities whose ratio has an \(F\) distribution with those degrees of freedom. Thus, under the null and the stated assumptions, \(F\sim F_{p,n-p-1}\). Without the assumptions supporting this result, the arithmetic ratio can still be calculated, but its usual exact F-test interpretation does not automatically follow.
Record \(p\) as the number of explanatory variables, excluding the intercept.
Use regression df \(p\), error df \(n-p-1\), and corrected total df \(n-1\). Check that the first two add to the third.
Use \(SST=SSR+SSE\) for an intercept model. If \(R^2\) is supplied instead, use \(R^2=SSR/SST\).
Divide each regression or error sum of squares by its own degrees of freedom, then divide \(MSR\) by \(MSE\).
Verify both the sums-of-squares identity and the degrees-of-freedom identity. Recalculate \(R^2\) from either \(SSR/SST\) or \(1-SSE/SST\) when possible.
Worked Example: Recovering Missing Sums From R-Squared
A hypothetical health plan uses four predictors to model a positive-valued annual utilization measure. For \(n=31\) observations, the corrected total sum of squares is \(500\), and the reported \(R^2\) is \(0.72\). Reconstruct the sums of squares, mean squares, and overall F-statistic. Assume the regression includes an intercept and has full rank. For the exact F-test interpretation, assume independent normal errors with mean zero and constant variance, conditional on the predictors.
Method. Since \(R^2=SSR/SST\), the regression sum of squares is \(SSR=0.72(500)=360\). Then \(SSE=SST-SSR=500-360=140\).
There are \(p=4\) predictors, so regression df \(=4\), error df \(=31-4-1=26\), and total df \(=31-1=30\). The regression mean square is \(MSR=360/4=90\). The error mean square is \(MSE=140/26=70/13\), or approximately \(5.3846\).
Thus \(F=90/(70/13)=90(13/70)=117/7\), approximately \(16.7143\). No significance decision is made here because no significance level or critical value is specified.
Check. The sums add to \(360+140=500\), and \(360/500=0.72\). The degrees of freedom add to \(4+26=30\). The calculation avoids the mistake of treating \(R^2=0.72\) as the fraction of variation left unexplained; it is the fraction explained, so the residual fraction is \(1-0.72=0.28\), giving \(SSE=0.28(500)=140\).
Consistency Checks and Common Traps
A reconstructed table should pass simple checks before its F-statistic is interpreted. These checks are especially useful when a question gives a mixture of sums of squares, \(R^2\), sample size, and predictor count.
- Predictor count: \(p\) excludes the intercept. Counting it as a predictor adds one to the regression degrees of freedom and subtracts one from the error degrees of freedom.
- Degrees of freedom: for a model with an intercept, \(p+(n-p-1)=n-1\). If the displayed values do not add up, revisit the predictor count and sample size.
- Sums of squares: \(SSR+SSE=SST\) for the corrected total in an ordinary least-squares model with an intercept. A mismatch may indicate a calculation error, rounded reported values, or quantities that are not from the same model.
- Mean squares: divide by the degrees of freedom from the same row. In particular, \(MSE\) uses residual degrees of freedom, not total degrees of freedom.
- Meaning of \(R^2\): \(R^2=SSR/SST=1-SSE/SST\). A value near one indicates a large explained fraction in the fitted sample, but by itself does not establish that the model assumptions hold.
Worked Example: Checking a Reported Table for an Intercept-Counting Error
A hypothetical agricultural analyst fits an intercept model with three predictors to \(n=20\) plots. The reported values are \(SSR=120\), \(SSE=80\), regression df \(=4\), error df \(=15\), and total df \(=19\). Identify whether the table is internally consistent and provide the corrected entries.
Method. There are three predictors, so the correct regression df is \(p=3\), not \(4\). The correct error df is \(20-3-1=16\), and the total df is \(20-1=19\).
The sums of squares give \(SST=120+80=200\). The corrected mean squares are \(MSR=120/3=40\) and \(MSE=80/16=5\). The overall F-statistic is \(F=40/5=8\), and \(R^2=120/200=0.60\).
Check. Correct degrees of freedom add to \(3+16=19\), and the sums add to \(120+80=200\). The reported degrees of freedom, \(4\) and \(15\), also add to \(19\), so checking only the total would fail to detect the error. The predictor count identifies the issue: the intercept is not included in the regression df.
Check Your Understanding
For each problem, assume an ordinary least-squares regression with an intercept and a full-rank design. Where an F-test interpretation is requested or implied, assume independent normal errors with mean zero and constant variance, conditional on the predictors.
- A model has \(n=16\), \(p=3\), \(SSR=90\), and \(SSE=60\). Find the corrected total sum of squares, all three degrees of freedom, both mean squares, \(F\), and \(R^2\). (Answer: \(SST=150\); df are \(3,12,15\); \(MSR=30\), \(MSE=5\), \(F=6\), \(R^2=0.60\).)
- A regression with \(n=26\) and \(p=2\) has \(SST=240\) and \(R^2=0.75\). Reconstruct \(SSR\), \(SSE\), the degrees of freedom, both mean squares, and \(F\). (Answer: \(SSR=180\), \(SSE=60\); df are \(2,23,25\); \(MSR=90\), \(MSE=60/23\), \(F=34.5\).)
- A model with \(n=40\) observations and five predictors has \(SSR=255\) and \(SSE=170\). Find \(SST\), \(R^2\), both mean squares, and \(F\). (Answer: \(SST=425\), df are \(5,34,39\); \(R^2=0.60\), \(MSR=51\), \(MSE=5\), \(F=10.2\).)
- A one-predictor model has \(n=12\), \(SST=88\), and \(SSE=44\). Calculate \(SSR\), all degrees of freedom, \(MSR\), \(MSE\), \(F\), and \(R^2\). (Answer: \(SSR=44\); df are \(1,10,11\); \(MSR=44\), \(MSE=4.4\), \(F=10\), \(R^2=0.50\).)
- A model with \(n=20\) and \(p=3\) is reported to have \(SSR=120\), \(SSE=80\), and corrected \(SST=190\). Can these three sums of squares all be correct for the same intercept model? (Answer: No; \(SSR+SSE=200\), not \(190\), so at least one reported sum is inconsistent.)