Connecting ANOVA Sums of Squares to R-Squared

Why Connect ANOVA Sums of Squares to R-Squared?

A regression ANOVA table separates variation in the observed response into a part associated with fitted values and a part left in the residuals. R-squared summarizes that split as one proportion. The calculation is short, but it depends on using the right sums of squares and on understanding when their decomposition is valid.

This connection is useful when a regression output gives sums of squares rather than R-squared, or when a question asks what fraction of the response variation is accounted for by the fitted model. It is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

The total sum of squares measures the observed response’s variation around its sample mean. The regression sum of squares measures how much the fitted values vary around that same mean. The error sum of squares measures the remaining squared differences between observations and fitted values. All three quantities are nonnegative.

The stated formula uses an ordinary least squares (OLS) fit with an intercept. With an intercept, OLS includes the constant-only fit as a possible model and satisfies the orthogonality conditions needed for the decomposition. The definition of \(SS_{\mathrm{Tot}}\) still makes sense for other fits, but the equality \(SS_{\mathrm{Tot}}=SS_{\mathrm{Reg}}+SS_E\), and hence the equivalence of the two formulas for \(R^2\), need not hold for a model without an intercept or for arbitrary fitted values.

Why the Sums-of-Squares Decomposition Holds

For an OLS model with an intercept, the residuals sum to zero, and the residual vector is orthogonal to every column of the design matrix. The fitted-value vector is a linear combination of those columns, so it is also orthogonal to the residual vector. In particular, \(\sum_{i=1}^n(\hat y_i-\bar y)e_i=0\). These facts give the decomposition directly.

The first equality in the last step follows because fitted values lie in the design matrix’s column space and OLS residuals are orthogonal to that space. The second term vanishes because the residuals sum to zero. Thus, \[ \frac{SS_{\mathrm{Reg}}}{SS_{\mathrm{Tot}}} =\frac{SS_{\mathrm{Tot}}-SS_E}{SS_{\mathrm{Tot}}} =1-\frac{SS_E}{SS_{\mathrm{Tot}}}. \] This also explains the two common ways to calculate \(R^2\).

Calculating R-Squared from Fitted Values

When observations and fitted values are provided, calculate all relevant quantities using the same observations. First find the observed mean; then compute the centered total, fitted-value, and residual sums of squares. For an OLS model with an intercept, check that the decomposition is satisfied before reporting \(R^2\).

Calculating R-Squared from an ANOVA Table

A regression ANOVA table commonly reports \(SS_{\mathrm{Reg}}\) and \(SS_E\), with \(SS_{\mathrm{Tot}}\) either shown directly or obtained by addition. For an OLS model with an intercept, the total equals the sum of the regression and error sums of squares. Degrees of freedom and mean squares are different ANOVA quantities; they are not substituted for sums of squares in the \(R^2\) formula.

What R-Squared Does—and Does Not—Say

For an OLS regression with an intercept and positive total variation, the decomposition implies \(0\leq R^2\leq1\): both \(SS_{\mathrm{Reg}}\) and \(SS_E\) are nonnegative parts of \(SS_{\mathrm{Tot}}\). An \(R^2\) of \(0\) means the fitted values have no variation around \(\bar y\), as in the constant-only fit. An \(R^2\) of \(1\) means every residual is zero, so the fitted values match the observed responses exactly.

R-squared describes the fitted model’s accounting for variation in the observed response values. It does not say that predictors caused the response, establish that the model’s assumptions are appropriate, or guarantee accurate predictions for new observations. Nor does an \(R^2\) value by itself indicate whether a relationship is practically important. Interpretation should stay tied to the sample and response being analyzed.

For the same response observations, adding predictors to an OLS model with an intercept cannot increase \(SS_E\): the expanded model can always reproduce the original fit by assigning zero coefficients to the added predictors. Since \(SS_{\mathrm{Tot}}\) is unchanged, \(R^2=1-SS_E/SS_{\mathrm{Tot}}\) cannot decrease. An increase in training-sample \(R^2\) therefore does not, by itself, demonstrate that the added predictors are useful.

If a model is fitted without an intercept, the usual centered decomposition may fail, and the formula \(1-SS_E/SS_{\mathrm{Tot}}\) can even be negative. Some software instead reports an uncentered measure based on \(\sum_i y_i^2\); that quantity uses a different baseline and should not be silently treated as the centered \(R^2\) defined here. Always identify the model and the sums of squares used.

Common Calculation Traps

• Mixing definitions. Use \(SS_{\mathrm{Reg}}=\sum_i(\hat y_i-\bar y)^2\) and \(SS_{\mathrm{Tot}}=\sum_i(y_i-\bar y)^2\) for the centered decomposition.
• Ignoring the intercept condition. The familiar identity and the usual range from zero to one are justified for OLS with an intercept, not for every possible prediction rule.
• Using the wrong denominator. \(R^2\) is a fraction of total response variation, not a ratio to \(\sum_i y_i^2\) under the centered definition.
• Interpreting “accounted for” as “caused.” R-squared is a descriptive measure of the fitted sample, not a causal conclusion.
• Reporting a percentage without checking the arithmetic. Confirm \(SS_{\mathrm{Tot}}=SS_{\mathrm{Reg}}+SS_E\) before converting the proportion to a percentage.