Why Measure Variation in the Fitted Values?
A regression model produces a fitted response for each observation. These fitted values may vary across observations: for example, a model may give higher fitted blood pressure for patients with higher fitted risk scores. Regression sum of squares summarizes how much the fitted responses vary around their average. It describes variation in the model's predictions, not the remaining differences between the observed responses and those predictions.
To keep the three common sums of squares distinct, this tutorial uses \(SSR\) for regression sum of squares, \(SSE\) for residual sum of squares, and \(SST\) for total sum of squares. The residual and total sums of squares were introduced in the earlier material on those topics; here the emphasis is on what is being squared in \(SSR\), and how that differs from the other two quantities.
The quantity in each squared term is a fitted response minus the average fitted response. For the ordinary least-squares model with an intercept, the average fitted response equals \(\bar{y}\), so the definition can equivalently be described as variation in the fitted values around their own mean. Each deviation is measured in the response variable's units, and \(SSR\) is measured in squared response units.
Three Sums of Squares, Three Sets of Deviations
The sums of squares differ according to which values are being compared and what reference value is used. Their names can sound similar, so it helps to inspect the deviation inside each square.
| Quantity | Formula | What varies around what? | Units |
|---|---|---|---|
| Regression sum of squares, \(SSR\) | \(\sum(\hat{y}_i-\bar{y})^2\) | Fitted responses around the response mean | Squared response units |
| Residual sum of squares, \(SSE\) | \(\sum(y_i-\hat{y}_i)^2\) | Observed responses around their fitted values | Squared response units |
| Total sum of squares, \(SST\) | \(\sum(y_i-\bar{y})^2\) | Observed responses around their mean | Squared response units |
For example, if the response is measured in millimetres of mercury, each of these sums is measured in squared millimetres of mercury. The formulas show why \(SSR\) is not a sum of squared residuals: residuals are \(y_i-\hat{y}_i\), whereas \(SSR\) uses \(\hat{y}_i-\bar{y}\). Nor is \(SSR\) the total observed variation, which uses \(y_i-\bar{y}\).
These definitions do not by themselves say that a model is useful. A large \(SSR\) means the fitted responses vary substantially around their mean, but it does not tell us whether they are close to the observed responses. Conversely, a small \(SSR\) may simply reflect fitted values that are all similar. Interpreting fit requires keeping the residual and total variation in view as well.
Worked Example: Calculating Regression Sum of Squares From Fitted Values
Suppose a least-squares regression with an intercept is fitted to four hypothetical observations. The observed responses are \(2, 4, 5,\) and \(9\), and the fitted responses are \(1.7, 3.9, 6.1,\) and \(8.3\). Calculate \(SSR\).
First find the observed response mean: $$ \bar{y}=\frac{2+4+5+9}{4}=\frac{20}{4}=5. $$ The fitted values also average to \(5\): \((1.7+3.9+6.1+8.3)/4=20/4=5\). Apply the definition: $$ SSR=(1.7-5)^2+(3.9-5)^2+(6.1-5)^2+(8.3-5)^2. $$ Substituting and squaring gives $$ SSR=(-3.3)^2+(-1.1)^2+(1.1)^2+(3.3)^2 =10.89+1.21+1.21+10.89=24.20. $$ Thus the regression sum of squares is \(24.20\) squared response units. It measures the spread of the fitted values around \(5\); it is not calculated from the gaps between each observed response and its fitted value.
The values are consistent with an intercept-containing least-squares fit: the residuals are \(0.3, 0.1, -1.1,\) and \(0.7\), which sum to zero, and the fitted values have the same mean as the observations. No claim about statistical significance or causal effect follows from this calculation.
Why the Fitted Values Have the Observed Mean
The use of \(\bar{y}\) in the definition is natural for an intercept-containing least-squares model. In that setting, the fitted values average to the observed responses. This fact follows directly from how least squares chooses the intercept; no assumption about normally distributed errors is needed for the calculation.
This justification concerns the center of the fitted values. It does not derive a relationship among \(SSR\), \(SSE\), and \(SST\); that sum-of-squares decomposition requires a separate argument. The calculation of \(SSR\) itself is simply the sum of squared deviations of fitted values from their mean.
Worked Example: Regression Sum of Squares for Two Fitted Group Means
A hypothetical study records an outcome in two groups, with three observations in each. A least-squares model uses group membership to assign a fitted value equal to the mean outcome in that group. The observed outcomes are \(4, 5, 6\) in the first group and \(7, 8, 9\) in the second. Calculate the fitted values and \(SSR\).
The first group mean is \((4+5+6)/3=5\), and the second group mean is \((7+8+9)/3=8\). Thus the six fitted values are \(5,5,5,8,8,8\). The overall response mean is $$ \bar{y}=\frac{4+5+6+7+8+9}{6}=\frac{39}{6}=6.5. $$ The fitted values have mean \((3(5)+3(8))/6=39/6=6.5\), as expected for this intercept-containing least-squares fit. Therefore $$ SSR=3(5-6.5)^2+3(8-6.5)^2 =3(2.25)+3(2.25)=13.50. $$ The regression sum of squares is \(13.50\) squared outcome units. It records the variation between the two fitted group means around the overall mean.
For contrast, the residual sum of squares is based on within-group differences between observations and their group fitted values: $$ SSE=(4-5)^2+(5-5)^2+(6-5)^2+(7-8)^2+(8-8)^2+(9-8)^2 =1+0+1+1+0+1=4. $$ The two sums describe different patterns: \(SSR\) reflects variation among the fitted group means, while \(SSE\) reflects variation of observations around those fitted means.
Conditions and Scope of the Calculation
The arithmetic definition of \(SSR\) requires observed responses, fitted responses, and a stated reference mean. For the standard centered regression sum of squares used here, the fitted values come from a least-squares regression model with an intercept, and the reference is \(\bar{y}\). The equality between the mean fitted response and \(\bar{y}\) follows from least squares as shown above. No distributional assumption is needed merely to calculate this sum of squares.
Regression inference may require additional model assumptions, depending on the question being asked. Those assumptions are not needed to define or compute \(SSR\). In particular, a numerical \(SSR\) does not itself provide a p-value, confidence interval, or guarantee that the model form is appropriate. The observed data, the model, and the purpose of the analysis still matter.
Worked Example: A Model With No Variation in Its Fitted Values
Consider three hypothetical response values, \(3, 5,\) and \(7\), fitted using an intercept-only least-squares model. This model assigns every observation the same fitted value, the response mean. Calculate \(SSR\) and explain what the result says.
The response mean is $$ \bar{y}=\frac{3+5+7}{3}=5. $$ The fitted values are \(5,5,5\), so $$ SSR=(5-5)^2+(5-5)^2+(5-5)^2=0+0+0=0. $$ There is no variation in the fitted values around their mean. This does not mean that the observed responses are identical: they are \(3,5,\) and \(7\). It means only that this model gives the same prediction for every observation. Here the residual sum of squares is \((3-5)^2+(5-5)^2+(7-5)^2=4+0+4=8\), demonstrating that zero regression sum of squares is not the same as zero residual sum of squares.
The calculation is valid because the intercept-only least-squares model includes an intercept and fits the overall mean. It makes no claim that this is the most useful model for a particular scientific question.
Interpreting and Reporting \(SSR\)
A clear report names the quantity, gives its value, and includes squared units. For example: “The regression sum of squares for the fitted model was \(13.50\) squared outcome units.” If the response's units are minutes, the sum is in minutes squared—not minutes. The original response units are recovered in individual fitted-value deviations, but the sum of their squares has squared units.
The magnitude of \(SSR\) depends on the scale of the response and on the fitted values. If every response and fitted value is converted from one unit to another by multiplying by a constant \(a\), each deviation is multiplied by \(a\), and the sum of squared deviations is multiplied by \(a^2\). Thus comparing raw \(SSR\) values across differently scaled outcomes may be misleading.
It is also important not to treat “regression” as synonymous with “caused by the predictors.” The fitted values are determined by a statistical model; their variation does not establish that changing a predictor would change the response. Causal conclusions require an appropriate study design and additional justification.
Common Mistakes
- Using observed values instead of fitted values. The expression \(\sum(y_i-\bar{y})^2\) is \(SST\), not \(SSR\). Use \(\hat{y}_i\) in the deviations for regression sum of squares.
- Calling squared residuals regression sum of squares. The expression \(\sum(y_i-\hat{y}_i)^2\) is \(SSE\). It measures the discrepancies left between observations and fitted values.
- Using a different center without explanation. For the standard centered definition here, compare fitted values with \(\bar{y}\). In an intercept-containing least-squares fit, that is also the mean fitted value.
- Interpreting a large \(SSR\) as proof of a good model. \(SSR\) describes variation in predictions, not how close those predictions are to the observations. Consider what the other sums of squares represent, and do not infer significance or causation from \(SSR\) alone.
- Reporting the wrong units. A response measured in grams leads to sums of squares in grams squared.
Check Your Understanding
Answer each question by identifying the relevant deviations and explaining what the result measures.
- In \(SSR=\sum(\hat{y}_i-\bar{y})^2\), which values are compared, and what does their squared deviation measure?
- A response is measured in kilograms. What are the units of its regression sum of squares?
- For a model with fitted values \(2, 4, 6\) and mean fitted value \(4\), calculate the regression sum of squares.
- Explain why \(\sum(y_i-\hat{y}_i)^2\) is not regression sum of squares.
- An intercept-only model gives \(SSR=0\). Does that imply that all observed responses are equal? Explain.