Why the Residual Sum Matters
A regression line produces one fitted value for each observation. The difference between an observed response and its fitted value is a residual. For an ordinary least-squares line that includes an intercept, these residuals have a useful check built into them: their sum is zero, apart from possible rounding in displayed calculations.
This is an algebraic property of the fitted line, not a claim that the observations have no errors or that positive and negative errors cancel in every subgroup. It follows from the way least squares chooses the intercept. The identity is useful when calculating a fit by hand or checking a reported regression calculation, but it is only one necessary check—not proof that every part of the fit is correct.
The Intercept Normal Equation
Assume there are \(n\) observed pairs and that an ordinary least-squares line is fitted with a freely chosen intercept and slope. Ordinary least squares chooses \(\hat{a}\) and \(\hat{b}\) to minimize the sum of squared vertical residuals,
No probability distribution for the responses is needed for the residual-sum identity. It is a property of the minimization itself. In particular, normality, constant error variance, and independence are not required for this algebraic result.
To see why, first hold the slope \(b\) fixed and consider how the squared-error criterion changes with the intercept \(a\). At the least-squares minimum, the derivative with respect to this freely chosen intercept must be zero. Differentiating gives the intercept normal equation:
At the fitted values \(a=\hat{a}\) and \(b=\hat{b}\), each term inside the sum is a residual. Dividing the equation by \(-2\) therefore gives the identity. The second derivative with respect to \(a\) is \(2n>0\), so for any fixed slope, the intercept satisfying this equation is the unique intercept minimizing the squared errors at that slope.
Use \(e_i=y_i-\hat{a}-\hat{b}x_i\), with observed response minus fitted response.
At an ordinary least-squares minimum, \(\sum_{i=1}^{n}(y_i-\hat{a}-\hat{b}x_i)=0\).
The terms in the normal equation are exactly the residuals, so \(\sum_{i=1}^{n}e_i=0\).
Worked Examples
Worked Example: Correcting an Intercept While Holding the Slope Fixed
A hypothetical laboratory studies the relationship between a calibration setting \(x\) and a measured response \(y\). For \(x=1,2,3\), the observed responses are \(4,5,8\). A draft calculation uses the line \(\hat{y}=2+x\). Find its residual sum, then find the intercept adjustment that makes the residual sum zero while leaving the slope unchanged.
Method and assumptions. Calculate observed-minus-fitted residuals, then use the intercept normal equation with the slope fixed. This is a deterministic least-squares calculation; no interest rate, mortality assumption, or response probability distribution is specified or needed.
The fitted values at \(x=1,2,3\) are respectively \(3,4,5\). Thus the residuals are
Their sum is \(1+1+3=5\), so the draft line does not satisfy the intercept normal equation. Let \(c\) be the change to the intercept. The adjusted line is \(\hat{y}=(2+c)+x\). Increasing the fitted value by \(c\) decreases each residual by \(c\), so the new residual sum is \(5-3c\). Set this equal to zero:
The intercept must therefore increase from \(2\) to \(2+\frac{5}{3}=\frac{11}{3}\). To check, the adjusted fitted values are \(\frac{14}{3},\frac{17}{3},\frac{20}{3}\), giving residuals \(-\frac{2}{3},-\frac{2}{3},\frac{4}{3}\). Their sum is \(-\frac{2}{3}-\frac{2}{3}+\frac{4}{3}=0\).
Common mistake avoided: Decreasing the intercept would increase each residual, making a positive residual sum larger rather than reducing it. The sign follows directly from \(e_i=y_i-\hat{y}_i\).
Worked Example: Checking a Computed Least-Squares Fit
A hypothetical analyst records a predictor \(x\) and response \(y\) for four policy groups. The data are \((0,2),(1,2),(2,5),(3,5)\). Compute the ordinary least-squares line and verify that its residuals sum to zero.
Method and assumptions. Use the simple linear least-squares formulas for the slope and intercept, then calculate each fitted value and residual. This is a deterministic calculation; no interest rate, mortality assumption, or response probability distribution is specified or required.
First calculate the sample means:
The centered predictor sum of squares and centered cross-product are
Therefore the slope is \(\hat{b}=S_{xy}/S_{xx}=6/5=1.2\), and the intercept is \(\hat{a}=\bar{y}-\hat{b}\bar{x}=3.5-(1.2)(1.5)=1.7\). The fitted line is \(\hat{y}=1.7+1.2x\).
| \(x_i\) | \(y_i\) | \(\hat{y}_i=1.7+1.2x_i\) | \(e_i=y_i-\hat{y}_i\) |
|---|---|---|---|
| 0 | 2 | 1.7 | 0.3 |
| 1 | 2 | 2.9 | -0.9 |
| 2 | 5 | 4.1 | 0.9 |
| 3 | 5 | 5.3 | -0.3 |
The residual sum is \(0.3-0.9+0.9-0.3=0\), as required. This checks the intercept equation for the computed line. The arithmetic is exact to the displayed tenths, so no rounding adjustment is involved.
Common mistake avoided: A residual is observed minus fitted, not fitted minus observed. Reversing that convention would change each sign, though the sum would still be zero in this particular check; it would give incorrect individual residual interpretations.
Worked Example: Auditing a Reported Line
A hypothetical claims team reports observations \((x,y)=(0,5),(1,7),(2,8),(3,10)\) and claims that \(\hat{y}=4+1.5x\) is an ordinary least-squares fit with an intercept. Check that claim. If the slope is held at \(1.5\), find the intercept adjustment that makes the residual sum zero.
Method and assumptions. Calculate residuals from the reported line and apply the intercept normal equation. This is a deterministic regression check; no interest rate, mortality assumption, or response probability distribution is specified or needed.
The fitted values are \(4,5.5,7,8.5\). The observed-minus-fitted residuals are \(1,1.5,1,1.5\), whose sum is \(1+1.5+1+1.5=5\). Since the sum is not zero, the reported line cannot be the ordinary least-squares fit with a freely estimated intercept.
Let \(c\) be the change to the intercept. There are four observations, and raising the intercept by \(c\) lowers each residual by \(c\). Hence the new residual sum is \(5-4c\). Solving \(5-4c=0\) gives \(c=1.25\). With slope still fixed at \(1.5\), the adjusted intercept is \(4+1.25=5.25\).
The adjusted fitted values are \(5.25,6.75,8.25,9.75\), so the residuals become \(-0.25,0.25,-0.25,0.25\). They sum to zero. This correction satisfies the intercept normal equation for the fixed slope. It does not by itself establish that the slope is the least-squares slope; the zero-sum identity checks the intercept condition only.
Common mistake avoided: Do not infer that a line is fully verified just because its residuals sum to zero. The check is necessary for a least-squares fit with an intercept, but other aspects of the fit also need to be correct.
When the Identity Applies—and When It Does Not
The unweighted zero-sum identity depends on the fitting criterion and the presence of a free intercept. If the line is fitted by ordinary least squares with an intercept, the intercept normal equation forces the residual sum to zero. If the intercept is fixed in advance, or the line is constrained to pass through the origin, that normal equation is not generally imposed, so the unweighted residual sum need not vanish.
The same distinction matters for weighted least squares. If the criterion is \(\sum_{i=1}^{n}w_i e_i^2\), with positive weights \(w_i\), and an intercept is freely estimated, differentiating with respect to the intercept gives \(\sum_{i=1}^{n}w_i e_i=0\). It is the weighted residual sum that is zero; the ordinary sum \(\sum e_i\) need not be. Thus the fitting method must be identified before applying the check.
In practical calculations, a displayed residual sum such as \(0.0001\) may result from rounding fitted values or residuals. Use unrounded values where possible. If only rounded values are available, a small discrepancy may be a rounding effect, while a substantial discrepancy is a reason to recheck the data, residual sign convention, intercept, and fitting method.
Common Exam Traps
- Using the wrong residual sign. This tutorial defines \(e_i=y_i-\hat{y}_i\). Keep that convention consistent when summing or adjusting an intercept.
- Getting the intercept-adjustment sign backwards. Increasing the intercept raises fitted values and therefore lowers residuals. If the residual sum is positive, the intercept must increase to bring it down to zero.
- Treating a check as a complete proof of the fit. A zero residual sum verifies the intercept normal equation, not every condition needed for the full least-squares solution.
- Applying the unweighted identity to a different fitting procedure. Weighted least squares gives a weighted residual-sum condition; a fit without a free intercept need not have residuals summing to zero.
Check Your Understanding
Use \(e_i=y_i-\hat{y}_i\). For each problem, identify whether the residual-sum check is satisfied or calculate the indicated intercept adjustment.
- A fitted line with a free intercept has five residuals \(-2,1,0,4,-3\). Calculate their sum and state whether they satisfy the intercept normal equation. (Answer: Sum \(=0\); the intercept condition is satisfied.)
- For observations \((0,3),(1,6),(2,8)\), a draft line is \(\hat{y}=2+2x\). Find its residual sum and the intercept adjustment, holding the slope fixed, that makes the sum zero. (Answer: Residual sum \(=5\); increase the intercept by \(5/3\), to \(11/3\).)
- A reported line has residual sum \(-6\) for four observations. How must its intercept change, with the slope held fixed, to make the residual sum zero? (Answer: Decrease the intercept by \(1.5\).)
- A weighted least-squares fit minimizes \(\sum w_i e_i^2\) and includes a free intercept. Which sum is forced to zero by the intercept equation: the unweighted residual sum or the weighted residual sum? (Answer: The weighted sum \(\sum w_i e_i\) is forced to zero; the unweighted sum need not be.)
- A line includes a free intercept, and its exact residuals sum to zero. Does this fact alone prove that the line is the full ordinary least-squares fit? (Answer: No; it verifies the intercept normal equation only, not the full least-squares conditions.)