The Zero-Sum Residual Property

Why Residuals Balance Around Zero

A regression line leaves a residual for each observation: the observed response minus the response predicted by the line. Some residuals are positive and some are negative. When ordinary least squares fits a line with an intercept, these signed differences balance exactly: their sum is zero. The balance is a consequence of the fitting rule, not an extra assumption about the data.

This property is useful as a quick check on a calculation. It also has a simple interpretation: the average residual is zero, so the fitted responses and observed responses have the same average. The result applies to the residuals used to fit the line; it does not say that predictions for new observations will have errors that sum to zero.

Here \(b_0\) is the fitted intercept and \(b_1\) is the fitted slope. The residual has the same units as the response \(y\), not the predictor \(x\). A positive residual means the observation is above the fitted line; a negative residual means it is below. The zero-sum result concerns the signed residuals. It does not say that the sum of their absolute values or their squares is zero.

Why the Intercept Forces the Sum to Zero

The key is that least squares is free to adjust the intercept. Hold the slope \(b_1\) fixed temporarily and consider the sum of squared residuals as a function of a candidate intercept \(b_0\):

At the least-squares minimum, changing \(b_0\) slightly in either direction cannot reduce \(Q\). Because \(Q\) is a differentiable quadratic function, its derivative with respect to \(b_0\) must be zero at its minimum. Differentiating each squared term gives:

At the fitted coefficients, \(y_i-b_0-b_1x_i\) is exactly \(e_i\). Dividing the equation by \(-2\) therefore gives the result:

This argument does not require the responses to follow a particular probability distribution. It follows from minimizing squared residuals over an intercept that can vary freely. In particular, it still applies if the predictor values have no variation: a slope may then be undetermined, but minimizing over the intercept still makes the residuals sum to zero.

Another way to see the role of the intercept is to write the fitted response as \(\hat{y}_i=b_0+b_1x_i\). The derivative condition says the intercept adjusts until the total observed response matches the total fitted response. Thus the fitted responses and observations have equal means. This is a statement about the sample used for fitting, not a claim that each fitted response equals its observed response.

Checking the Property in a Dataset

Consider the following small hypothetical dataset. The response is measured in units, and the predictor is measured in hours. We first calculate the ordinary least-squares line, then check its residuals directly.

The calculation also illustrates why signs matter. A residual sum uses the original positive and negative values. If we squared the residuals instead, we would get \(0.5^2+(-1)^2+0.5^2=1.5\) squared response units, not zero. Least squares minimizes this sum of squares; it does not make the sum of squared residuals vanish.

More Worked Checks

The intercept condition is essential. If the line is forced through the origin, there is no adjustable intercept to balance the total signed residuals. The following small calculation shows that the sum can then be nonzero.

Using the Property Carefully

For an intercept model, the zero-sum result is a useful calculation check, but it is not a general measure of whether the line fits well. A poor-fitting line can still have residuals that sum to zero. For example, a few large positive residuals can balance several negative ones even when the individual prediction errors are substantial.

The property also has a practical limitation: displayed values are often rounded. A computer may use more decimal places internally than it shows in a table. If residuals are rounded before being added, their displayed sum might be a small amount away from zero. That tiny discrepancy does not necessarily indicate an incorrect fit; check using unrounded values when available.

The zero-sum property applies to the observed data used to estimate the line. It does not guarantee that residuals from a separate set of observations, or prediction errors for future cases, will sum to zero. Those values were not used to select the intercept.

Common Mistakes

• Adding absolute residuals instead of signed residuals. The theorem is about \(\sum e_i\), where negative residuals remain negative. Absolute values remove the cancellation that makes the sum zero.
• Expecting every residual to be zero. A zero sum means the residuals balance overall; it does not mean every point lies on the fitted line.
• Applying the result to a line forced through the origin. Without an intercept, the derivative condition with respect to an intercept is unavailable, and the residual sum need not be zero.
• Treating a small rounded sum as a failure. Rounding individual fitted values or residuals can introduce a small displayed discrepancy. The exact result concerns unrounded least-squares residuals.