Properties of the Least-Squares Line

Three Useful Checks on a Least-Squares Line

A least-squares regression line summarizes the linear pattern in a set of paired data. In “Prediction Versus Observed Values,” you learned that a residual is the observed response minus the predicted response, \(y-\hat{y}\). Looking at all the residuals together reveals a few important properties of the fitted line. These properties can help you check a calculation and interpret how the line relates to the data.

For a least-squares line with an intercept, three facts are especially useful: the line passes through the point \((\bar{x},\bar{y})\), the signed residuals for the fitted data sum to zero, and the slope \(b\) has the same sign as the correlation \(r\) when both variables vary. These are properties of the line fitted to the data, not guarantees that every individual point lies on the line.

The Line Passes Through the Point of the Means

The point \((\bar{x},\bar{y})\) combines the sample mean of the explanatory variable with the sample mean of the response variable. It is a point on the least-squares regression line. In other words, if you substitute \(\bar{x}\) into the line, the predicted response is \(\bar{y}\):

This agrees with the earlier tutorial “Finding the Line Through the Means,” which used the identity \(a=\bar{y}-b\bar{x}\). Here, the mean point is also a useful check: if you have the line and the sample means, substitute \(\bar{x}\) and see whether the result is \(\bar{y}\). A mismatch may indicate an arithmetic error or coefficients rounded for display.

The mean point is not necessarily an observed case. For example, \(\bar{x}\) might be a value that no individual in the sample had. The property says that the line passes through the averages, not that the sample contains an observation at those averages.

Why the Signed Residuals Sum to Zero

For each observed case, the residual is \(y-\hat{y}\). Residuals above the line are positive, and residuals below the line are negative. In the data used to fit a least-squares line with an intercept, the positive and negative signed residuals balance exactly, so their sum is zero:

This is a statement about the signed residuals. It does not say that the distances from the line add to zero, or that the residuals all equal zero. Positive and negative values can cancel even when individual residuals are not small. The mean residual is also zero because the sum of the residuals divided by the number of observations is zero.

The property applies to the same observations used to fit the line. It does not claim that residuals for a new set of cases, or for future predictions, must sum to zero. It also relies on fitting a line with an intercept; a regression forced through the origin does not generally have this residual-sum property.

The Slope and Correlation Have the Same Sign

In “Linking the Slope to Correlation,” you learned the relationship between the least-squares slope and the correlation:

When both variables vary, \(s_x\) and \(s_y\) are positive. Therefore, the ratio \(s_y/s_x\) is positive, and multiplying \(r\) by that positive number does not change its sign. A positive \(r\) gives a positive slope, and a negative \(r\) gives a negative slope. If \(r=0\), the slope is zero.

This is a sign relationship, not an equality between the numerical values of \(b\) and \(r\). The slope has units of response units per predictor unit, while \(r\) is unitless. The magnitude of the slope also depends on the relative standard deviations of the two variables. As covered in “Effect of Changing Units on the Regression Line,” changing measurement units can change the slope’s numerical value without changing its direction.

How to Use These Properties Carefully

These facts are useful checks, but each has a specific meaning. Keep the following distinctions clear when explaining or using them:

• The mean point is a point on the fitted line. It does not have to be one of the observed data points.
• The residual sum is a signed total. Opposite signs cancel. A zero sum does not mean the observations all match their predictions.
• The sign comparison concerns direction. A positive slope and positive \(r\) indicate an upward linear direction; a negative slope and negative \(r\) indicate a downward direction.
• The slope and correlation have different units and meanings. The slope describes predicted response change per predictor unit. Correlation summarizes the direction and strength of a linear association and has no units.
• Use the line for the data it fits. The residual-sum property describes the observations used to calculate that least-squares line, not an unrelated sample.

Displayed regression coefficients are often rounded. If you substitute \(\bar{x}\) into an equation with rounded \(a\) and \(b\), the result might be slightly different from \(\bar{y}\). Likewise, adding residuals calculated from rounded predictions might give a small value near zero rather than exactly zero. That small discrepancy can be due to rounding; it does not by itself contradict the least-squares properties. Use full calculator precision when available, and report sensible rounded results.

Common Mistakes and AP Exam Tips

• Claiming every residual is zero. The property is that the signed residuals sum to zero. A full-credit explanation notes that positive and negative residuals can cancel.
• Adding absolute residuals instead. The absolute values \(|y-\hat{y}|\) are nonnegative and generally do not sum to zero. The property applies to \(y-\hat{y}\) with its sign preserved.
• Calling \((\bar{x},\bar{y})\) an observed case. It is the point formed from the two sample means and lies on the line, but it may not correspond to any individual in the data.
• Saying the slope equals \(r\). The correct relationship is \(b=r(s_y/s_x)\). The positive standard-deviation ratio ensures the same sign when both variables vary.
• Ignoring the variables’ roles. The slope sign describes the direction of predicted \(y\) as \(x\) increases. Keep the predictor and response fixed when explaining the direction.
• Overreacting to a small numerical mismatch. Rounded coefficients or predictions can prevent a calculator check from showing exact equality. State that the properties hold for the unrounded least-squares fit and recognize rounding effects.

For an AP-style response, name the property precisely and connect it to the data. For example: “The least-squares line with an intercept passes through \((\bar{x},\bar{y})\), so at the mean practice time the line predicts the mean performance score.” For residuals, specify that the signed residuals for the fitted observations sum to zero. For the slope, state that it has the same sign as \(r\) because \(s_y/s_x\) is positive when both variables vary.