Regression Toward the Mean

Why Extreme Values Lead to Less Extreme Predictions

In “Standardized Version of the Regression Line,” you learned that the predicted response z-score is \(\hat{z}_y=r z_x\). This equation gives a direct explanation for a familiar pattern: when the correlation is positive but less than 1, a predictor value far above its mean leads to a response prediction above its mean, but usually not as far above. A predictor value far below its mean leads to a prediction below its mean, but usually not as far below.

This pattern is called regression toward the mean. It refers to the regression line’s predictions: values of the predicted response tend to be less extreme, in standard-deviation units, than the corresponding predictor values when the variables have a positive correlation weaker than a perfect correlation. It does not mean the observed response for every individual must be closer to its mean.

The key is the size of \(r\). A positive correlation means the prediction is on the same side of its mean as the predictor. Because a correlation has absolute value no greater than 1, multiplying by a positive \(r\) smaller than 1 reduces the predictor’s distance from zero on the standardized scale. If \(r=1\), the predicted z-score is just as extreme as the predictor z-score. If \(r=0\), the predicted response z-score is 0 for every predictor value: the line predicts the response mean.

The word “regression” here does not mean that a person, family, or population necessarily changes or improves. It describes a feature of predictions from a regression line. The line captures the overall linear pattern, while individual responses vary around that pattern.

A Parent-Child Height Example

Consider an invented example involving adult parents and their adult children of the same gender. Let \(x\) be a parent’s height and \(y\) be the adult child’s height, both measured in centimeters. Suppose a fictional sample has \(\bar{x}=170\) cm, \(s_x=8\) cm, \(\bar{y}=170\) cm, \(s_y=7\) cm, and \(r=0.60\). These summaries are chosen for illustration; they are not results from a real study.

A parent who is much taller than the sample mean has a positive \(z_x\). Since \(r=0.60\), the predicted child z-score is positive too, but only 60% as large as the parent’s z-score. The same pattern applies in the other direction: a very short parent leads to a below-average predicted child height, but the prediction is not as far below the child-height mean in standard-deviation units.

What Happens at the Low End?

Regression toward the mean works symmetrically in this positive-association example. A parent who is unusually short relative to the parent-height sample mean has a negative z-score. Multiplying by a positive correlation gives a negative predicted child z-score as well, but with a smaller absolute value when \(0<r<1\). The prediction remains below the child-height mean, just not as far below in standard-deviation units.

A Prediction Is Not a Guarantee

The word “tendency” matters. A regression line predicts an average response for a given predictor value; it does not say that every observed response will equal its prediction. In the parent-child example, a tall parent can have an adult child who is taller than the line predicts. Another tall parent can have a child who is shorter than predicted. As covered in “Prediction Versus Observed Values,” the difference between an observed response and its predicted value is the residual, \(y-\hat{y}\).

For example, a parent 182 cm tall is \(1.5\) standard deviations above the parent-height mean. The line predicts a child height \(0.60(1.5)=0.90\) standard deviations above the child-height mean, or \(170+(0.90)(7)=176.3\) cm. If one fictional child in the sample is actually 181 cm tall, the residual is \(181-176.3=4.7\) cm. That child’s actual height is more extreme than the line’s prediction, but this does not contradict regression toward the mean. The concept describes the prediction, not a rule that every observed child must be less extreme than their parent.

The same distinction applies if a person is selected because of an extreme response. For instance, a student chosen for an unusually high score on one test might not score as far above average on another test. A regression line can predict a less extreme score when the two measurements are positively associated but imperfectly correlated. That is a prediction pattern, not proof that the first score caused the second score to change.

How Correlation Strength Affects the Pattern

The closer a positive \(r\) is to 1, the less the prediction is pulled toward the response mean in standardized units. For example, if \(z_x=2\), then \(r=0.90\) gives \(\hat{z}_y=1.80\), while \(r=0.40\) gives \(\hat{z}_y=0.80\). Both predictions are above the response mean, but the first is more extreme. If \(r=0\), the line predicts the response mean, regardless of how far the predictor is from its mean.

A negative correlation changes the direction of the prediction: a predictor above its mean leads to a predicted response below its mean. When \(-1<r<0\), multiplying by \(r\) also reduces the absolute standardized distance. This course’s usual parent-child height illustration has a positive association, so the predictor and predicted response are on the same side of their respective means.

Remember that standardized distances are relative to each variable’s own mean and standard deviation. Saying a parent is 2 standard deviations above the parent mean does not mean a child prediction is 2 centimeters, or even the same number of centimeters, from the child mean. Use \(s_x\) to standardize parent height and \(s_y\) to convert the predicted child z-score back to centimeters.

Common Mistakes and AP Exam Tips

• Claiming every child must be closer to the mean than the parent. Regression toward the mean describes the line’s prediction, not every observed parent-child pair. A full-credit response distinguishes the predicted child height from an individual child’s actual height.
• Comparing raw centimeter differences when the standard deviations differ. The claim that the prediction is less extreme is made in standard-deviation units. Compare \(|z_x|\) with \(|\hat{z}_y|\), not simply the raw distances in centimeters.
• Forgetting the sign of \(r\). Use \(r\), not \(|r|\), in \(\hat{z}_y=r z_x\). A negative correlation reverses the direction of the predicted response relative to the response mean.
• Calling regression toward the mean a causal effect. A regression line summarizes an association and makes predictions. It does not show that a parent’s height causes a child’s height to move toward an average.
• Using the wrong variable’s standard deviation. Standardize parent height with \(s_x\); convert a predicted child z-score with \(s_y\). Each variable has its own mean and standard deviation.
• Overstating a prediction outside the observed range. As discussed in “Predicting Within the Data Range,” a prediction for an extreme predictor value beyond the observed range is extrapolation and may be unreliable. Check the data range before treating the prediction as useful.

For an AP-style explanation, name the predictor and response, state what the line predicts, and compare the distances in standard-deviation units. For example: “Because the correlation between parent and adult child heights is positive but less than 1, a parent who is 2 standard deviations above the parent mean has a predicted child height only 1.2 standard deviations above the child mean. This is a tendency in the regression prediction, not a guarantee about an individual child.”