Why a Correlation Can Shrink When a Range Narrows
Imagine measuring study time and exam score across a broad group of students, then calculating the correlation again using only students whose study times fall within a narrow band. The second scatterplot may still slope upward, yet its points can look less closely aligned and its correlation can be smaller. The underlying tendency for scores to rise with study time need not have disappeared. The data included in the calculation have changed.
This effect matters whenever a sample represents only a limited part of a broader group. For example, a correlation calculated among students admitted to a selective program may differ from one calculated among all applicants because the admitted students have a narrower range of prior scores. A correlation describes the particular data analyzed; it does not automatically describe a broader population from which some values have been excluded.
Correlation measures the direction and strength of a linear pattern in the points included. If a scatterplot covers a wide range of \(x\)-values, a rising trend can be easier to distinguish from the vertical scatter around it. If only a narrow strip of those \(x\)-values is observed, the same amount of vertical scatter can look large relative to the small horizontal spread. The point cloud may therefore look less line-like, and the calculated correlation may be closer to zero.
A Before-and-After Data Example
The following hypothetical data give paired values of \(x\) and \(y\). The full set has a broad range of \(x\)-values. We will compare its scatterplot and correlation with those from a subset that keeps only the central \(x\)-values.
| Observation | \(x\) | \(y\) | Kept when \(x\) is restricted to \([-1,1]\)? |
|---|---|---|---|
| A | -2 | -3 | No |
| B | -1 | 0 | Yes |
| C | 0 | -1 | Yes |
| D | 1 | 2 | Yes |
| E | 2 | 1 | No |
Plotting all five pairs gives a generally rising cloud, with the points at \(x=-2\) and \(x=2\) helping to show the broad upward pattern. Keeping only observations B, C, and D removes both ends of the horizontal range. In the resulting scatterplot the points still have an upward tendency, but their vertical spread is relatively large compared with their narrower horizontal spread.
Worked Example: Correlation Before and After Restricting \(x\)
Calculate Pearson’s correlation for all five pairs, then for the three pairs with \(-1 \leq x \leq 1\). For a set of \(n\) paired observations, use \(r=S_{xy}/\sqrt{S_{xx}S_{yy}}\), where \(S_{xx}\) and \(S_{yy}\) are sums of squared deviations from their respective sample means, and \(S_{xy}\) is the sum of products of those deviations.
All five observations. The means are \(\bar{x}=0\) and \(\bar{y}=-0.2\). The centered values of \(x\) are \(-2,-1,0,1,2\); the centered values of \(y\) are \(-2.8,0.2,-0.8,2.2,1.2\). Therefore,
Thus the full-set correlation is \(r=10/\sqrt{10(14.8)}\approx 0.822\), a positive linear association.
Restricted set. The retained pairs are \((-1,0),(0,-1),(1,2)\), with \(\bar{x}=0\) and \(\bar{y}=1/3\). The centered values are \(-1,0,1\) for \(x\) and \(-1/3,-4/3,5/3\) for \(y\). Hence,
The restricted-set correlation is \(r=2/\sqrt{2(14/3)}\approx 0.655\). It is still positive, but its magnitude is smaller than the full-set value of about \(0.822\). Here, restricting \(x\) reduced the observed correlation; the calculation alone does not establish that every such restriction will do so.
A before-and-after scatterplot makes the comparison visible: the full plot shows all five points across \(x=-2\) to \(x=2\); the restricted plot shows only the three central points across \(x=-1\) to \(x=1\). The vertical deviations among those remaining points have not vanished, while the horizontal spread has contracted. Correlation standardizes both variables, so merely changing units or multiplying all \(x\)-values by a positive constant would not alter it. Removing observations, however, changes which deviations are included and can change the correlation.
Why Restricting \(x\) Can Attenuate Correlation
A simple model explains the mechanism. Suppose the relationship in a broader group follows \(Y=\beta X+\varepsilon\), where \(\beta\) is a fixed slope, the random error \(\varepsilon\) has mean zero and variance \(\sigma^2\), and \(\varepsilon\) is independent of \(X\). Assume the analysis keeps observations according to a range rule involving \(X\) alone. Write \(A\) for the event that an observation is kept. Because selection depends only on \(X\), the error remains independent of \(X\) within the selected group, with mean zero and variance \(\sigma^2\).
Within that group, the covariance between \(X\) and \(Y\) is \(\beta\operatorname{Var}(X\mid A)\). The variance of \(Y\) is \(\beta^2\operatorname{Var}(X\mid A)+\sigma^2\). Substituting these into the definition of correlation gives the following result.
This result is not a universal correction formula. It describes a specific linear model and a specific selection rule. Its value is to show, under clear assumptions, how narrowing the \(X\)-range can reduce the correlation even when the slope and the error variance stay the same.
Worked Example: Applying the Linear-Error Model
Suppose, hypothetically, that \(Y=2X+\varepsilon\), with \(\varepsilon\) independent of \(X\), mean zero, and variance \(9\). Compare the model-implied correlation when the variance of \(X\) is \(16\) with the correlation when range restriction reduces that variance to \(4\). The slope and error variance are unchanged.
For variance \(16\), the model gives
For variance \(4\), it gives
Under the stated model, reducing the variance of \(X\) lowers the correlation from about \(0.936\) to \(0.800\), despite the unchanged slope of \(2\). These are model-based values, not correlations calculated from a particular finite sample.
Restricting \(y\), or Restricting Both Variables
The simple model’s result concerns selection based on \(X\) alone. When selection depends on \(Y\), or on both coordinates, the selected points can have a different pattern for reasons beyond simply reducing the spread of \(X\). In particular, keeping observations only when \(Y\) lies inside a band can remove points in ways that alter the covariance as well as the variances. Therefore, a range restriction on \(Y\) can also attenuate correlation, but no single formula above applies to every such selection.
The same five-point data give a second before-and-after comparison. Keep only observations with \(-1\leq y\leq 2\). This retains B, C, D, and E: their pairs are \((-1,0),(0,-1),(1,2),(2,1)\). The scatterplot now omits the low-\(y\) point A. The retained points still show an upward tendency, but their correlation differs from that of the full set.
Worked Example: Restricting the Range of \(y\)
For all five observations, the earlier centered-sum calculation gives \(r=10/\sqrt{148}\approx0.822\). For the four pairs retained by \(-1\leq y\leq2\), the means are \(\bar{x}=0.5\) and \(\bar{y}=0.5\). Their centered \(x\)-values are \(-1.5,-0.5,0.5,1.5\), and their centered \(y\)-values are \(-0.5,-1.5,1.5,0.5\). Thus,
The restricted correlation is \(r=3/\sqrt{5(5)}=0.600\), smaller than the full-set value of about \(0.822\). In this particular data set, restricting \(y\) removes one endpoint that contributed to the full pattern. This example shows a possible outcome, not a guarantee for every \(y\)-range rule.
Restricting both variables at once may change the observed correlation still further. A selection rule based on both coordinates can keep or discard points according to their joint position in the scatterplot, affecting the pattern directly. It is not safe to assume that any restriction must make \(r\) closer to zero: depending on which observations are retained, a correlation can decrease, increase, or even change sign. Always inspect the selection rule and the resulting scatterplot.
Common Mistake and Practical Interpretation
A common mistake is to compare a correlation from a narrowly selected sample with one from a broad sample and conclude that the underlying relationship must have become weaker. That conclusion does not follow from the two correlations alone. Different ranges can produce different observed correlations even when a simple underlying linear trend remains similar. Conversely, range restriction is not proof that an unexpectedly small correlation would have been large in the broader group; the model assumptions and selection process matter.
When reporting or interpreting a correlation, describe whose observations were included and whether the range of either variable was limited. A correlation among people selected for a narrow score range describes the linear association in that selected group. Extending the interpretation to people outside that range requires additional evidence and assumptions. The scatterplot is useful because it shows both the included range and the spread around the visible trend.
Check Your Understanding
Use the ideas and calculations in this tutorial to answer the following questions.
- In your own words, what is range restriction, and what does it mean for a correlation to be attenuated?
- In the five-point example, why did the correlation change after keeping only the observations with \(-1\leq x\leq1\)? State both calculated correlations.
- Under the linear-error model, what assumptions allow the correlation formula for selection based on \(X\) alone to be derived?
- For the model \(Y=2X+\varepsilon\) in the worked example, what happened to the correlation when \(\operatorname{Var}(X)\) decreased from \(16\) to \(4\), and what stayed unchanged?
- Why should you not assume that restricting \(Y\), or restricting both variables, must always make the correlation smaller in magnitude?