How Outliers Alter Correlation

Why One Observation Can Matter

A scatterplot can show a clear linear pattern, yet a single unusual point may change how strong that pattern appears. The point might reinforce the overall direction, blur it, or even make the computed correlation have the opposite sign. This matters because Pearson’s correlation summarizes the linear association across all paired observations; it does not simply describe the main cluster of points.

The effect depends on where a point lies relative to the other observations in both coordinates. A point far out in the horizontal direction can have a different effect from one far out vertically, and being unusual in one coordinate alone does not determine whether the point will strengthen or weaken the correlation. The calculations below compare the same starting data with and without a specified added point.

Definition: For $n$ paired quantitative observations $(x_i,y_i)$, Pearson’s sample correlation is $$ r=\frac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}, \qquad S_{xy}=\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y}), \qquad S_{xx}=\sum_{i=1}^{n}(x_i-\bar{x})^2, \qquad S_{yy}=\sum_{i=1}^{n}(y_i-\bar{y})^2. $$ Here $\bar{x}$ and $\bar{y}$ are the sample means of the two variables. The calculation requires variation in both variables, so $S_{xx}$ and $S_{yy}$ must be positive. The sign of $r$ describes the direction of the linear association, while its absolute value describes its strength.

Correlation is calculated from the full paired data set, not by choosing a “typical” subset of points. To assess a point’s influence, compare the correlation calculated with that observation to the correlation calculated without it, and examine the scatterplot as well. There is no universal numerical cutoff that defines an outlier or guarantees how much a point must change $r$ to count as influential.

How Adding a Point Changes the Centered Sums

A useful way to understand an added observation is to calculate its displacement from each original sample mean. Suppose the original data contain $n$ pairs, with means $\bar{x}$ and $\bar{y}$, and centered sums $S_{xx}$, $S_{yy}$, and $S_{xy}$. Add one new pair $(x_0,y_0)$, and let $\delta_x=x_0-\bar{x}$ and $\delta_y=y_0-\bar{y}$. The updated centered sums are

S_{xx}'=S_{xx}+\frac{n}{n+1}\delta_x^2,\qquad S_{yy}'=S_{yy}+\frac{n}{n+1}\delta_y^2,\qquad S_{xy}'=S_{xy}+\frac{n}{n+1}\delta_x\delta_y.

The primes indicate sums after adding the point. These formulas show that the new point contributes to the updated sums through its displacement in both coordinates. For the cross-product sum, the sign of $\delta_x\delta_y$ matters: when the displacements have the same sign, the added contribution is positive; when they have opposite signs, it is negative. The correlation must then be recalculated using all three updated sums.

A Short Derivation of the Update Formula

After adding $x_0$, the new mean of the $x$-values is $\bar{x}'=\bar{x}+\delta_x/(n+1)$. For an original observation, its displacement from the new mean is its old displacement minus $\delta_x/(n+1)$. The new point’s displacement from the new mean is $n\delta_x/(n+1)$. Expanding the sum of squared displacements gives

\begin{aligned} S_{xx}' &= \sum_{i=1}^{n}\left[(x_i-\bar{x})-\frac{\delta_x}{n+1}\right]^2 +\left[\frac{n\delta_x}{n+1}\right]^2\\ &=S_{xx}-\frac{2\delta_x}{n+1}\sum_{i=1}^{n}(x_i-\bar{x}) +\frac{n\delta_x^2}{(n+1)^2} +\frac{n^2\delta_x^2}{(n+1)^2}\\ &=S_{xx}+\frac{n}{n+1}\delta_x^2. \end{aligned}

The middle term vanishes because the original deviations from their mean sum to zero: $\sum_{i=1}^{n}(x_i-\bar{x})=0$. The two remaining added terms combine to $(n+n^2)\delta_x^2/(n+1)^2=n\delta_x^2/(n+1)$. Repeating the same expansion for the $y$-values gives the formula for $S_{yy}'$. Expanding the products of the updated $x$- and $y$-deviations gives the formula for $S_{xy}'$; its cross-terms involving a sum of old deviations vanish for the same reason.

Worked Examples: One Point, Different Effects

Use the same four hypothetical starting pairs in each example: $(1,1),(2,3),(3,2),(4,4)$. The means are $\bar{x}=2.5$ and $\bar{y}=2.5$. Their centered sums are $S_{xx}=5$, $S_{yy}=5$, and $S_{xy}=4$, so the original correlation is $r=4/\sqrt{5\cdot5}=0.8000$. These points have a positive linear association. Each example adds one hypothetical pair and compares the new $r$ with this baseline.

Worked Example: An Observation Strengthens Correlation

Add the point $(10,5)$. It lies to the right of the original $x$-values and somewhat above the original $y$-mean. Its displacements from the original means are $\delta_x=10-2.5=7.5$ and $\delta_y=5-2.5=2.5$. With $n=4$, the update factor is $n/(n+1)=4/5=0.8$.

The updated cross-product sum is $S_{xy}'=4+0.8(7.5)(2.5)=4+15=19$. The updated sums of squares are $S_{xx}'=5+0.8(7.5)^2=5+45=50$ and $S_{yy}'=5+0.8(2.5)^2=5+5=10$.

Therefore, $r'=19/\sqrt{50\cdot10}=19/\sqrt{500}\approx0.8497$. The positive correlation is stronger in magnitude than the original $0.8000$. The added point reinforces the positive direction enough to increase the overall linear association, even though it is separated from the original group.

Worked Example: An Observation Weakens Correlation

Instead, add $(10,3)$ to the original four pairs. The $x$-displacement is still $7.5$, but the $y$-displacement is only $3-2.5=0.5$. The point is far to the right, but its vertical position does not continue the original positive pattern as strongly as the previous point did.

The updated sums are $S_{xy}'=4+0.8(7.5)(0.5)=4+3=7$, $S_{xx}'=5+0.8(7.5)^2=50$, and $S_{yy}'=5+0.8(0.5)^2=5.2$.

Thus, $r'=7/\sqrt{50\cdot5.2}=7/\sqrt{260}\approx0.4341$. The correlation remains positive, but its magnitude falls from $0.8000$ to $0.4341$. The observation weakens the measured linear association; it does not reverse its direction.

Worked Example: An Observation Reverses Correlation

Now add $(10,0)$. Its displacements are $\delta_x=7.5$ and $\delta_y=-2.5$, which have opposite signs. The point is far to the right but below the original $y$-mean, so its added cross-product contribution goes against the original positive association.

The updated cross-product sum is $S_{xy}'=4+0.8(7.5)(-2.5)=4-15=-11$. The updated sums of squares are $S_{xx}'=5+0.8(7.5)^2=50$ and $S_{yy}'=5+0.8(-2.5)^2=10$.

The new correlation is $r'=-11/\sqrt{50\cdot10}=-11/\sqrt{500}\approx-0.4919$. The sign has reversed: the full data set now has a negative linear association. The original four observations still have $r=0.8000$ on their own, so it is essential to report that the reversal occurs only after including the added point.

Being Unusual on One Axis Is Not the Whole Story

A point can look unusual horizontally, vertically, or in both directions. A point far from the other $x$-values is not automatically influential for correlation: its $y$-position also matters. Likewise, a point far in the $y$-direction can alter the cross-products and the spread in $y$, even if its $x$-value is close to the original mean. The next calculations separate these situations using the same starting data.

Worked Example: Unusual in One Coordinate

First add $(10,2.5)$, a point far to the right but exactly at the original $y$-mean. Its displacements are $\delta_x=7.5$ and $\delta_y=0$. Hence $S_{xy}'=4+0.8(7.5)(0)=4$, $S_{xx}'=5+0.8(7.5)^2=50$, and $S_{yy}'=5$. The correlation becomes $4/\sqrt{50\cdot5}=4/\sqrt{250}\approx0.2530$.

Although the added point contributes nothing to the updated cross-product sum, it greatly increases the spread in $x$, reducing the correlation’s magnitude. A point unusual only horizontally can therefore weaken $r$.

Now consider a point unusual mainly vertically: add $(2,10)$. Its displacements are $\delta_x=-0.5$ and $\delta_y=7.5$. The updated sums are $S_{xy}'=4+0.8(-0.5)(7.5)=1$, $S_{xx}'=5+0.8(0.5)^2=5.2$, and $S_{yy}'=5+0.8(7.5)^2=50$. Thus $r'=1/\sqrt{5.2\cdot50}=1/\sqrt{260}\approx0.0620$.

This point also weakens the original positive correlation substantially. These examples show why “unusual in one coordinate” is not enough to predict the effect: the point’s position relative to both means affects the cross-product, and its distances affect the sums of squares in the denominator.

How to Judge Influence Responsibly

For correlation, an observation is influential when including it makes a substantial difference to the correlation’s value or interpretation. This is a comparison, not a label that can be assigned from the point’s distance alone. A practical assessment has three parts:

Inspect the scatterplot.
Look for the main pattern and any point that is separated from the rest horizontally, vertically, or both. Check the axes and scales so apparent distance is interpreted correctly.

Compare the correlations.
Calculate $r$ with all valid pairs, then calculate it again without the suspected point as a diagnostic comparison. Record both values and describe any change in sign or magnitude.

Investigate before deciding.
Check whether the observation is a recording or measurement error. If it is valid, do not remove it solely because it changes $r$; explain the sensitivity and retain the point when it belongs to the target data.

This comparison is descriptive. It does not establish that the unusual observation is erroneous, nor does it explain why the point differs. If a point is valid, reporting both the full-data result and a clearly labelled sensitivity comparison can show readers how dependent the summary is on that observation. The reason for any exclusion must be justified independently of whether exclusion produces a preferred result.

Common Mistake

Common mistake: “This point is far from the others, so it must make the correlation weaker.” Distance by itself does not determine the effect. A point far from the means in both coordinates on the same side can strengthen a positive correlation, while a point far out in one coordinate or on the opposite side can weaken or reverse it. Calculate the updated correlation and inspect the paired positions before drawing a conclusion.

A second mistake is to call a point influential simply because its value is extreme on one axis. “Unusual” describes its position; “influential for correlation” describes how much the computed $r$ changes when it is included. The distinction prevents a visual impression from substituting for a comparison.

Key takeaway: An unusual observation can strengthen, weaken, or reverse Pearson’s correlation. Its effect depends on its paired location relative to both means and on how it changes both the cross-product sum and the sums of squares. Judge influence by comparing correlations with and without the point, while treating valid observations responsibly.

Check Your Understanding

Use the ideas and update formulas in this tutorial to answer the questions.

For the original four pairs, what happens to the sign and magnitude of $r$ when $(10,0)$ is added? What feature of the point helps explain the result?
Why does adding $(10,2.5)$ change the correlation even though its displacement from the original $y$-mean is zero?
Can a point unusual only in its $y$-coordinate substantially affect $r$? Explain using the updated sums or the worked example.
What does “influential for correlation” mean, and how would you assess whether a particular observation is influential?
Why is it not acceptable to remove a valid observation solely because removing it produces a correlation that seems more reasonable?

What You'll Learn