How Extreme Observations Affect Summaries

Why the Same Data Change Can Affect Summaries Differently

A numerical summary compresses a data set into a small amount of information. But summaries do not all respond to a change in the data in the same way. If one unusually high value is replaced by a much larger one, the mean and standard deviation can change substantially, even when the middle of the ordered data remains exactly the same. The median and interquartile range often change less in this situation because they depend on the data’s middle positions rather than on how far an extreme value lies from the rest.

This difference matters when describing data with outliers or other extreme observations. It does not mean that one pair of summaries is always preferable. It means that the summaries answer different questions and have different sensitivities to changes in individual observations.

Definition: An observation is extreme relative to a data set when it lies far from most of the other observations. The mean is the sum of the observations divided by their number. The median is the middle value after the data are ordered, or the average of the two middle values when the sample size is even. The interquartile range is \(IQR=Q_3-Q_1\), the difference between the third and first quartiles. The sample standard deviation \(s\) describes spread around the sample mean; for calculations here, use \(s=\sqrt{\frac{\sum (x_i-\bar{x})^2}{n-1}}\).

For quartiles in the examples, use the median-of-halves convention: when there is an odd number of observations, leave the overall median out and find the medians of the lower and upper halves. The convention matters, especially for small data sets. A single extreme value can leave the quartiles unchanged in one set of data and alter a quartile in another.

A Large Change to the Largest Observation

Consider the ordered observations \(10,12,13,14,15,16,18\). Their middle value is 14. The lower half is \(10,12,13\), and the upper half is \(15,16,18\), so \(Q_1=12\), \(Q_3=16\), and \(IQR=4\). Now replace the largest value, 18, with 100. The changed data remain ordered, with the same first six observations and the same middle position.

Worked Example: Replacing a High Value

Compare the mean, sample standard deviation, median, and interquartile range for \(10,12,13,14,15,16,18\) and \(10,12,13,14,15,16,100\).

Original mean: The sum is \(98\), so \(\bar{x}=98/7=14\). After replacing 18 with 100, the sum is \(180\), and the new mean is \(180/7=25.714\). The mean rises by \(82/7=11.714\).

Original sample standard deviation: The sum of squares is \(1414\): \(100+144+169+196+225+256+324=1414\). Using the computational form of the sample standard deviation,

s=\sqrt{\frac{\sum x_i^2-(\sum x_i)^2/n}{n-1}} =\sqrt{\frac{1414-98^2/7}{6}} =\sqrt{7} \approx 2.646.

For the changed data, the sum of squares is \(1414-18^2+100^2=11090\). Therefore,

s=\sqrt{\frac{11090-180^2/7}{6}} \approx 32.816.

Median and IQR: In both data sets, the middle observation is 14. The lower half is \(10,12,13\), with median 12, and the upper half is \(15,16,18\) or \(15,16,100\), with median 16. Thus \(IQR=16-12=4\) in both cases.

The changed value raises the mean by about 11.7 units and increases the sample standard deviation from about 2.65 to 32.8 units. In contrast, the median and IQR are unchanged. The quartiles stay the same because the modified observation remains the largest value and is not one of the middle positions used to calculate them.

Why the Mean and Standard Deviation Respond

The mean responds to an observation’s value directly. Suppose one observation \(x_j\) is replaced by \(x_j+d\), where \(d\) is the amount of change. The total of all observations then increases by \(d\), while the sample size stays \(n\). Dividing the new total by \(n\) gives the new mean:

\bar{x}_{\mathrm{new}}=\frac{\sum x_i+d}{n} =\bar{x}+\frac{d}{n}.

So changing one observation by \(d\) changes the mean by exactly \(d/n\). A larger sample spreads that particular change over more observations, but does not make the mean completely insensitive to a very large change.

The standard deviation is also affected by distance from the mean: it is calculated from squared deviations. For clarity, let \(T=\sum (x_i-\bar{x})^2\), the sum of squared deviations for the original data. If \(S=\sum x_i\), replacing \(x_j=x\) by \(x+d\) changes the sum of the observations to \(S+d\) and the sum of their squares to \(\sum x_i^2+2xd+d^2\). The identity \(T=\sum x_i^2-S^2/n\) therefore gives the change in the sum of squared deviations:

T_{\mathrm{new}}-T =2xd+d^2-\frac{2Sd+d^2}{n} =2d(x-\bar{x})+d^2\left(1-\frac{1}{n}\right).

This calculation shows why the size of the change can matter so much: the expression includes a term proportional to \(d^2\). When a value is moved a very long distance, its squared deviation can grow rapidly. Dividing the sum of squared deviations by \(n-1\) and taking the square root gives \(s\), so a large increase in squared deviations can produce a substantial increase in the sample standard deviation.

Worked Example: Changing a Low Extreme Value

Suppose seven hypothetical daily temperatures, in degrees Celsius, are \(20,22,24,26,28,30,32\). Compare their summaries with those for \(0,22,24,26,28,30,32\), in which the lowest value has been changed from 20 to 0.

Mean: The original sum is \(182\), giving \(\bar{x}=182/7=26\) °C. The changed sum is \(162\), giving \(\bar{x}=162/7=23.143\) °C. The mean falls by \(20/7=2.857\) °C, as expected from changing one observation by \(-20\) °C in a sample of seven.

Sample standard deviation: The original sum of squares is \(4844\), and its sum of squared deviations is \(4844-182^2/7=112\). Thus \(s=\sqrt{112/6}\approx4.320\) °C. For the changed data, the sum of squares is \(4844-20^2=4444\), so

s=\sqrt{\frac{4444-162^2/7}{6}} \approx 10.762\text{ °C}.

Median and IQR: Both ordered lists have median 26 °C. Their lower halves have median 22 °C and their upper halves have median 30 °C, so both have \(IQR=30-22=8\) °C. The standard deviation changes much more than these middle-based summaries, even though the extreme value has moved downward rather than upward.

Resistance Is Not the Same as Invariance

The examples show a common pattern, not an unconditional guarantee. When a changed extreme value stays outside the central positions, the median and quartiles can remain fixed even while the distance between that value and the rest of the data grows. But the IQR depends on quartile positions, and those positions can include an extreme observation, particularly in a small sample. In those cases, changing an extreme value can change the IQR.

Worked Example: A Small Sample Where the IQR Changes

Consider \(1,2,3,4,5\), then replace the largest observation, 5, with 20. Find the summaries using the median-of-halves convention.

Original data: The mean is \(15/5=3\), and the median is 3. The halves are \(1,2\) and \(4,5\), so \(Q_1=1.5\), \(Q_3=4.5\), and \(IQR=3\). The sum of squares is 55, giving

s=\sqrt{\frac{55-15^2/5}{4}} =\sqrt{2.5} \approx 1.581.

Changed data: The mean is \(30/5=6\), while the median remains 3. The halves are now \(1,2\) and \(4,20\), so \(Q_1=1.5\), \(Q_3=12\), and \(IQR=10.5\). The sum of squares is \(430\), so

s=\sqrt{\frac{430-30^2/5}{4}} =\sqrt{62.5} \approx 7.906.

Here the median does not change, but the IQR does because, under this convention, the upper half used to find \(Q_3\) contains the extreme observation. The standard deviation also increases markedly. This example is a useful reminder to check the positions used to calculate quartiles rather than assume that the IQR is always unchanged.

Common Mistake: Treating “Robust” as “Unaffected”

A common mistake is to reason, “The median and IQR ignore extreme values, so they cannot change when an extreme observation changes.” That is too strong. These summaries are often less sensitive to the magnitude of extreme observations because they are based on ordered positions, but an extreme value can still affect a position used to find a quartile, as the five-observation example shows.

The reverse overstatement is also misleading: the mean and standard deviation do not always change dramatically whenever a value is described as extreme. The effect depends on how far the value moves, the sample size, and the other observations. A sound comparison calculates or checks the summaries for the actual data rather than relying only on labels such as “outlier” or “robust.”

Key takeaway: Moving one observation by \(d\) changes the mean by \(d/n\). The sample standard deviation can be especially sensitive to a large movement because it is based on squared deviations. The median and IQR often change less when the central ranks remain in place, but the IQR is not guaranteed to stay fixed.

Check Your Understanding

Try these questions using the median-of-halves convention where quartiles are needed.

If one observation in a sample of 10 is increased by 30 units, by how much does the mean increase?
Why can moving the largest observation farther above all the others increase the standard deviation substantially?
For \(2,4,5,6,8\), replace 8 by 50. Does the median change? Does the IQR change? Explain using the positions in the two halves.
Explain why “the IQR is resistant to extreme values” does not mean “the IQR can never change.”
Two data sets have the same median, but one has a much larger standard deviation. What does that suggest about the observations’ spread around their means?