Why the Mean Minimizes Squared Deviations

Why Ask Which Center Is Best?

A data set can be summarized by a center, but different choices of center reward different notions of closeness. The median minimizes the total absolute distance from the observations. For squared distance, a different center is best: the arithmetic mean. This result is more than a rule to memorize. An identity shows exactly how much the total squared distance increases when a proposed center moves away from the mean.

The result applies to any finite collection of numerical observations. It does not require the observations to follow a probability model, have a particular shape, or come from a random sample. The key condition is simply that the same observations are used when comparing candidate centers.

The Sum of Squared Distances

Definition: For numerical observations $x_1,x_2,\ldots,x_n$ and a proposed center $a$, the sum of squared distances from the observations to $a$ is $$ Q(a)=\sum_{i=1}^{n}(x_i-a)^2. $$ The arithmetic mean of the observations is $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$. The claim is that $Q(a)$ is smallest when $a=\bar{x}$.

Here, $a$ is any real number that could be used as a center; it does not have to be one of the observed values. The number $n$ is at least 1, and the observations are treated as fixed. Squaring makes a distance below the center contribute just as much as a distance of the same size above it. It also makes larger distances count more heavily: a distance of 4 contributes 16, while a distance of 2 contributes 4.

Each observation and the center are measured in the same units. Thus $Q(a)$ is measured in squared units. For example, if the observations are in minutes, the sum of squared distances is in minutes squared. The unit does not affect which center makes the sum smallest.

The Key Identity

To compare a proposed center $a$ with the mean, express each observation’s distance from $a$ as two parts: its distance from the mean, plus the difference between the mean and $a$. Expanding the resulting square separates the contribution from the mean from the extra contribution caused by using $a$.

Theorem: For any $n\geq 1$ real observations and any real number $a$, $$ \sum_{i=1}^{n}(x_i-a)^2 = \sum_{i=1}^{n}(x_i-\bar{x})^2+n(a-\bar{x})^2. $$ Consequently, the sum of squared distances is minimized uniquely at $a=\bar{x}$.

The proof uses the fact that deviations from the arithmetic mean sum to zero. This follows directly from the definition of $\bar{x}$:

\sum_{i=1}^{n}(x_i-\bar{x}) = \sum_{i=1}^{n}x_i-n\bar{x} = \sum_{i=1}^{n}x_i-n\left(\frac{1}{n}\sum_{i=1}^{n}x_i\right) =0.

Now write $x_i-a=(x_i-\bar{x})+(\bar{x}-a)$ and expand each square. Summing gives

\begin{aligned} \sum_{i=1}^{n}(x_i-a)^2 &=\sum_{i=1}^{n}\left[(x_i-\bar{x})+(\bar{x}-a)\right]^2\\ &=\sum_{i=1}^{n}(x_i-\bar{x})^2 +2(\bar{x}-a)\sum_{i=1}^{n}(x_i-\bar{x}) +\sum_{i=1}^{n}(\bar{x}-a)^2\\ &=\sum_{i=1}^{n}(x_i-\bar{x})^2+n(\bar{x}-a)^2\\ &=\sum_{i=1}^{n}(x_i-\bar{x})^2+n(a-\bar{x})^2. \end{aligned}

The middle term vanishes because the deviations from $\bar{x}$ sum to zero. The final term is $n$ copies of the same squared difference. Since a square cannot be negative,

Q(a)-Q(\bar{x})=n(a-\bar{x})^2\geq 0.

If $a\neq\bar{x}$, then $n>0$ and $(a-\bar{x})^2>0$, so $Q(a)>Q(\bar{x})$. If $a=\bar{x}$, the extra term is zero. This proves both that the mean achieves the minimum and that no other center achieves the same minimum.

Worked Example: Comparing Centers for Three Observations

A hypothetical set of three daily practice times is 2, 4, and 9 minutes. Compare the sum of squared distances using the mean with the sum using a proposed center of 4 minutes.

First calculate the mean: $$ \bar{x}=\frac{2+4+9}{3}=\frac{15}{3}=5\text{ minutes}. $$ At the mean, the sum of squared distances is $$ Q(5)=(2-5)^2+(4-5)^2+(9-5)^2=9+1+16=26\text{ minutes}^2. $$

The identity gives the total at the proposed center $a=4$: $$ Q(4)=Q(5)+3(4-5)^2=26+3(1)=29\text{ minutes}^2. $$ A direct check agrees: $$ Q(4)=(2-4)^2+(4-4)^2+(9-4)^2=4+0+25=29\text{ minutes}^2. $$

The mean gives a smaller total: 26 rather than 29 minutes squared. The increase is exactly $3(4-5)^2=3$ minutes squared, as the identity predicts.

Reading the Identity

The identity separates the total for any proposed center into two parts. The first, $\sum_{i=1}^{n}(x_i-\bar{x})^2$, is the total at the mean. It depends on the observations but not on the proposed center. The second, $n(a-\bar{x})^2$, is the additional amount paid for choosing a center $a$ instead. The farther $a$ is from $\bar{x}$, the larger this extra amount becomes, in proportion to the square of that distance.

For example, moving a proposed center twice as far from the mean makes the extra amount four times as large, because $(2d)^2=4d^2$. This statement concerns the additional squared-distance total, not the total itself.

Worked Example: Predicting the Cost of Moving the Center

Suppose four hypothetical measurements, in centimetres, have mean 6 cm and a total squared distance of 20 cm² from that mean. Find the total squared distance from the measurements to a proposed center of 8 cm.

Here, $n=4$, $\bar{x}=6$, and $a=8$. Apply the identity: $$ Q(8)=Q(6)+4(8-6)^2=20+4(2^2)=20+16=36\text{ cm}^2. $$

The new center is 2 cm from the mean, so the added amount is 16 cm². No individual measurement values are needed to calculate this new total, provided the mean and the total at the mean are known.

A Direct Check With a Different Data Set

The identity is useful even when we can also calculate every squared distance directly. The direct calculation gives a check on the formula and makes clear that the result concerns all observations together, not just one distance.

Worked Example: Verifying the Identity With Four Measurements

A hypothetical set of four completion times is 12, 14, 15, and 19 minutes. Find the sum of squared distances from the mean, then find the sum from a proposed center of 16.5 minutes.

The mean is $$ \bar{x}=\frac{12+14+15+19}{4}=\frac{60}{4}=15\text{ minutes}. $$ The total at the mean is $$ Q(15)=(12-15)^2+(14-15)^2+(15-15)^2+(19-15)^2 =9+1+0+16=26\text{ minutes}^2. $$

The identity predicts that using $a=16.5$ minutes adds $$ 4(16.5-15)^2=4(1.5^2)=4(2.25)=9\text{ minutes}^2. $$ Therefore, $$ Q(16.5)=26+9=35\text{ minutes}^2. $$

Direct calculation confirms the answer: $$ Q(16.5)=(12-16.5)^2+(14-16.5)^2+(15-16.5)^2+(19-16.5)^2 =20.25+6.25+2.25+6.25=35\text{ minutes}^2. $$ The mean is the unique minimizing center, and the proposed center’s total is 9 minutes squared larger.

Why This Result Matters

A center is often used as a single representative value for a collection of observations. If the penalty for an observation is its squared distance from that center, the identity proves that the arithmetic mean is the best representative under that specific criterion. This gives a precise reason for using the mean in settings where squared errors are the quantity to minimize.

The criterion matters. The result does not say that the mean is best for every purpose or every way of measuring distance. If the total absolute distance is the criterion, the median minimizes that total. If a center is chosen to minimize squared distance, however, the arithmetic mean is uniquely optimal. The data and the loss being minimized determine which center is appropriate.

The squared-distance total is closely related to variance: variance is obtained by dividing a squared-deviation total by a fixed positive divisor. Dividing all candidate totals by the same positive number does not change which candidate is smallest. The identity itself is about the sum of squared distances and does not require choosing a variance denominator.

Common Mistake: Confusing Signed and Squared Deviations

A tempting but incorrect argument is: “The mean must minimize the total distance because the deviations from the mean add to zero.” The sum of signed deviations is not the sum of distances or the sum of squared distances. In fact, for any proposed center $a$, the signed deviations add to $$ \sum_{i=1}^{n}(x_i-a)=n(\bar{x}-a). $$ This quantity can be negative, zero, or positive; it does not measure total squared distance. The mean’s special property for squared distances follows from the identity, not merely from the signed sum being zero.

Another mistake is to conclude that every center other than the mean must produce a much larger total. The exact increase is $n(a-\bar{x})^2$, so it depends on both the sample size and the distance between the proposed center and the mean. A center very close to the mean may give a total only slightly larger; a center farther away gives a larger increase.

Key takeaway: For a fixed finite data set, the total squared distance from a proposed center $a$ equals the total at the mean plus $n(a-\bar{x})^2$. That extra term is never negative and is zero only when $a=\bar{x}$. Therefore the arithmetic mean uniquely minimizes the sum of squared distances.

Check Your Understanding

Use the identity to answer these questions. Keep the units squared when the observations have measurement units.

For observations 3, 5, and 10, calculate the mean and the sum of squared distances from the mean.
A data set has $n=6$, mean 4, and a squared-distance total of 30 at the mean. What is the total at proposed center $a=5$?
For any $n\geq 1$, explain why $n(a-\bar{x})^2$ proves that no center other than the mean can tie the mean’s squared-distance total.
Why does the fact that signed deviations sum to zero not, by itself, prove that the mean minimizes squared distances?
If a proposed center is 3 units from the mean, what is the additional squared-distance total compared with a center 1 unit from the mean, for the same data set?