Why Cluster Centers Are Arithmetic Means

Why the Center Is a Mean

In k-means clustering, a cluster is represented by a center, and its fit is measured by the sum of squared distances from its observations to that center. The center is not chosen merely because it looks typical: for this particular objective, the arithmetic mean has a precise minimizing property. That property explains the center-update calculation used in k-means.

The result applies to a nonempty set of numerical observations, with each observation given equal weight and distance measured by ordinary Euclidean distance. For one-dimensional data, the center is a number. For observations with several coordinates, the center is a vector, and the arithmetic mean is calculated separately in each coordinate.

Definition: For \(n\geq 1\) scalar observations \(x_1,\ldots,x_n\), the squared-distance objective at a proposed center \(c\) is \(Q(c)=\sum_{i=1}^{n}(x_i-c)^2\). For vectors \(\mathbf{x}_i=(x_{i1},\ldots,x_{ip})\) and center \(\mathbf{c}=(c_1,\ldots,c_p)\), it is \(Q(\mathbf{c})=\sum_{i=1}^{n}\|\mathbf{x}_i-\mathbf{c}\|^2=\sum_{i=1}^{n}\sum_{j=1}^{p}(x_{ij}-c_j)^2\). A center minimizing this objective is called a squared-distance-minimizing center.

The objective is the within-cluster sum of squares for the specified cluster. Squaring makes larger deviations count more heavily than smaller ones. The question here is limited to the best center for a given, fixed set of observations; it does not determine which observations belong in a cluster.

The Minimizing Center in One Dimension

For scalar observations, write \(\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i\) for their arithmetic mean. The objective \(Q(c)\) is a quadratic function of \(c\). Differentiating gives \(Q'(c)=2\sum_{i=1}^{n}(c-x_i)\). Setting the derivative to zero yields \(nc=\sum_i x_i\), so the only stationary point is \(c=\bar{x}\). Since \(Q''(c)=2n>0\), the function is strictly convex and this stationary point is its unique minimum.

A useful identity gives a direct comparison with any other proposed center. It also works for vectors:

\sum_{i=1}^{n}\|\mathbf{x}_i-\mathbf{c}\|^2 = \sum_{i=1}^{n}\|\mathbf{x}_i-\bar{\mathbf{x}}\|^2 +n\|\mathbf{c}-\bar{\mathbf{x}}\|^2, \qquad \bar{\mathbf{x}}=\frac{1}{n}\sum_{i=1}^{n}\mathbf{x}_i.

To derive the identity, write each difference as \(\mathbf{x}_i-\mathbf{c}=(\mathbf{x}_i-\bar{\mathbf{x}})+(\bar{\mathbf{x}}-\mathbf{c})\), then expand its squared length using the dot product:

\begin{aligned} \sum_{i=1}^{n}\|\mathbf{x}_i-\mathbf{c}\|^2 &=\sum_{i=1}^{n}\|\mathbf{x}_i-\bar{\mathbf{x}}\|^2 +2\left(\sum_{i=1}^{n}(\mathbf{x}_i-\bar{\mathbf{x}})\right)\mathbin{\cdot}(\bar{\mathbf{x}}-\mathbf{c}) +n\|\bar{\mathbf{x}}-\mathbf{c}\|^2. \end{aligned}

The cross-term is zero, because \(\sum_i(\mathbf{x}_i-\bar{\mathbf{x}})=\sum_i\mathbf{x}_i-n\bar{\mathbf{x}}=\mathbf{0}\). Removing it gives the identity. The first term on the right does not depend on \(\mathbf{c}\); the remaining term is nonnegative and equals zero only when \(\mathbf{c}=\bar{\mathbf{x}}\). Thus the mean is the unique minimizing center. For scalar data, the same argument holds with ordinary multiplication in place of the dot product.

Theorem: For a nonempty, finite set of equally weighted scalar observations or coordinate vectors, the arithmetic mean (coordinate-wise in the vector case) uniquely minimizes the sum of squared Euclidean distances. The identity above shows that choosing any other center increases the objective by exactly \(n\) times the squared distance between that center and the mean.

Worked Calculations

Worked Example: Finding a Center for Scalar Observations

A hypothetical risk analyst groups four profile scores, recorded on the same numerical scale: \(2,4,7,11\). Find the center that minimizes the sum of squared distances and calculate that minimum. All observations have equal weight; no probability distribution is assumed.

Method. For an equal-weight scalar cluster, calculate the arithmetic mean, then evaluate the squared deviations from it. This avoids treating a visually central observation as the minimizer without checking the objective.

The number of observations is \(n=4\), and their sum is \(2+4+7+11=24\). Therefore,

\bar{x}=\frac{24}{4}=6.

At \(c=6\), the deviations are \(2-6=-4\), \(4-6=-2\), \(7-6=1\), and \(11-6=5\). Their squares are \(16,4,1,25\), so the minimum objective is

$$ Q(6)=16+4+1+25=46. $$

Check against another center, \(c=5\): the squared deviations are \(9,1,4,36\), giving \(Q(5)=50\). This exceeds \(46\), as expected. The identity predicts an increase of \(n(5-6)^2=4\), and indeed \(50-46=4\).

Common mistake. Using \(5\), a nearby whole-number center, because it seems convenient. The calculated objectives show that it is not the minimizer. The mean is \(6\), and its squared-distance total is \(46\).

Worked Example: A Center With Two Coordinates

A hypothetical dataset contains four two-coordinate profiles: \((1,2),(3,2),(2,5),(4,3)\). Treat each coordinate as a numerical measurement on a common scale, and give each profile equal weight. Find the mean center and its total squared Euclidean distance.

Method. Calculate the mean of each coordinate. The squared Euclidean objective is the sum of the squared deviations in both coordinates, so each coordinate can be handled separately.

The first-coordinate sum is \(1+3+2+4=10\), giving mean \(10/4=2.5\). The second-coordinate sum is \(2+2+5+3=12\), giving mean \(12/4=3\). Thus \(\bar{\mathbf{x}}=(2.5,3)\).

For the first profile, the coordinate deviations are \((-1.5,-1)\), and its squared distance is \(1.5^2+1^2=2.25+1=3.25\). For the second, the deviations are \((0.5,-1)\), giving \(0.25+1=1.25\). For the third, they are \((-0.5,2)\), giving \(0.25+4=4.25\). For the fourth, they are \((1.5,0)\), giving \(2.25+0=2.25\). Therefore,

$$ Q(2.5,3)=3.25+1.25+4.25+2.25=11. $$

As a check, at the alternative center \((2,2)\), the four squared distances are \(1,1,9,5\), for a total of \(16\). The identity predicts an increase of \(4\{(2-2.5)^2+(2-3)^2\}=4(0.25+1)=5\); indeed, \(16-11=5\).

Common mistake. Averaging the two coordinates together into one number, or using a single center coordinate for both dimensions. Each coordinate has its own mean, and both are needed to specify the vector center.

Worked Example: Verifying the Center by the Increase in Squared Distance

A hypothetical cluster consists of the two-coordinate profiles \((2,2),(4,5),(6,2)\), each equally weighted. Find the center and its squared-distance total, then determine the total at the alternative center \((3,3)\).

Method. First use coordinate-wise means. Then apply the sum-of-squares identity to find how much the objective changes at the alternative center; also calculate its distances directly as a check.

The first-coordinate mean is \((2+4+6)/3=12/3=4\). The second-coordinate mean is \((2+5+2)/3=9/3=3\). Thus the minimizing center is \((4,3)\). The squared distances are \( (2-4)^2+(2-3)^2=4+1=5\), \( (4-4)^2+(5-3)^2=0+4=4\), and \( (6-4)^2+(2-3)^2=4+1=5\). The total is \(5+4+5=14\).

The alternative center differs from the mean by \((-1,0)\), whose squared length is \(1\). With \(n=3\), the identity gives an increase of \(3(1)=3\), so its objective is \(14+3=17\). Direct calculation confirms this: the squared distances from \((3,3)\) are \(1+1=2\), \(1+4=5\), and \(9+1=10\), totaling \(2+5+10=17\).

Common mistake. Reporting \(14\) for the alternative center as well as the mean center. The identity makes clear that any different center has a strictly larger objective for a nonempty cluster.

What the Result Does—and Does Not—Say

The arithmetic-mean result follows from the specific objective: a sum of squared Euclidean distances. It is not a claim that the mean is always the most representative summary under every possible measure of distance or loss. For example, replacing squared deviations with absolute deviations changes the optimization problem; the arithmetic-mean proof above does not establish a minimizer for that different objective.

The equal-weight condition also matters. The formula here gives each listed observation one contribution to the sum. If observations are assigned different weights, the objective and its minimizing center must be defined using those weights; the unweighted arithmetic mean need not minimize that altered objective.

For a nonempty cluster, the minimizer is unique even if some observations are repeated, because the identity’s added term is zero only at the mean. The center itself need not be one of the observed points. In the two-coordinate example, \((2.5,3)\) is a valid minimizing center even though no profile equals it.

A practical check is to compare a proposed center with the coordinate-wise mean, or to use the identity to calculate the objective difference. This helps catch common errors such as dividing a coordinate sum by the wrong count, omitting a coordinate’s squared deviations, or comparing unsquared distances with a squared-distance objective. The arithmetic-mean update is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Check Your Understanding

For each problem, assume a nonempty set of equally weighted observations and use squared Euclidean distance. Find the requested center and objective, showing the coordinate calculations before checking the answer.

A one-dimensional cluster has values \(3,5,8\). Find its minimizing center and the sum of squared deviations from that center. (Answer: center \(16/3\); minimum sum of squares \(38/3\).)
A two-coordinate cluster contains \((1,1),(3,1),(2,4)\). Find its mean center and squared-distance total. (Answer: center \((2,2)\); total \(8\).)
A cluster contains \((0,0),(2,0),(2,6),(4,2)\). Find its mean center and minimum total; then use the identity to find the total at \((3,2)\). (Answer: mean \((2,2)\), minimum \(32\), total at \((3,2)\) is \(36\).)
Scalar observations are \(2,6,10\). Find the minimizing center and objective, then calculate the objective at \(c=7\). (Answer: mean \(6\), minimum \(32\), objective at \(7\) is \(35\).)
A three-coordinate cluster contains \((1,2,0),(3,0,4),(2,4,2)\). Find the coordinate-wise mean and the total squared Euclidean distance from it. (Answer: center \((2,2,2)\); total \(18\).)

What You'll Learn