Why K-Means Favors Compact Spherical Clusters

Why Cluster Shape Matters

K-means assigns observations to the nearest of $K$ centroids and updates each centroid to the mean of its assigned observations. Because the method uses squared Euclidean distance, it favors partitions that keep observations close to their assigned means. This creates a useful tendency toward compact, roughly spherical clusters, but it does not guarantee that the clusters in the data actually have those shapes.

This geometric tendency matters when a data set contains long, narrow, curved, or ring-shaped groups. A partition can have a smaller k-means objective while cutting across a group that appears meaningful for the analysis. The objective measures squared distance to centroids; it does not know the intended group labels or the shape a practitioner expects.

Definition: For $n$ observations $x_1,\ldots,x_n$ in a numerical feature space, a partition into $K$ nonempty clusters $C_1,\ldots,C_K$ has centroid $\bar{x}_k$, the componentwise arithmetic mean of the observations in $C_k$. Its within-cluster sum of squares (WCSS) is $$ \mathrm{WCSS}=\sum_{k=1}^{K}\sum_{i\in C_k}\|x_i-\bar{x}_k\|^2. $$ Here, $\|x_i-\bar{x}_k\|^2$ is squared Euclidean distance. K-means seeks a partition and centroids with small WCSS. No probability distribution assumption is required to define or calculate this objective.

The centroid is the point that minimizes the sum of squared Euclidean distances from the observations in a cluster. Consequently, once a partition is fixed, using its centroids gives the smallest WCSS available for that partition. In the assignment step, each observation is placed in a nearest-centroid region. Those regions are separated by straight boundaries, not curved ones.

Why the Mean Minimizes Squared Distance

The connection between the centroid and the squared-distance objective is the source of k-means’ shape preference. For a cluster with $m$ observations $x_1,\ldots,x_m$, let $\bar{x}$ be their mean. For any candidate point $a$, expand each squared distance around $\bar{x}$:

\sum_{j=1}^{m}\|x_j-a\|^2 =\sum_{j=1}^{m}\|(x_j-\bar{x})+(\bar{x}-a)\|^2.

On expansion, the cross-product terms sum to zero because $\sum_{j=1}^{m}(x_j-\bar{x})=0$. The result is:

\sum_{j=1}^{m}\|x_j-a\|^2 =\sum_{j=1}^{m}\|x_j-\bar{x}\|^2+m\|\bar{x}-a\|^2.

The second term is nonnegative and is zero only when $a=\bar{x}$. Thus the mean minimizes the sum of squared distances. This result also shows that the objective penalizes deviations in every feature direction: a large displacement along even one direction contributes its square.

For an elongated cluster, observations may be spread far along the cluster’s long axis. Splitting the group along that axis can substantially reduce WCSS, even if a subject-matter interpretation treats the elongated points as one group. A compact, roughly spherical group has less directional imbalance, so centroid distance is often a more natural summary of its internal spread. These are tendencies of the objective, not guarantees about the data.

Worked Example: Comparing Two Partitions of Four Corner Locations

Eight observations lie at the four locations $(-1,-2)$, $(-1,2)$, $(1,-2)$, and $(1,2)$, with each location observed twice. For $K=2$, compare the partition by the sign of $x$ with the partition by the sign of $y$. Use the appropriate centroid for each cluster.

Method. Calculate each cluster mean, then sum the squared distances of all observations to their assigned means. Repeated observations count repeatedly in WCSS.

Partition by $x$. The left cluster contains two copies each of $(-1,-2)$ and $(-1,2)$. Its centroid is $(-1,0)$. Each of its four observations has squared distance $0^2+2^2=4$, so its WCSS is $4(4)=16$. The right cluster has centroid $(1,0)$ and also has WCSS $16$. Therefore, the total is $16+16=32$.

Partition by $y$. The lower cluster contains two copies each of $(-1,-2)$ and $(1,-2)$. Its centroid is $(0,-2)$. Each of its four observations has squared distance $1^2+0^2=1$, so its WCSS is $4(1)=4$. The upper cluster has centroid $(0,2)$ and also has WCSS $4$. Therefore, the total is $4+4=8$.

The split by $y$ has lower WCSS: $8$ rather than $32$. The calculation is consistent with the geometry: within each $y$-group, observations vary by only one unit in $x$, while within each $x$-group they vary by two units in $y$. Common mistake: counting the four corner locations only once even though the problem specifies two observations at each location. That would undercount both WCSS values by a factor of two.

How Elongated Groups Can Be Split Poorly

The next calculation illustrates a more consequential limitation. Suppose the data visibly form two long, horizontal rows. A partition by row might seem natural, but the rows have substantial spread along their long direction. A partition that cuts across the rows can reduce the squared-distance objective instead.

Worked Example: A Lower-WCSS Split Cuts Across Two Long Rows

Consider eight observations arranged in two rows: $(-3,1),(-1,1),(1,1),(3,1)$ and $(-3,-1),(-1,-1),(1,-1),(3,-1)$. Compare the partition into the two rows with a partition into a left half ($x<0$) and a right half ($x>0$).

Method. Compute the centroid and WCSS for each of the two candidate partitions. The goal is to compare the objective values, not to assert which partition is substantively correct.

Partition by row. The upper-row centroid is $(0,1)$, since the mean $x$-coordinate is $(-3-1+1+3)/4=0$ and every $y$-coordinate is 1. The four squared distances to this centroid are $9,1,1,9$, giving WCSS $9+1+1+9=20$. The lower-row centroid is $(0,-1)$, and its WCSS is also $20$. The total is $40$.

Partition into left and right halves. The left half consists of two observations at $x=-3$ and two at $x=-1$, with $y=1$ and $y=-1$ at each $x$. Its centroid is $(-2,0)$. Each of its four observations has squared distance $1^2+1^2=2$, giving WCSS $4(2)=8$. The right half has centroid $(2,0)$ and WCSS $8$. The total is $16$.

The left-right split has WCSS $16$, compared with $40$ for the row split. It cuts across both horizontal rows, but it is favored by this objective because it reduces spread along the long direction. Common mistake: deciding that the visually apparent rows must be the k-means result. K-means selects according to squared distance to centroids, not according to visual continuity or an external definition of group membership.

Curved Groups and Straight-Line Boundaries

K-means has a second geometric limitation: its assignment regions are determined by distances to centroids. For two centroids, the set of points equally distant from them is a straight line in two dimensions (a hyperplane in higher dimensions). This is not a boundary that can trace a curved separation such as a crescent, ring, or winding band.

A ring-shaped group illustrates the issue. Its center of mass can lie near the middle of the ring, where there may be no observations from that group. A centroid represents an average location, not the path or shape occupied by the points. If a second group lies inside the ring, nearest-centroid assignment does not naturally encode “inside” versus “around the outside”: distances to two centroids partition space by a straight boundary.

Worked Example: Centroids Do Not Identify Inner and Outer Rings

Suppose the intended groups are four inner points $(1,0),(-1,0),(0,1),(0,-1)$ and four outer points $(3,0),(-3,0),(0,3),(0,-3)$. Evaluate the WCSS for this inner-versus-outer partition and determine whether the two centroids alone distinguish the groups.

Inner group. The coordinate sums are $1-1+0+0=0$ and $0+0+1-1=0$, so its centroid is $(0,0)$. Each of its four points is one unit from the centroid, giving WCSS $4(1^2)=4$.

Outer group. Its coordinate sums are $3-3+0+0=0$ and $0+0+3-3=0$, so its centroid is also $(0,0)$. Each point is three units from the centroid, giving WCSS $4(3^2)=36$. The total WCSS of the proposed partition is $4+36=40$.

Both centroids coincide at $(0,0)$, so nearest-centroid assignment cannot distinguish the inner points from the outer points: every observation has the same distance to both centroids. The proposed radial grouping is therefore not represented by two distinct nearest-centroid regions. This calculation does not establish that the inner-outer partition minimizes WCSS; it shows why centroid locations and the k-means assignment rule do not capture that radial distinction. Common mistake: treating a low or calculable WCSS as proof that the clusters correspond to the intended real-world groups.

What “Spherical” Does and Does Not Mean

Calling k-means suited to compact, roughly spherical clusters is a practical description of its geometry, not a formal assumption that every cluster must be an exact circle or sphere. The objective uses the same squared Euclidean distance in every direction, and the assignment rule favors regions around centroids. Consequently, it tends to work best when groups are reasonably compact and their separation is compatible with centroid-based boundaries.

The label “spherical” should not be read as a claim that k-means estimates a spherical probability distribution. Rather, it describes the shape of groups that the distance-and-centroid mechanism can represent relatively naturally. Strong elongation, curvature, a hollow center, or a complicated boundary can make a low-WCSS partition disagree with a meaningful grouping. A clustering result should therefore be assessed against the analysis objective and the geometry of the observations, not interpreted from the WCSS alone.

For actuarial analysis, consider a hypothetical portfolio represented by numerical features such as claim frequency and average claim amount. K-means could produce compact profiles in that feature space, but a portfolio segment defined by a curved business rule or a long progression of risks may not align with its centroid-based regions. The clusters are a result of the specified features, scaling, distance measure, and objective; they are not automatically risk categories with substantive meaning.

Common Exam Traps

Using ordinary distance instead of squared distance. WCSS adds squared Euclidean distances. A point two units from its centroid contributes $2^2=4$, not 2.
Ignoring repeated observations. Each observation contributes once to the sum, so duplicated locations contribute repeatedly.
Assuming the centroid is an observed point. It is the componentwise mean and can fall in an unoccupied part of the feature space.
Equating low WCSS with meaningful clusters. WCSS assesses compactness around centroids; it does not validate an interpretation or recover curved shapes by itself.
Assuming all cluster shapes are equally natural for k-means. Centroid distance and straight assignment boundaries favor some geometries over others.

Key takeaway: K-means minimizes squared distances to cluster means. Because that objective penalizes spread in every direction and assigns observations through nearest-centroid regions, it often favors compact, roughly spherical groups. Elongated or curved structure can be split in a way that lowers WCSS but conflicts with the intended grouping.

Check Your Understanding

For each problem, calculate using squared Euclidean distances and the stated partition.

Four observations are $(-2,0),(-1,0),(1,0),(2,0)$. For $K=1$, find the centroid and WCSS. (Answer: centroid $(0,0)$, WCSS $10$.)
Eight observations consist of two copies of each point $(-1,-2),(-1,2),(1,-2),(1,2)$. For $K=2$, compare the WCSS for the split by $x$ and the split by $y$. (Answer: split by $x$, WCSS $32$; split by $y$, WCSS $8$.)
For the points $(-3,1),(-1,1),(1,1),(3,1),(-3,-1),(-1,-1),(1,-1),(3,-1)$, compare the two-row partition with the left-right partition. (Answer: two-row WCSS $40$; left-right WCSS $16$.)
A cluster contains $(0,0),(2,0),(4,0)$. Find its centroid and WCSS. If the first two points form one cluster and the last point a singleton, find the two-cluster WCSS. (Answer: one cluster has centroid $(2,0)$ and WCSS $8$; the stated two-cluster partition has WCSS $2$.)
Four inner points are $(1,0),(-1,0),(0,1),(0,-1)$; four outer points are $(3,0),(-3,0),(0,3),(0,-3)$. For the inner-versus-outer partition, find both centroids and its total WCSS. Can nearest-centroid assignment distinguish the two groups? (Answer: both centroids are $(0,0)$, total WCSS $40$; no, the distances to the two centroids are identical for every point.)

What You'll Learn