Common K-Means Exam Traps

What K-Means Does—and What Its Output Does Not Prove

K-means assigns each observation to one of $K$ clusters and represents each cluster by its centroid, the coordinate-wise mean of its assigned observations. The method is useful for summarizing numerical profiles, but its output is easy to overinterpret. A small objective value does not establish that the clusters are meaningful, and a particular run of the algorithm need not find the best possible partition.

The following review focuses on common calculation and interpretation traps. It assumes observations have numerical features and uses ordinary squared Euclidean distance. Before fitting a model, the analyst must decide whether that distance and the chosen feature scales make sense for the question.

Definition: For observations $z_1,\ldots,z_n$, each with $p$ numerical features, a partition into $K$ nonempty clusters $C_1,\ldots,C_K$ has centroid $\bar{z}_k$, the vector of feature means among observations in $C_k$. Its within-cluster sum of squares (WCSS) is $$ \operatorname{WCSS}=\sum_{k=1}^{K}\sum_{i:z_i\in C_k}\|z_i-\bar{z}_k\|^2. $$ K-means seeks a partition with small WCSS. The objective is defined relative to the supplied features, their scales, and squared Euclidean distance; it is not a scale-free measure of cluster quality.

Trap 1: Miscalculating or Misreading WCSS

For each observation, WCSS adds the squared distance to its own cluster centroid. It does not add the distance to the nearest centroid if that centroid belongs to a different cluster, and it does not take a square root at the end. In more than one dimension, the squared Euclidean distance is the sum of the squared coordinate differences.

Worked Example: Comparing Two Partitions Carefully

Four hypothetical risk profiles have two numerical features, recorded here as coordinates: $A=(0,0)$, $B=(0,2)$, $C=(10,0)$, and $D=(10,2)$. For $K=2$, compare the partition by first coordinate, $\{A,B\},\{C,D\}$, with the partition by second coordinate, $\{A,C\},\{B,D\}$. Use squared Euclidean distance.

Method. Find each cluster centroid, square each coordinate deviation, and sum the deviations within the clusters.

For the first partition, the centroid of $\{A,B\}$ is $(0,1)$. Its contributions are $0^2+(-1)^2=1$ for $A$ and $0^2+1^2=1$ for $B$, giving $2$. The centroid of $\{C,D\}$ is $(10,1)$; the contributions are also $1$ and $1$, giving $2$. Thus the total WCSS is $2+2=4$.

For the second partition, the centroid of $\{A,C\}$ is $(5,0)$. The squared distances are $(0-5)^2+(0-0)^2=25$ and $(10-5)^2+(0-0)^2=25$, giving $50$. The centroid of $\{B,D\}$ is $(5,2)$, and its squared distances are $25$ and $25$, also giving $50$. The total WCSS is $50+50=100$.

Check and trap avoided. The first partition has the smaller WCSS, $4$ versus $100$. In each of the second partition’s clusters, the first coordinates $0$ and $10$ are centered at $5$, so that cluster contributes $(0-5)^2+(10-5)^2=25+25=50$. Two such clusters contribute $100$, not $200$. Do not multiply the two squared deviations by an extra factor of two.

Trap 2: Treating Feature Scales as Irrelevant

Squared Euclidean distance gives more influence to features that vary across larger numerical ranges. Changing a feature’s units or multiplying it by a weight can therefore change both the WCSS and the assignments. Standardizing features is one possible response, but it is not automatic or always appropriate: the choice should reflect the meaning of the features and the intended distance between profiles.

Worked Example: A Change of Units Changes the Preferred Partition

Use the four profiles from the preceding example. Suppose the second feature is now recorded in units ten times as large, so each point $(x,y)$ becomes $(x,10y)$. Recalculate WCSS for the same two candidate partitions.

First-coordinate partition. Its clusters are $\{(0,0),(0,20)\}$ and $\{(10,0),(10,20)\}$. Each centroid has second coordinate $10$. In each cluster, the two squared deviations in that coordinate are $100$ and $100$; the first-coordinate deviations are zero. Each cluster contributes $200$, so WCSS is $400$.

Second-coordinate partition. Its clusters are $\{(0,0),(10,0)\}$ and $\{(0,20),(10,20)\}$. Each centroid has first coordinate $5$. Each cluster contributes $25+25=50$, so WCSS is $100$.

In the original units, the first-coordinate partition had WCSS $4$ and the second-coordinate partition had WCSS $100$. After the change, their WCSS values are $400$ and $100$, respectively. The lower-WCSS candidate has changed because the geometry changed. The trap avoided is to call this a mere change in labels or units that cannot affect the result: multiplying one feature by ten multiplies its squared deviations by one hundred.

Trap 3: Assuming an Initialization Finds the Global Minimum

The usual k-means algorithm alternates between assigning observations to their nearest current centroid and recalculating centroids as means. These steps do not increase WCSS, but they can settle at a solution that is not globally optimal. Different initial centroids can lead to different final partitions, so a single run is not proof that the best partition has been found. Multiple starts can reduce this risk, though they do not guarantee a global optimum.

Worked Example: A Run Can Settle at a Higher-WCSS Partition

Consider the one-dimensional observations $0,2,3,10$, with $K=2$. Run the assignment-and-update procedure starting with centroids $0$ and $2$, then compare its result with another valid partition.

First assignment. With centroids $0$ and $2$, observation $0$ is assigned to the first cluster; $2,3,10$ are assigned to the second. The updated centroids are $0$ and $(2+3+10)/3=5$.

Second assignment and update. With centroids $0$ and $5$, observation $2$ is nearer $0$, while $3$ and $10$ are nearer $5$. The clusters are now $\{0,2\}$ and $\{3,10\}$, with updated centroids $1$ and $(3+10)/2=6.5$. Using these centroids, $0$ and $2$ are nearer $1$, and $3$ and $10$ are nearer $6.5$. The assignments do not change.

WCSS of this result. The first cluster contributes $(0-1)^2+(2-1)^2=1+1=2$. The second contributes $(3-6.5)^2+(10-6.5)^2=12.25+12.25=24.5$. Total WCSS is $2+24.5=26.5$.

Comparison and check. The alternative partition $\{0,2,3\},\{10\}$ has centroids $5/3$ and $10$. Its WCSS is $(0-5/3)^2+(2-5/3)^2+(3-5/3)^2+0=25/9+1/9+16/9=42/9=14/3$, approximately $4.667$. This is less than $26.5$. The starting run therefore stopped at a higher-WCSS solution. The trap avoided is treating convergence of the algorithm as proof of global optimality.

Trap 4: Concluding That More Clusters Must Be Better

For the optimized WCSS, adding a cluster cannot make the minimum larger: an existing cluster can be split, and the additional cluster can initially be placed so that the old assignments and their fit are preserved. Re-optimizing can only maintain or reduce the objective. This explains why WCSS alone cannot choose $K$: it rewards adding clusters, even when the extra complexity is not useful.

Theorem: If $W_K$ is the minimum WCSS over all partitions into $K$ nonempty clusters, then $W_{K+1}\leq W_K$ whenever a feasible split into $K+1$ nonempty clusters exists. This statement concerns the optimized objective. A particular algorithm run, stopped at a local solution, need not produce a WCSS that is lower than every run at a smaller $K$.

To see why the inequality holds, take a partition attaining $W_K$. Split one of its clusters into two nonempty parts and keep all other assignments. If each part is assigned its own mean, its combined squared deviations cannot exceed the original cluster’s contribution: using the original centroid for both parts would reproduce the old contribution, while each part’s mean minimizes its own sum of squared deviations. Thus this $K+1$ partition has WCSS no larger than $W_K$. Minimizing over all $K+1$-cluster partitions cannot increase it.

A lower WCSS at larger $K$ is therefore expected, not by itself evidence of a better scientific or actuarial model. Choosing $K$ requires a separate purpose and diagnostic judgment; for example, silhouette widths and stability across initializations are relevant tools. Even these assess aspects of a clustering rather than prove that the population contains natural, sharply separated groups.

Trap 5: Ignoring Outliers and Overinterpreting Assignments

Because squared deviations grow quadratically, an extreme observation can have a large effect on centroids and WCSS. It may pull a centroid toward itself or, in some settings, receive a cluster of its own. Check unusual observations and their influence before interpreting the resulting profiles. Do not remove a value solely because it is inconvenient: first determine whether it is a recording error or a genuine observation relevant to the analysis.

A cluster assignment is the result of applying a chosen algorithm to chosen features, scales, and settings. It is not, by itself, evidence that an individual belongs to a naturally distinct population. K-means assigns every observation to a cluster even when groups overlap or the data have no meaningful group structure. Use assignments as a descriptive summary, and examine the context, cluster sizes, within-cluster spread, and stability before making decisions.

Check the objective.
Confirm that the stated distance is squared Euclidean distance and calculate each observation’s contribution to its assigned centroid.

Check the geometry.
Ask whether the feature units and weights represent the intended importance of differences between profiles.

Check the fitting process.
Distinguish convergence from global optimality; compare results from multiple initializations when appropriate.

Check the interpretation.
Remember that optimized WCSS tends to decrease with $K$, and that an assignment does not establish a naturally distinct group.

Exam Traps to Keep in View

Omitting the square. WCSS sums squared distances, not ordinary distances.
Counting a contribution twice. For each cluster, add the deviations of its observations once; do not multiply the total by the number of coordinates or observations again.
Ignoring a scale change. Multiplying a feature by $c$ multiplies its squared deviations by $c^2$.
Equating convergence with optimality. A run can stop at a local solution that has higher WCSS than another feasible partition.
Using raw WCSS to choose $K$. The optimized value cannot increase as $K$ increases, so its decline alone does not identify a useful number of clusters.
Treating clusters as discovered facts. Assignments depend on the modeling choices and do not prove that natural groups exist.

Key takeaway: Interpret k-means WCSS conditionally: it summarizes fit under a particular feature representation, distance, partition, and value of $K$. Check the arithmetic and assumptions, and do not confuse a converged algorithm or a set of labels with proof of a unique, meaningful structure.

Check Your Understanding

For each problem, track the units and assignment rule before interpreting the result.

Four profiles have coordinates $(1,0),(1,4),(9,0),(9,4)$. For $K=2$, calculate WCSS for the partition by first coordinate and for the partition by second coordinate. Which has the smaller WCSS? (Answer: First-coordinate partition $16$; second-coordinate partition $64$; the first-coordinate partition is smaller.)
For the same four profiles, multiply the second coordinate by $3$. Recalculate the two WCSS values and identify the smaller one. (Answer: First-coordinate partition $144$; second-coordinate partition $64$; the second-coordinate partition is smaller.)
For the one-dimensional observations $2,4,6,8,40$ and $K=2$, compare the partition $\{2,4,6,8\},\{40\}$ with $\{2,4,6\},\{8,40\}$. Show the centroids and WCSS, and state what the comparison says about isolating the extreme value. (Answer: The first partition has centroids $5,40$ and WCSS $20$; the second has centroids $4,24$ and WCSS $520$. The first has smaller WCSS for these two candidates.)
An analyst reports optimized WCSS values $72$, $49$, and $37$ for $K=2,3,4$, respectively. What can be concluded from their ordering, and what cannot be concluded from it alone? (Answer: WCSS decreases as $K$ increases, as expected; this alone does not show that $K=4$ is the most useful choice or that natural groups exist.)
Two k-means runs return the same two sets of observations but call them cluster 1 and cluster 2 in opposite orders. Are these different partitions? What should be compared when assessing whether the runs agree? (Answer: They are the same partition; compare which observations are grouped together, not the arbitrary cluster labels.)

What You'll Learn