Why Comparing WCSS Requires Care
K-means divides observations into \(K\) nonempty clusters and measures their compactness using within-cluster sum of squares, or WCSS. It is tempting to choose the number of clusters by selecting the value of \(K\) with the smallest WCSS. But increasing \(K\) gives the method more freedom to fit the data, so WCSS is expected to decline even when the extra clusters do not provide a useful description.
A meaningful comparison holds the observations, features, feature scaling, and distance measure fixed. Changing any of these changes the objective being measured. In particular, WCSS is in squared distance units: its numerical size depends on the scale of the features as well as the number of observations.
The centroid in this definition is the arithmetic mean of the observations in its cluster, calculated feature by feature. For a fixed cluster assignment, that centroid minimizes the sum of squared Euclidean distances to a single center. The distinction between the global minimum \(W_K^*\) and the value returned by a particular k-means run will matter: the algorithm can converge to a local solution rather than the global one.
Worked WCSS Comparisons
Worked Example: Checking Every Two-Cluster Split
A hypothetical portfolio is represented by one numeric feature, a standardized measure of claim activity. Its four observations are \(0,2,8,10\). Calculate the optimal WCSS for \(K=1,2,3,4\), and verify which split is best for \(K=2\).
Method. For each proposed cluster, calculate its mean and sum the squared deviations from that mean. For \(K=1\), the mean is \((0+2+8+10)/4=5\), so \[ W_1^*=(0-5)^2+(2-5)^2+(8-5)^2+(10-5)^2=25+9+9+25=68. \]
For \(K=2\), the three partitions into contiguous groups in sorted order have the following WCSS values. The first split is \(\{0\}\) and \(\{2,8,10\}\). The second is \(\{0,2\}\) and \(\{8,10\}\). The third is \(\{0,2,8\}\) and \(\{10\}\).
For the first split, the mean of \(\{2,8,10\}\) is \(20/3\). Its WCSS is \[ (2-20/3)^2+(8-20/3)^2+(10-20/3)^2 =196/9+16/9+100/9=312/9=104/3. \] The singleton contributes zero, so the split's total is \(104/3\). For the second split, each pair has mean \(1\) or \(9\), giving \[ (0-1)^2+(2-1)^2+(8-9)^2+(10-9)^2=1+1+1+1=4. \] For the third split, the mean of \(\{0,2,8\}\) is \(10/3\), and its WCSS is \[ (0-10/3)^2+(2-10/3)^2+(8-10/3)^2 =100/9+16/9+196/9=312/9=104/3. \] Again the singleton contributes zero. Thus the best contiguous split has WCSS \(4\). In one dimension, an optimal squared-error partition can be taken to have contiguous groups in sorted order: if two observations are assigned to centers in reversed order, assigning each to the nearer center does not increase squared error. Therefore \(W_2^*=4\).
For \(K=3\), one optimal partition is \(\{0\},\{2\},\{8,10\}\). The first two clusters contribute zero, and the last contributes \((8-9)^2+(10-9)^2=2\), so \(W_3^*=2\). No three-cluster partition can do better: with four observations, one cluster must contain a pair, and the smallest possible pair contribution is \(2\), from \(\{8,10\}\) or \(\{0,2\}\). For \(K=4\), each observation can be its own cluster, giving \(W_4^*=0\).
The results are \(68,4,2,0\) for \(K=1,2,3,4\), respectively. A common calculation trap is to report \(40\) for the third \(K=2\) split. The mean of \(\{0,2,8\}\) is \(10/3\), and the squared deviations sum to \(312/9=104/3\), not \(40\). The first split also has WCSS \(104/3\), as shown above. The calculations confirm that the middle split is optimal.
Worked Example: Comparing the Same Features at Different K
A hypothetical study represents four policy groups by two fixed, standardized numeric features. The observed points are \((0,0),(0,2),(4,0),(4,2)\). Compare the optimal WCSS for \(K=1,2,3,4\).
For \(K=1\), the centroid is \((2,1)\). Each point has squared distance \(2^2+1^2=5\) from it, so \(W_1^*=4(5)=20\).
For \(K=2\), group the two points with first coordinate \(0\) together and the two with first coordinate \(4\) together. Each centroid is the midpoint of its vertical pair, and each point is distance \(1\) from its centroid. Thus the WCSS is \(1+1+1+1=4\). This is optimal: pairing points across the two columns gives a pair at least \(4\) units apart, whose within-pair WCSS is at least \(4^2/2=8\), already exceeding \(4\); the other cross-column pairing is no better. The column split therefore gives \(W_2^*=4\).
For \(K=3\), one cluster must contain at least two observations. A nearest pair in the square is separated by distance \(2\), and a two-point cluster has WCSS equal to half the squared distance between its points: \(2^2/2=2\). A partition that groups one vertical pair and leaves the other two points as singletons attains this value. Hence \(W_3^*=2\). With \(K=4\), all clusters can be singletons and \(W_4^*=0\). The sequence \(20,4,2,0\) shows the same decline, now with squared distances in the two-feature space.
The comparison is valid because the points and feature scales remain fixed. If one feature were rescaled, the distances and all these WCSS values could change, so values calculated before and after rescaling would not be a like-for-like comparison.
Why Optimal WCSS Cannot Increase
The monotonicity result is a consequence of the feasible partitions, not a special property of a particular data set. The argument applies to squared Euclidean distance and globally optimal WCSS, with the same observations and feature representation for every \(K\).
Because \(K<n\), at least one of its clusters contains two or more observations; otherwise there would be fewer than \(n\) observations assigned to \(K\) nonempty singleton clusters.
Keep all other clusters unchanged. The result is a feasible partition into \(K+1\) nonempty clusters. Such a split exists because the selected cluster has at least two observations.
Assign both new groups their old cluster's centroid temporarily. This reproduces the old cluster's contribution to WCSS exactly. Replacing each temporary center with the mean of its own group can only reduce or preserve its sum of squared distances.
The split partition has WCSS no greater than \(W_K^*\). Since \(W_{K+1}^*\) is the minimum over all feasible \(K+1\)-cluster partitions, it is no greater than the WCSS of this particular split partition. Therefore \(W_{K+1}^*\leq W_K^*\).
The claim that the mean minimizes squared distance has a short algebraic justification. For a cluster of \(m\) vectors \(x_1,\ldots,x_m\), let \(\bar{x}\) be their mean and let \(c\) be any proposed center. Expand each vector as \(x_i-c=(x_i-\bar{x})+(\bar{x}-c)\). The cross terms sum to zero because \(\sum_i(x_i-\bar{x})=0\). Thus
This proves the center-minimization fact used in the split argument. It also explains why allowing each new cluster to have its own mean cannot raise the objective.
Monotonicity Is Not a Cluster-Selection Rule
The theorem describes a mechanical property of the objective. At \(K=n\), every observation can be its own centroid and the WCSS is zero. That zero is always attainable, but it does not usually make \(n\) a useful choice: a collection of singleton clusters does not summarize the data.
Even before \(K=n\), a lower WCSS only says that the observations are closer, in squared-distance terms, to their assigned centroids. It does not establish that the added clusters correspond to meaningful groups, are stable under changes in initialization, or provide a useful summary for the application. WCSS also tends to favor compact, roughly spherical groups because of the squared Euclidean objective. The data and the purpose of the analysis still matter.
Worked Example: Reading a Declining WCSS Sequence
For a fixed hypothetical data set and fixed feature scaling, suppose a clustering analysis reports best-known WCSS values of \(120,50,28,19,14\) for \(K=1,2,3,4,5\), respectively. Calculate the absolute and percentage reduction at each step, then assess what the sequence establishes.
From \(K=1\) to \(K=2\), the reduction is \(120-50=70\), or \(70/120=0.5833\ldots\), which is \(58.3\%\) to one decimal place. From \(K=2\) to \(K=3\), it is \(50-28=22\), or \(22/50=44.0\%\). From \(K=3\) to \(K=4\), it is \(28-19=9\), or \(9/28=32.1\%\). From \(K=4\) to \(K=5\), it is \(19-14=5\), or \(5/19=26.3\%\).
The values decline at every step, as required for global optima. The absolute decreases become smaller: \(70,22,9,5\). This may make a plot worth examining for a bend, but the arithmetic alone does not prove that any particular \(K\) is useful. There is no universal percentage-reduction threshold that identifies a correct number of clusters. The analyst should interpret the candidate solutions in context and use other relevant diagnostics, such as silhouette widths or stability across initializations, rather than treating the lowest WCSS as a selection rule.
Method and trap avoided. Compare successive reductions on the same data and scale, and distinguish a descriptive pattern from a decision rule. The common mistake is to choose \(K=5\) simply because \(14\) is the smallest listed value; that conclusion ignores the fact that the objective can be driven to zero by continuing to add clusters.
Global Optima and Algorithm Outputs
In practice, k-means uses an iterative algorithm and may converge to a local solution. Its output for a particular \(K\) is therefore not necessarily \(W_K^*\). Separate runs for different values of \(K\) may also use different initial centers, stopping behavior, or numbers of restarts. Consequently, the WCSS values from those runs need not be monotone, even though the global optima must be.
Worked Example: Interpreting an Apparent Increase
On the same fixed data and feature scale, suppose one run reports WCSS \(7.2\) for \(K=2\), while a separate run reports WCSS \(8.1\) for \(K=3\). Does this contradict the monotonicity theorem, and what should be checked?
Method. First identify what the reported quantities represent. If they were global optima, the theorem would require \(W_3^*\leq W_2^*\); \(8.1>7.2\) would be impossible. But the figures are outputs of particular runs, not established global minima. The \(K=3\) run has returned a feasible clustering with WCSS \(8.1\), while the \(K=2\) run has found one with WCSS \(7.2\). This does not show that the best three-cluster partition has WCSS above \(7.2\); it shows only that this run did not produce a sufficiently good candidate to demonstrate an improvement.
A practical response is to check the implementation and confirm that the same observations, features, scaling, and WCSS definition were used. Then use multiple initializations for each \(K\), compare the best attained WCSS values, and inspect convergence. The \(K=3\) candidate can also be refined by splitting a cluster from the \(K=2\) solution and optimizing the resulting centers. More restarts can improve the evidence, but they do not by themselves prove global optimality.
Common mistake. It is incorrect to conclude that the theorem is false because one observed algorithm output increased from \(7.2\) to \(8.1\). The correction is to distinguish the mathematical global minimum from the value found by a particular run.
Practical Interpretation
When comparing WCSS across values of \(K\), use one fixed data representation and record whether each value is a global optimum (usually unknown) or the best result found across the chosen initializations. A useful WCSS curve can help describe how compactness changes as more clusters are allowed. It cannot, by itself, decide how many clusters the problem needs.
- Compare values calculated from the same observations, features, scaling, and squared-distance objective.
- Expect globally optimal WCSS to be nonincreasing; an increase in separate run outputs calls for checking the optimization process.
- Do not infer that a cluster structure is meaningful merely because WCSS is low.
- Use the purpose of the analysis and complementary diagnostics when judging candidate values of \(K\).
Check Your Understanding
For each question, keep the data and feature scale fixed unless the problem says otherwise.
- For observations \(0,4,10\) on a line, calculate the WCSS for \(K=1\) and the optimal WCSS for \(K=2\). (Answer: \(W_1^*=52\); \(W_2^*=8\), from \(\{0,4\}\) and \(\{10\}\).)
- A fixed data set has reported WCSS values \(84,31,12,4\) for \(K=1,2,3,4\). What is the absolute and percentage reduction from \(K=1\) to \(K=2\)? (Answer: \(53\), or \(53/84=63.1\%\) to one decimal place.)
- Four points form a square with vertices \((0,0),(0,2),(4,0),(4,2)\). A three-cluster partition leaves two points as singletons and groups a pair of adjacent vertices. Find its WCSS and explain why this is optimal for \(K=3\). (Answer: WCSS \(=2\); a pair of adjacent vertices has squared separation \(4\), and its two-point WCSS is \(4/2=2\), the smallest possible contribution from the one pair that must remain.)
- Two separate runs on the same data report WCSS \(7.2\) for \(K=2\) and \(8.1\) for \(K=3\). Does this violate the theorem, and name one appropriate next step. (Answer: No; these are run outputs, not necessarily global optima. Use additional initializations and check that the data representation and objective are consistent.)
- A WCSS curve decreases from \(200\) to \(90\), then \(55\), then \(39\) as \(K\) increases from \(1\) to \(4\). What are the three absolute reductions, and do they alone identify a useful \(K\)? (Answer: \(110,35,16\); no. They show diminishing absolute reductions here, but do not by themselves establish which clustering is useful.)