Why the Numbers Entering K-Means Matter
K-means does not understand what a feature represents. It receives coordinates, measures Euclidean distances between them, and places each observation in a cluster to reduce within-cluster sums of squares. A coordinate such as annual claim count has a meaningful numerical scale. A category such as payment channel—paper, online, or telephone—does not automatically have one. If the categories are replaced by arbitrary integers, k-means treats those integers as real measurements, including their order and spacing.
This is not a minor labeling issue. Changing the codes can change the distances, the centroids, and the resulting clusters, even though no underlying observation has changed. The key question is not whether software accepts a column of numbers. It is whether Euclidean distance between those numbers represents a meaningful difference for the clustering task.
For example, suppose a nominal feature has categories A, B, and C. Codes A = 1, B = 2, C = 3 imply that A is closer to B than to C, and that the A-to-C gap is twice the A-to-B gap. Another equally arbitrary coding, A = 1, B = 3, C = 100, implies very different distances. Unless the categories genuinely have an ordered quantitative scale with meaningful spacing, neither implication comes from the data.
How Integer Codes Can Determine the Clusters
Worked Example: Changing Category Spacing Changes the Preferred Split
A hypothetical administrator records six service records by payment channel. There are two records in each nominal category: A, B, and C. Compare the optimal two-cluster solution when the categories are coded as A = 0, B = 1, C = 2 and when they are coded as A = 0, B = 1, C = 10. Treat each code as the only feature, as ordinary k-means would.
Method. In one dimension, a minimum-WCSS k-means solution can be taken to have contiguous groups in sorted order. With these repeated codes and \(K=2\), compare the two cuts between distinct category values.
First coding: 0, 1, 2. The sorted observations are \(0,0,1,1,2,2\). If the cut is between A and B, the clusters are \(\{0,0\}\) and \(\{1,1,2,2\}\). Their centroids are 0 and \(1.5\). The WCSS is $$ 0+2(1-1.5)^2+2(2-1.5)^2 =0+2(0.25)+2(0.25)=1. $$ If the cut is between B and C, the clusters are \(\{0,0,1,1\}\) and \(\{2,2\}\), with centroids \(0.5\) and 2. The WCSS is $$ 2(0-0.5)^2+2(1-0.5)^2+0 =2(0.25)+2(0.25)=1. $$ The two cuts tie. Thus this coding does not favor one of these category groupings over the other.
Second coding: 0, 1, 10. The sorted observations are \(0,0,1,1,10,10\). A cut between A and B gives clusters \(\{0,0\}\) and \(\{1,1,10,10\}\). The second centroid is \((1+1+10+10)/4=5.5\), so the WCSS is $$ 2(1-5.5)^2+2(10-5.5)^2 =2(20.25)+2(20.25)=81. $$ A cut between B and C gives \(\{0,0,1,1\}\) and \(\{10,10\}\). The first centroid is \(0.5\), so the WCSS is $$ 2(0-0.5)^2+2(1-0.5)^2+0 =2(0.25)+2(0.25)=1. $$ The second coding strongly favors grouping A and B together and isolating C. That preference was introduced by the large numerical gap between codes 1 and 10; it was not established by the nominal payment-channel labels.
Check and common mistake avoided. The arithmetic confirms that the best WCSS is 1 under each coding, but the tied versus strongly preferred clusterings differ. It would be a mistake to interpret the code-10 result as evidence that category C is substantively far from A and B. The gap is a property of the arbitrary labels, not of the records.
Why the Centroid Is a Warning Sign
The arithmetic mean is central to k-means. If category codes are used, a centroid may be a number that does not correspond to any category. For codes A = 0 and B = 1, a cluster containing one A and one B has centroid 0.5. There is no category 0.5. More importantly, the centroid makes sense as a location on a number line only if the codes and their spacing represent a meaningful scale.
This feature of k-means follows directly from its objective, rather than being a separate convention. For a cluster with observations \(x_1,\ldots,x_n\), write \(\bar{x}\) for their coordinate-wise mean. For any proposed numeric center \(c\), expand each squared distance around the mean:
To see why, write \(x_i-c=(x_i-\bar{x})+(\bar{x}-c)\), expand the squared norm, and sum. The cross-term is zero because \(\sum_i(x_i-\bar{x})=0\). Since \(n\|c-\bar{x}\|^2\) is nonnegative, the sum of squared distances is minimized at \(c=\bar{x}\). Thus the k-means centroid is the arithmetic mean. Applying this calculation to arbitrary category codes does not validate the codes; it simply shows exactly how k-means uses them.
One-Hot Encoding for Nominal Categories
A common way to represent a nominal feature numerically is one-hot encoding. With three categories, represent A by \((1,0,0)\), B by \((0,1,0)\), and C by \((0,0,1)\). Each category gets its own coordinate rather than an ordered integer code. The squared Euclidean distance between any two different category vectors is 2, so every category mismatch has distance \(\sqrt{2}\). The representation therefore avoids inventing a nearer and farther pair of categories.
Worked Example: A One-Hot Centroid and Its WCSS
A hypothetical book-service study records five users' preferred contact channels: two choose North Desk, two choose South Desk, and one chooses West Desk. These are nominal labels. One-hot encode the channel, find the cluster centroid if all five records are placed in one cluster, and calculate its WCSS.
Method. Use the coordinate order North, South, West. The vectors are \((1,0,0)\) twice, \((0,1,0)\) twice, and \((0,0,1)\) once. The centroid is the coordinate-wise mean: $$ \bar{x}=\left(\frac{2}{5},\frac{2}{5},\frac{1}{5}\right)=(0.4,0.4,0.2). $$
For either North record, the squared distance to the centroid is $$ (1-0.4)^2+(0-0.4)^2+(0-0.2)^2 =0.36+0.16+0.04=0.56. $$ The two North records contribute \(2(0.56)=1.12\). Each South record also contributes \(0.56\), so the two South records contribute \(1.12\). The West record contributes $$ (0-0.4)^2+(0-0.4)^2+(1-0.2)^2 =0.16+0.16+0.64=0.96. $$ The total WCSS is \(1.12+1.12+0.96=3.20\).
Check and common mistake avoided. The centroid coordinates sum to 1, as expected from the category proportions, and the contributions sum to 3.20. The centroid \((0.4,0.4,0.2)\) describes proportions in this cluster; it is not itself one of the observed contact channels. Do not mistake it for a new channel or interpret fractional coordinates as category labels.
One-hot encoding gives equal distance to every pair of distinct categories for this feature considered by itself. It does not automatically make every k-means analysis appropriate. A categorical feature represented by many coordinates can have substantial influence on distances, and in a mixed data set its contribution interacts with the numeric features. The encoding also makes an assumption: all category mismatches are treated alike. If the application requires a different dissimilarity structure, ordinary Euclidean k-means may not be the appropriate method.
Mixed Features: A Small Coding Change, a Different Result
Worked Example: A Coded Region Reverses a Two-Cluster Choice
A hypothetical risk team records three profiles using annual event frequency and region. The profiles are P: frequency 2, North; Q: frequency 3, South; and R: frequency 4, North. Compare \(K=2\) k-means using North = 0, South = 1 with the result using North = 0, South = 10. Frequency is used in its stated units, and the region code is the second coordinate.
Method. With three observations and two clusters, one cluster contains a pair and the other is a singleton. For a pair of points \(x_a,x_b\), their centroid is \((x_a+x_b)/2\), and their two-point WCSS is half their squared Euclidean distance. This follows because each point is one-half of the pair's difference from the centroid, so the two squared deviations sum to \(\|x_a-x_b\|^2/2\).
Codes 0 and 1. The points are \(P=(2,0)\), \(Q=(3,1)\), and \(R=(4,0)\). The squared pairwise distances are $$ \|P-Q\|^2=(2-3)^2+(0-1)^2=2,\quad \|Q-R\|^2=(3-4)^2+(1-0)^2=2,\quad \|P-R\|^2=(2-4)^2+(0-0)^2=4. $$ Pairing P with Q gives WCSS \(2/2=1\); pairing Q with R also gives 1; pairing P with R gives \(4/2=2\). The minimum is 1, so either cross-region pair is optimal, with the remaining profile as a singleton.
Codes 0 and 10. Now the points are \(P=(2,0)\), \(Q=(3,10)\), and \(R=(4,0)\). The squared distances are $$ \|P-Q\|^2=1+100=101,\quad \|Q-R\|^2=1+100=101,\quad \|P-R\|^2=4. $$ The corresponding pair WCSS values are \(50.5\), \(50.5\), and \(2\). K-means now pairs P with R, both North records, and leaves Q alone. The frequency observations did not change. The result changed because the arbitrary code gap gave the region coordinate much larger numerical separation.
Check and common mistake avoided. The pairwise calculations and the half-distance WCSS rule agree: under the first coding, a cross-region pair has WCSS 1; under the second, the same kind of pair has WCSS 50.5. It is a mistake to conclude that North is meaningfully closer to South under the first coding than under the second. For a nominal region, both are simply different categories; the apparent size of the difference is imposed by the code.
Exam Traps and Practical Checks
A frequent trap is to treat “numeric input” and “quantitative feature” as synonyms. A software package may accept integer labels and produce a valid numerical optimization, but that does not mean the output answers a meaningful clustering question. For example, coding payment method as 1, 2, and 3 creates a Euclidean geometry that treats method 1 as closer to method 2 than to method 3. That conclusion is not warranted merely because the labels are integers.
Here is the wrong calculation and the correction. Suppose two nominal regions have codes 2 and 8. A mistaken interpretation says, “Their distance is 6, so this pair is twice as far apart as two regions coded 2 and 5, whose distance is 3.” The arithmetic on the codes is correct, but the conclusion is not: neither distance has a defined interpretation for nominal labels. Renumber the same regions as 0 and 1, and the numerical gap becomes 1 without changing the underlying records. Correction: do not interpret Euclidean distances between arbitrary nominal codes; use a representation and distance structure justified by the variables, or choose a method designed for the data types.
Before applying k-means, check each input feature:
- Is the feature quantitative, or is it a category represented by labels?
- If it is ordinal, do both its order and the spacing between values have a defensible interpretation?
- Would a different arbitrary relabeling change the distances or apparent cluster structure?
- If one-hot encoding is used, is equal treatment of every category mismatch reasonable for this application?
- In mixed data, are the contributions of numeric and encoded categorical features sensible for the intended analysis?
These checks help separate a computational answer from a meaningful statistical interpretation. Understanding what the coordinates mean is essential to using k-means responsibly and is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.
Check Your Understanding
For each problem, distinguish numerical computation from whether the resulting distances have a meaningful interpretation.
- A hypothetical dataset records a nominal device type as codes 4, 7, and 20. A colleague claims that type 4 is closer to type 7 than to type 20 because the code gaps are 3 and 16. Is that a valid k-means interpretation, and why? (Answer: No. For nominal device types, the codes do not establish order or distance, so the claimed closeness is unsupported.)
- Three nominal categories are one-hot encoded as \((1,0,0)\), \((0,1,0)\), and \((0,0,1)\). Find the Euclidean distance between two different category vectors and state whether it depends on which two categories are chosen. (Answer: \(\sqrt{2}\), the same for every pair of distinct categories.)
- A cluster contains four one-hot observations: two of type A, one of type B, and one of type C. Give its centroid and WCSS. (Answer: Centroid \((1/2,1/4,1/4)\); WCSS \(4[1-(1/4+1/16+1/16)]=5/2\).)
- Six nominal observations are A, A, B, B, C, C, coded as 0, 0, 1, 1, 5, 5. For \(K=2\), compare the cuts between distinct values and give the minimum WCSS and grouping. (Answer: The A-and-B versus C cut has WCSS 1; the A versus B-and-C cut has WCSS 16. The minimum groups A and B together and isolates C.)
- Three profiles are P: \((5,\text{North})\), Q: \((6,\text{South})\), and R: \((8,\text{North})\), with \(K=2\). Find the paired cluster under North = 0, South = 1, then under North = 0, South = 8, using the numeric value as the first coordinate. (Answer: With codes 0 and 1, pair P and Q (pair WCSS 1). With codes 0 and 8, pair P and R (pair WCSS \(9/2\)); the changed result reflects arbitrary categorical spacing.)