Why Coordinate Changes Matter in K-Means
K-means groups observations according to Euclidean distance. That makes its results depend not just on the observations, but on how their features are represented. A change of coordinates can leave every distance comparison intact, or it can change which observations are nearest to which centroids. The distinction matters when interpreting a clustering result or changing the units used to record features.
This tutorial considers transformations applied consistently to every observation and every centroid. The basic model is a finite set of \(n\) observations \(x_1,\ldots,x_n\), each represented by a vector in \(p\)-dimensional Euclidean space. For a fixed partition into \(K\) nonempty clusters, the centroid of a cluster is the arithmetic mean of its observations, and the within-cluster sum of squares is the sum of squared Euclidean distances from observations to their assigned centroids.
Assignments can be discussed either for fixed centroids or for an entire clustering solution. For a full solution, the cluster means must be transformed along with the observations. Translations, rotations, and positive uniform scalings have this property: the transformed mean is the mean of the transformed observations. The results below concern Euclidean k-means and do not automatically apply to other distance measures or clustering methods.
Translations and Rotations Preserve Distances
A translation adds the same vector \(b\) to every point. If the observation is \(x\) and a centroid is \(c\), their transformed difference is \((x+b)-(c+b)=x-c\). The common shift cancels, so the distance is unchanged.
A rotation changes the direction of vectors without changing their lengths. In matrix notation, a rotation matrix \(R\) is orthogonal: \(R^\mathsf{T}R=I\), where \(I\) is the identity matrix. Thus the squared distance between rotated points is \((Rx-Rc)^\mathsf{T}(Rx-Rc)=(x-c)^\mathsf{T}R^\mathsf{T}R(x-c)=(x-c)^\mathsf{T}(x-c)\).
The same reasoning applies to every observation-centroid pair. Therefore a translation, rotation, or sequence of the two preserves all squared distances, nearest-centroid comparisons, and the within-cluster sum of squares (WCSS). If a solution has a tie between centroids, the tie remains a tie; the transformation does not resolve it.
Worked Example: Translate and Rotate a Nearest-Centroid Comparison
A hypothetical risk profile is \(x=(2,1)\), with candidate centroids \(c_1=(0,0)\) and \(c_2=(4,1)\). First determine the nearest centroid. Then translate all three points by \(b=(3,-2)\) and rotate them \(90\) degrees counterclockwise. Check whether the assignment changes.
Method. Compare squared Euclidean distances; taking square roots is unnecessary because square root is increasing and therefore does not change which distance is smaller.
Before transformation. To \(c_1\), the squared distance is \((2-0)^2+(1-0)^2=4+1=5\). To \(c_2\), it is \((2-4)^2+(1-1)^2=4+0=4\). Since \(4<5\), the profile is assigned to \(c_2\).
After translation. The transformed profile is \(x+b=(5,-1)\). The centroids become \(c_1+b=(3,-2)\) and \(c_2+b=(7,-1)\). The squared distances are \((5-3)^2+(-1+2)^2=4+1=5\) and \((5-7)^2+(-1+1)^2=4+0=4\). The nearest centroid is still the transformed \(c_2\).
After rotation. The \(90\)-degree counterclockwise rotation maps \((a_1,a_2)\) to \((-a_2,a_1)\). The profile becomes \((1,5)\), and the translated centroids become \((2,3)\) and \((1,7)\). The squared distances are \((1-2)^2+(5-3)^2=1+4=5\) and \((1-1)^2+(5-7)^2=0+4=4\). The assignment remains with the second centroid.
Check and trap avoided. The original and final squared distances are both 5 and 4, respectively. A common error is to rotate the observation but not the centroids; the distance-preservation result requires the same transformation to be applied to both.
Uniform Scaling Preserves Assignments, but Changes WCSS
A positive uniform scale multiplies every coordinate of every point by the same constant \(s>0\). Differences are multiplied by \(s\), so distances are multiplied by \(s\), while squared distances are multiplied by \(s^2\). Because every candidate-centroid squared distance is multiplied by the same positive number, their ordering is unchanged.
For a fixed partition, scaling all observations by \(s\) also scales each cluster mean by \(s\). Consequently, every term in WCSS is multiplied by \(s^2\), and the total WCSS is multiplied by \(s^2\). The partition that minimizes WCSS is unchanged, but the numerical value of the minimum is not, unless \(s=1\). This conclusion assumes the scale is positive and common to all features.
Worked Example: Scale a Complete K-Means Solution
Four hypothetical one-dimensional profiles have values \(0,2,8,10\). Consider the partition \(\{0,2\}\) and \(\{8,10\}\), with centroids 1 and 9. Scale every value and centroid by \(s=3\). Verify the assignments and calculate WCSS before and after scaling.
Method. Calculate the original squared deviations, then transform the observations and centroids consistently and recalculate. This directly checks the \(s^2\) rule.
Original centroids and WCSS. The first mean is \((0+2)/2=1\), and the second is \((8+10)/2=9\). The four squared deviations are \((0-1)^2=1\), \((2-1)^2=1\), \((8-9)^2=1\), and \((10-9)^2=1\). Thus WCSS is \(1+1+1+1=4\). Each point is closer to its stated centroid than to the other: for example, 2 is 1 unit from 1 and 7 units from 9, while 8 is 1 unit from 9 and 7 units from 1.
After scaling. The observations become \(0,6,24,30\), and the centroids become \(3\) and \(27\). The transformed means are \((0+6)/2=3\) and \((24+30)/2=27\), confirming that the centroids transform consistently. The first two observations remain nearest to 3, and the last two remain nearest to 27. Each squared deviation is now 9: \((0-3)^2=9\), \((6-3)^2=9\), \((24-27)^2=9\), and \((30-27)^2=9\). The new WCSS is \(9+9+9+9=36\).
Check and trap avoided. Since \(s^2=3^2=9\), the predicted WCSS is \(9(4)=36\), matching the direct calculation. Uniform scaling preserves the partition, not the WCSS value itself.
Feature-Specific Scaling Can Change Assignments
If each feature is scaled by a different factor, the transformation is generally not a rotation or a uniform scale. For a diagonal scaling matrix with positive entries \(s_1,\ldots,s_p\), the squared distance becomes \(\sum_{j=1}^{p}s_j^2(x_j-c_j)^2\). A feature with a larger scale therefore contributes more to squared distance. The relative sizes of distances to different centroids can change, so assignments can change too.
This result does not say that every feature-specific scaling changes a particular assignment. An assignment stays the same whenever the nearest-centroid comparison remains in the same order. But, unlike translation, rotation, and positive uniform scaling, feature-specific scaling does not guarantee that outcome for all data. Subtracting a fixed feature mean before scaling is only a translation and, by itself, does not affect distances; unequal scale factors are the part that can alter the geometry.
Worked Example: A Feature-Specific Scale Reverses an Assignment
A hypothetical profile has two features, and its coordinates are \(x=(0,0)\). Its candidate centroids are \(c_A=(3,0)\) and \(c_B=(0,2)\). Find the nearest centroid, then double the second feature for the profile and both centroids and repeat the comparison.
Method. Compare squared Euclidean distances before and after the same feature-specific transformation. This identifies whether the nearest-centroid ordering is preserved.
Before scaling. The squared distance to \(c_A\) is \((0-3)^2+(0-0)^2=9\). The squared distance to \(c_B\) is \((0-0)^2+(0-2)^2=4\). Since \(4<9\), the profile is assigned to \(c_B\).
After doubling feature two. The profile remains \((0,0)\); \(c_A\) remains \((3,0)\); and \(c_B\) becomes \((0,4)\). The squared distance to \(c_A\) is still \((0-3)^2+(0-0)^2=9\). The squared distance to \(c_B\) is now \((0-0)^2+(0-4)^2=16\). Since \(9<16\), the profile is now assigned to \(c_A\).
Check and trap avoided. The original comparison favored \(c_B\), while the transformed comparison favors \(c_A\), so the feature-specific scaling reversed the assignment. Do not assume that scaling every point consistently must preserve assignments: it preserves the transformed coordinates, but unequal feature scales change the distance weights.
Common Exam Trap: Recalculate Every Coordinate
An easy calculation error is to apply a feature scale to a point but forget that the same feature of each centroid must also be scaled. Another is to transform the coordinates correctly but miscalculate the resulting squared distance. For example, take \(x=(1,2)\), \(c_1=(0,0)\), and \(c_2=(3,1)\). Before scaling, the squared distances are \(1^2+2^2=5\) and \((-2)^2+1^2=5\), so the point is tied between the centroids.
Now double the second feature for all three points. The transformed observation is \((1,4)\), and the transformed centroids are \((0,0)\) and \((3,2)\). The squared distances are \(1^2+4^2=17\) and \((-2)^2+2^2=4+4=8\). The second centroid is nearer. Writing 5 for that second transformed distance is incorrect: the vertical difference is \(4-2=2\), whose square is 4, and the horizontal difference contributes another 4.
Check Your Understanding
For each problem, transform all observations and centroids as specified. Use squared distances where convenient.
- An observation is \(x=(1,2)\), with centroids \(c_1=(0,0)\) and \(c_2=(2,3)\). Find the original squared distances. Translate all three points by \((4,-1)\). What are the new squared distances and nearest-centroid assignment? (Answer: Original squared distances are 5 and 2; translated squared distances are 5 and 2; \(c_2\) is nearest.)
- Rotate \(x=(1,0)\) and centroid \(c=(0,0)\) by \(90\) degrees counterclockwise. Calculate the squared distance before and after rotation. (Answer: The squared distance is 1 before and 1 after.)
- A fixed clustering has WCSS 12. Every feature of every point is multiplied by \(s=2\). What is the new WCSS, and do nearest-centroid assignments change? (Answer: New WCSS is \(2^2(12)=48\); assignments are unchanged.)
- An observation \(x=(0,0)\) has centroids \(c_1=(2,0)\) and \(c_2=(0,3)\). Which centroid is nearest before scaling? Then triple the second feature for all points and decide again. (Answer: Before scaling, squared distances are 4 and 9, so \(c_1\) is nearest; afterward they are 4 and 81, so \(c_1\) remains nearest.)
- An observation \(x=(1,1)\) has centroids \(c_A=(4,1)\) and \(c_B=(1,3)\). Find the nearest centroid before and after doubling the second feature for every point. Explain whether the assignment is guaranteed to remain unchanged by this transformation. (Answer: Before scaling, squared distances are 9 and 4, so \(c_B\) is nearest; afterward the transformed points are \(x=(1,2)\), \(c_A=(4,2)\), and \(c_B=(1,6)\), giving squared distances 9 and 16, so \(c_A\) is nearest; feature-specific scaling does not guarantee unchanged assignments.)