Assignments Are Not the Same as Separation
K-means returns cluster assignments, but an assignment is not by itself evidence that the groups are clearly separated. Under the nearest-centroid rule, each observation is assigned to a closest center. If observations lie near a boundary between centers, small changes in their coordinates can change the assigned cluster. If the groups overlap in the feature space, the algorithm still returns a partition: it does not label an observation “partly in each cluster.”
This distinction matters when interpreting a clustering result. A forced label can be useful for summarizing a data set, but it should not be mistaken for a known underlying category or a probability of membership. To assess ambiguity geometrically, compare an observation’s distance to its nearest center with its distance to the next-nearest center, while also considering how far the observation is from the centers themselves.
The distance calculation assumes the features and their scales are the ones intended for the clustering analysis. Euclidean distances can be dominated by features with large numerical scales if the inputs have not been prepared appropriately. The earlier material on feature scaling explains why the choice of scale affects K-means. Once the feature representation is fixed, distances determine the geometric assignment.
Nearest Centers and Equal-Distance Boundaries
For comparing two centers, squared Euclidean distances are often easier to compute than distances themselves. Since squaring preserves the order of nonnegative numbers, \(d_1(x)<d_2(x)\) if and only if \(d_1(x)^2<d_2(x)^2\). Thus an observation is assigned to whichever of the two centers has the smaller squared distance. Equal squared distances identify a tie.
For two distinct centroids, the set of points equally distant from them is a hyperplane: a line in two dimensions and a plane in three dimensions. This boundary separates the two nearest-center regions. If the centroids coincide, however, every observation is equally distant from them, so every observation is tied between those two centers; the equal-distance set is then the entire feature space, not a hyperplane.
This derivation explains why the boundary is linear: the \(x^\mathsf{T}x\) terms cancel, leaving an equation that is linear in \(x\). When \(c_1\ne c_2\), the coefficient vector \(c_2-c_1\) is nonzero, so this equation defines a hyperplane. When \(c_1=c_2\), the equation reduces to \(0=0\), consistent with every point being tied.
Worked Example: A Point Close to the Boundary
Suppose two centers in a two-feature space are \(c_1=(2,1)\) and \(c_2=(6,1)\). For each observation below, calculate both distances, assign the observation, and identify which one is tied or close to the boundary. The feature units are assumed to have already been chosen for this hypothetical analysis.
Method. Compare squared Euclidean distances first, then take square roots to report the distances. The boundary is halfway between the centers because their second coordinates are equal.
For \(P=(3.9,1.5)\), the squared distances are \(d_1(P)^2=(3.9-2)^2+(1.5-1)^2=1.9^2+0.5^2=3.61+0.25=3.86\) and \(d_2(P)^2=(3.9-6)^2+(1.5-1)^2=(-2.1)^2+0.5^2=4.41+0.25=4.66\). Therefore, \(d_1(P)=\sqrt{3.86}\approx1.9647\) and \(d_2(P)=\sqrt{4.66}\approx2.1587\). Point \(P\) is assigned to center 1; the distance gap is about \(2.1587-1.9647=0.1940\).
For \(Q=(4.1,0.5)\), the squared distances are \(d_1(Q)^2=2.1^2+(-0.5)^2=4.41+0.25=4.66\) and \(d_2(Q)^2=(-1.9)^2+(-0.5)^2=3.61+0.25=3.86\). Thus \(d_1(Q)\approx2.1587\) and \(d_2(Q)\approx1.9647\); \(Q\) is assigned to center 2, with the same distance gap, approximately \(0.1940\).
For \(R=(4,2)\), both squared distances equal \((4-2)^2+(2-1)^2=4+1=5\). Both distances are \(\sqrt{5}\approx2.2361\), so \(R\) is tied. The boundary equation is \(x=4\): the centers are distinct and aligned horizontally, so all points with first coordinate 4 are equally distant from them.
Interpretation and check. \(P\) and \(Q\) receive definite nearest-center labels, but their distance gaps are small and equal, while \(R\) is exactly tied. The equal gaps for \(P\) and \(Q\) are consistent with their positions on opposite sides of the boundary. The common mistake is to treat every returned label as evidence of clear separation; here the rule supplies labels even though these observations are near a tie.
Distance Gaps as a Borderline Diagnostic
For an observation \(x\), sort its distances to all centers from smallest to largest. Call the smallest \(d_{(1)}(x)\) and the second-smallest \(d_{(2)}(x)\). A small gap \(d_{(2)}(x)-d_{(1)}(x)\) indicates that the nearest-center decision is sensitive: a modest change in the observation or the centers could reverse the ordering. A zero gap means an exact tie for nearest center.
The gap should be interpreted alongside the nearest distance. Two observations can have the same gap while one is close to a center and the other is far from every center. A useful relative summary is the gap divided by the second-nearest distance, when that distance is positive. It can help compare observations on different distance scales, but it is still only a geometric diagnostic. Neither a small absolute gap nor a small relative gap is a calibrated uncertainty measure.
Worked Example: Ranking Borderline Observations
In a hypothetical portfolio segmentation, two centers in already-scaled two-feature coordinates are \(c_1=(0,0)\) and \(c_2=(4,0)\). Consider profiles \(U=(2.1,0.3)\), \(V=(3,0.2)\), and \(W=(0.5,0.4)\). Use distances to assign each profile and identify which is most borderline by the nearest-versus-second-nearest distance gap.
Method. Calculate the squared distances to both centers, take square roots, and compare the gaps. The boundary is \(x=2\), so closeness to that line provides an additional geometric check.
For \(U\), the squared distances are \(d_1(U)^2=2.1^2+0.3^2=4.41+0.09=4.50\) and \(d_2(U)^2=(-1.9)^2+0.3^2=3.61+0.09=3.70\). The distances are \(d_1(U)=\sqrt{4.50}\approx2.1213\) and \(d_2(U)=\sqrt{3.70}\approx1.9235\). Thus \(U\) is assigned to center 2. Its gap is approximately \(2.1213-1.9235=0.1978\), and its relative gap using the larger distance is \(0.1978/2.1213\approx0.0932\).
For \(V\), \(d_1(V)^2=3^2+0.2^2=9+0.04=9.04\), while \(d_2(V)^2=(-1)^2+0.2^2=1+0.04=1.04\). Hence \(d_1(V)\approx3.0067\) and \(d_2(V)\approx1.0198\). It is assigned to center 2, with gap approximately \(3.0067-1.0198=1.9869\).
For \(W\), \(d_1(W)^2=0.5^2+0.4^2=0.25+0.16=0.41\), while \(d_2(W)^2=(-3.5)^2+0.4^2=12.25+0.16=12.41\). Thus \(d_1(W)\approx0.6403\) and \(d_2(W)\approx3.5228\). It is assigned to center 1, with gap approximately \(3.5228-0.6403=2.8825\).
Interpretation and check. \(U\) is the most borderline by a wide margin: its gap is about \(0.1978\), compared with \(1.9869\) for \(V\) and \(2.8825\) for \(W\). It also lies only \(0.1\) units horizontally from the boundary \(x=2\). The assignment is still center 2, not a fractional assignment. The common mistake is to read the label as certainty; the calculation supports a nearest-center choice while highlighting its relative ambiguity.
What Overlap Does and Does Not Mean
In ordinary interpretation, clusters overlap when observations from the groups occupy some of the same region of feature space, or when many observations are close to the boundaries between their assigned centers. K-means itself does not estimate overlapping membership distributions. It divides the feature space into nearest-center regions and gives each observation one label, apart from ties that may be resolved by an implementation-specific rule.
A small distance gap is evidence that the nearest-center decision is geometrically borderline, not proof that the underlying groups overlap in a probabilistic sense. Likewise, a large gap does not prove that the clusters are well separated overall. It describes one observation relative to the fitted centers. To assess the overall clustering, examine the broader pattern of observations and the cluster structure rather than relying on a single label or distance.
This distinction is especially important in applied work. For example, a hypothetical insurer might cluster claim profiles using chosen numerical features. A profile near a boundary could be assigned to one segment for a summary or operational rule, but the assignment does not establish that the claim has a known “true” segment. A decision that requires certainty or a calibrated probability needs a method designed to provide that interpretation; nearest-centroid distances alone do not.
Common Exam Trap: Using a Midpoint on One Coordinate
A frequent error is to assume that the boundary between two centers is always the midpoint of one coordinate. That shortcut works only in special geometries, such as centers that differ in just one coordinate and are equal on all others. For general centers in multiple dimensions, use the equal-squared-distance equation.
Worked Example: Correcting a Misread Boundary
Let \(c_1=(1,1)\) and \(c_2=(5,4)\), and consider \(x=(3,2)\). A tempting but incorrect shortcut is to look only at the first coordinates, observe their midpoint is 3, and call \(x\) tied because its first coordinate is 3.
Correct method. Compute both full Euclidean distances. From \(x\) to \(c_1\), the squared distance is \((3-1)^2+(2-1)^2=4+1=5\), so the distance is \(\sqrt{5}\approx2.2361\). From \(x\) to \(c_2\), the squared distance is \((3-5)^2+(2-4)^2=4+4=8\), so the distance is \(\sqrt{8}\approx2.8284\). The observation is assigned to \(c_1\), not tied.
The actual boundary follows from the equal-distance equation: \(2((5-1)x_1+(4-1)x_2)=\|c_2\|^2-\|c_1\|^2\). Since \(\|c_2\|^2=25+16=41\) and \(\|c_1\|^2=1+1=2\), this becomes \(8x_1+6x_2=39\). At the observation \((3,2)\), the left side is \(8(3)+6(2)=36\), not 39. It is therefore not on the boundary. The wrong calculation ignored the second coordinate; the full distance comparison corrects the assignment.
Check Your Understanding
For each problem, use Euclidean distance in the stated feature coordinates. Treat the centers as fixed and report the nearest-center assignment or tie as requested.
- Centers are \(c_1=(1,0)\) and \(c_2=(5,0)\). For \(x=(2.8,0.6)\), calculate both distances, state the assignment, and give the distance gap to four decimals. (Answer: \(d_1\approx1.8974\), \(d_2\approx2.2804\); assign to \(c_1\); gap \(\approx0.3830\).)
- Centers are \(c_1=(0,0)\) and \(c_2=(0,6)\). For \(x=(1,3)\), determine whether it is tied and state the equal-distance boundary. (Answer: tied; the boundary is \(y=3\), and both distances are \(\sqrt{10}\approx3.1623\).)
- Centers are \(c_1=(0,0)\) and \(c_2=(3,4)\). For \(x=(2,1)\), calculate the squared distances, assign the point, and give the equal-distance boundary. (Answer: squared distances \(5\) and \(10\); assign to \(c_1\); boundary \(6x_1+8x_2=25\).)
- Centers are \(c_1=(0,0)\) and \(c_2=(4,0)\). Compare \(U=(2.2,0.4)\) and \(V=(3,0.2)\): calculate their nearest-center assignments and distance gaps, then identify the more borderline observation. (Answer: both are assigned to \(c_2\); \(U\)'s gap is \(\sqrt{5}-\sqrt{3.4}\approx0.3922\), while \(V\)'s is \(\sqrt{9.04}-\sqrt{1.04}\approx1.9869\); \(U\) is more borderline.)
- Two distinct centers are \(c_1=(2,-1)\) and \(c_2=(2,5)\). Give the equal-distance boundary and explain what happens for an observation on it. (Answer: the boundary is \(x_2=2\); every point on it is tied between the two centers.)