Why Inspect Cluster Sizes and Spread?
A k-means result assigns every observation to a cluster and places a centroid at the coordinate-wise mean of its assigned observations. The assignments and centroids are not, by themselves, a complete description of the result. A membership count can reveal that one group is much smaller than the others, while feature-level spread can show whether a group is compact in some measured variables and diffuse in others.
These summaries are diagnostic, not automatic verdicts. A small cluster may represent a meaningful rare profile, or it may be an unstable group formed around unusual observations. A large spread may indicate a poorly summarized group, but it can also arise from a meaningful range of values. Interpretation depends on the variables, their scales, and the purpose of the analysis.
The variance here divides by \(n_k\), because it describes the assigned observations themselves. It is not the usual sample-variance estimate with denominator \(n_k-1\). If a cluster contains one observation, its descriptive variance is zero in every feature: there is no within-group variation among its observed members. That zero is not evidence that the profile is reliably homogeneous.
The cluster’s proportion of all observations is \(n_k/n\). Counts and proportions answer different practical questions: the count gives the number of records, while the proportion gives the group’s share of the data set. A count that is small in a large data set may still be operationally substantial; the proportion helps put it in context.
Connecting Feature Spreads to Within-Cluster Variation
The within-cluster sum of squares (WCSS) adds the squared Euclidean distances from observations to their assigned centroids. For features recorded on the scale used for clustering, it can be written as the sum of the feature-level squared deviations:
The second equality follows directly from the definition of \(v_{kj}\): for a fixed cluster and feature, the sum of squared deviations is \(n_kv_{kj}\). Summing that identity over features and clusters gives the result. Thus WCSS combines both the number of observations and their feature-level variances. It is a total, not a per-observation spread.
A useful companion summary is the average squared distance from observations to their assigned centroids, \(\mathrm{WCSS}/n\). It is also the membership-weighted average of the clusters’ sums of feature variances:
The weighting matters. A simple average of the cluster variances gives a small cluster the same influence as a large one, even though it contains fewer observations. Neither summary should be mistaken for the complete distribution of values inside a cluster.
Worked Example: Comparing a Smaller and a Larger Cluster
A hypothetical two-feature data set has six observations assigned to two clusters. Cluster A contains \((0,0),(0,2),(2,0),(2,2)\); cluster B contains \((8,0),(12,0)\). Calculate each cluster’s size, centroid, feature-level variances, and WCSS contribution. Which cluster has greater average squared distance to its centroid?
Method. Calculate each centroid as the feature-wise mean, then use the descriptive variance with denominator \(n_k\). The cluster’s WCSS contribution is \(n_k\) times the sum of its feature variances.
Cluster A. It has \(n_A=4\) observations, so its proportion is \(4/6=2/3\). The first-feature mean is \((0+0+2+2)/4=1\); the second-feature mean is \((0+2+0+2)/4=1\). Its centroid is \((1,1)\). For either feature, the deviations from the mean are \(-1,-1,1,1\), so the variance is \((1+1+1+1)/4=1\). Its WCSS contribution is \(4(1+1)=8\).
Cluster B. It has \(n_B=2\), with proportion \(2/6=1/3\). Its centroid is \(((8+12)/2,(0+0)/2)=(10,0)\). The first-feature deviations are \(-2,2\), giving variance \((4+4)/2=4\); the second-feature variance is \(0\). Its WCSS contribution is \(2(4+0)=8\).
The total WCSS is \(8+8=16\). Cluster A’s average squared distance per member is \(8/4=2\); cluster B’s is \(8/2=4\). Thus the smaller cluster has the greater average squared distance, despite equal WCSS contributions. The common mistake is to read equal total contributions as equal compactness. Dividing each contribution by its membership count corrects that comparison.
Feature-Level Variation Shows Direction, Not the Whole Shape
Looking at each feature separately can reveal different kinds of spread that a single total conceals. For example, one group may vary substantially in a claim-frequency measure and little in average claim size, while another shows the reverse pattern. Standard deviations \(s_{kj}\) are often easier to interpret than variances \(v_{kj}\), because standard deviations use the feature’s original units. Variances use squared units.
This interpretation is tied to the scale of the clustering inputs. If features were standardized before clustering, spreads calculated on those standardized inputs describe variation in standard-deviation units. If values are transformed back to original units, their spreads must be interpreted on those original scales. A numerical comparison of variances across differently scaled features can be misleading; the variables’ scales must be understood before interpreting which spread is “larger.”
Worked Example: Equal WCSS, Different Feature Patterns
In a hypothetical two-feature clustering, cluster A contains \((2,5),(4,5),(6,5)\), while cluster B contains \((10,1),(10,3),(10,5)\). Find the centroid, feature-level variances, and WCSS contribution for each cluster. Explain why WCSS alone does not describe the pattern of variation.
Method. For each assigned group, calculate the centroid and then the mean squared deviation separately for each feature.
Cluster A. Its size is 3. The centroid is \(((2+4+6)/3,(5+5+5)/3)=(4,5)\). The first-feature deviations are \(-2,0,2\), so its variance is \((4+0+4)/3=8/3\). The second feature is constant, so its variance is 0. Its WCSS contribution is \(3(8/3+0)=8\).
Cluster B. Its size is also 3. Its centroid is \(((10+10+10)/3,(1+3+5)/3)=(10,3)\). The first feature is constant, so its variance is 0. The second-feature deviations are \(-2,0,2\), giving variance \((4+0+4)/3=8/3\). Its WCSS contribution is \(3(0+8/3)=8\).
Both clusters have WCSS contribution 8, but cluster A varies only in its first feature and cluster B only in its second. The calculation checks because each cluster has three observations and its nonzero feature contributes \(3(8/3)=8\). A total WCSS does not say which feature accounts for the variation.
Feature-level variances are still marginal summaries: they calculate variation one coordinate at a time. They do not reveal how features vary together. For instance, they do not distinguish all possible relationships between two features that have the same separate means and variances. A scatterplot or other joint summary may be needed to inspect such structure. A group may also contain distinct subgroups whose combined marginal spread hides that internal pattern.
Use Counts and Spreads as Diagnostics
A sensible inspection considers size and spread together. A tiny group with large spread on several features merits attention: it may contain unusual observations, or its membership may be sensitive to the clustering procedure. A tiny group with low spread may be a coherent rare profile, but its small membership still means that conclusions about its broader importance require care. A large group with high spread may be heterogeneous, or it may simply cover a broad range that is relevant to the application.
These summaries do not, on their own, establish that a clustering is good or bad. Counts do not measure separation between centroids. Within-cluster spreads do not show whether different clusters overlap, whether the assignments are stable under new starts, or whether the groups are useful for a particular decision. Those questions need additional analysis and context; the earlier material on silhouette widths, stability across initializations, and WCSS addresses related but distinct diagnostics.
Worked Example: Weighting Spread Across Unequal Group Sizes
Suppose a hypothetical one-feature clustering has cluster A values \(0,2\) and cluster B values \(10,10,10,10\). Calculate each cluster’s mean and variance, the total WCSS, and the overall within-cluster variance per observation. Explain why the unweighted average of the two variances is not that overall summary.
Method. Calculate each variance around its own cluster mean. Then combine the cluster sums of squared deviations and divide by the total number of observations.
Cluster A. Its mean is \((0+2)/2=1\). Its variance is \(((0-1)^2+(2-1)^2)/2=(1+1)/2=1\), so its squared-deviation total is \(2(1)=2\).
Cluster B. Its mean is 10, and all four observations equal 10. Its variance and squared-deviation total are both 0.
The total WCSS is \(2+0=2\), and the full data set has \(2+4=6\) observations. Therefore, the within-cluster variance per observation is \(2/6=1/3\). In contrast, the unweighted average of the two cluster variances is \((1+0)/2=1/2\). That gives the small cluster the same weight as the cluster with twice as many members, so it is not the per-observation summary. The check is \(n_Av_A+n_Bv_B=2(1)+4(0)=2\), as required.
Common Mistakes
- Confusing size with spread. A larger cluster does not necessarily have greater within-cluster variation. Compare variances or standard deviations, not counts alone.
- Confusing total WCSS with average spread. WCSS includes the number of observations. When comparing average squared distance per observation, divide by the total membership count.
- Using the wrong denominator for a descriptive summary. Here \(v_{kj}\) divides by \(n_k\), because it describes the assigned members. Replacing it with \(n_k-1\) changes the quantity being reported.
- Treating a singleton’s zero variance as strong evidence. One point has no observed within-group deviations, but that does not establish that the group is a stable or meaningful profile.
- Overinterpreting a feature-wise table. Marginal spreads do not show the joint shape of a group, overlap between groups, or separation between centroids.
- Ignoring input scales. A variance measured in squared dollars cannot be directly compared with one measured in squared years without considering the scales and preprocessing.
Check Your Understanding
For each problem, use descriptive within-cluster variances with denominator equal to the cluster size.
- A one-feature cluster contains \(1,3,5\). Find its centroid, variance, and WCSS contribution. (Answer: centroid \(3\), variance \(8/3\), WCSS contribution \(8\))
- A two-feature cluster contains \((1,1),(3,1),(5,1)\). Find both feature variances and its WCSS contribution. (Answer: variances \(8/3\) and \(0\); WCSS contribution \(8\))
- Two clusters have sizes 2 and 6 and one-feature variances 4 and 1, respectively. Find the combined within-cluster variance per observation. (Answer: \((2\cdot4+6\cdot1)/8=14/8=1.75\))
- A data set of 10 observations has cluster counts 7, 1, and 2. Give their proportions and identify the smallest group by proportion. (Answer: \(0.70, 0.10, 0.20\); the cluster of size 1 is smallest)
- In a two-feature clustering, cluster A has size 4 and variances 1 and 2; cluster B has size 6 and variances 3 and 0. Find the total WCSS and the average squared distance per observation. (Answer: WCSS \(=4(1+2)+6(3+0)=30\); average \(=30/10=3\))