Introduction: Clustering Is a Sequence of Decisions
Hierarchical clustering does not produce one uniquely correct grouping simply by being run on a data set. Its result depends on which features are included, how they are measured or scaled, how dissimilarity is defined, which linkage rule is used, and where the resulting hierarchy is cut. A defensible analysis makes those decisions deliberately and checks whether its main conclusions survive reasonable alternatives.
This workflow is useful when observations are described by several features and there is no response variable to predict. For example, a hypothetical insurer might want to explore groups of policy portfolios by claim frequency and average claim size. Such groups are descriptive: a clustering result alone does not establish that the groups are stable in a new sample or useful for a business decision.
A Decision Sequence
Before calculating distances, state what counts as an observation and what each feature means. Then decide whether a feature should be transformed, standardized, or omitted. These are substantive choices, not automatic data-cleaning steps: standardization can prevent large numerical units from dominating, but it can also give a low-importance feature the same influence as an important one.
Specify the unit being clustered and the interpretation and units of each feature. Decide how missing values and categorical features will be handled before computing distances.
Apply a transformation when it is meaningful, such as a logarithm for a strongly right-skewed positive measurement. Standardize features when their units or spreads would otherwise dominate and equal influence is appropriate.
Use a distance compatible with the feature types and a linkage rule whose cluster geometry fits the purpose. Record the exact choices, including the Ward distance convention if Ward linkage is used.
Look at the sequence and heights of merges. Choose a number of clusters or a dissimilarity threshold, and explain why that cut is useful for the analysis.
Repeat the analysis under reasonable alternative scales or linkages, and examine whether unusual observations control important merges. Summarize the resulting groups in the original feature units.
The feature and distance decisions are connected. For numerical features \(x_1,\ldots,x_p\), Euclidean distance between observations \(i\) and \(j\) is \(d(i,j)=\sqrt{\sum_{r=1}^{p}(x_{ir}-x_{jr})^2}\). A feature with a much larger numerical range can therefore dominate the sum of squared differences. Standardizing feature \(r\) replaces \(x_{ir}\) by \(z_{ir}=(x_{ir}-\bar{x}_r)/s_r\), where \(\bar{x}_r\) is its sample mean and \(s_r\) its sample standard deviation. The resulting Euclidean distance measures separation in standard-deviation units. That can be appropriate, but it is not neutral: it changes the meaning of closeness.
Worked Example: Preparing Features and Choosing a Cut
Worked Example: Preparing Features and Choosing a Cut
A hypothetical analyst wants to group four portfolios, A through D, using annual claim counts per policy and average claim amounts in thousands of dollars. The data are A \((1,10)\), B \((2,11)\), C \((8,19)\), and D \((9,20)\). Use range scaling on each feature, Euclidean distance, and complete linkage. Determine the groups at a dissimilarity threshold of \(0.20\). The method avoids letting the different feature ranges determine the result solely through their units.
Step 1: Scale each feature. The claim-count minimum is 1 and maximum is 9, so the range is 8. The scaled claim counts are A: \((1-1)/8=0\), B: \((2-1)/8=0.125\), C: \((8-1)/8=0.875\), and D: \((9-1)/8=1\). The claim-amount minimum is 10 and maximum is 20, so its range is 10. Its scaled values are likewise 0, 0.1, 0.9, and 1, respectively. The two scaled feature columns are not identical, so calculate distances using both: A \((0,0)\), B \((0.125,0.1)\), C \((0.875,0.9)\), D \((1,1)\).
Step 2: Find the smallest pairwise distances. For A and B, \(d(A,B)=\sqrt{(0.125)^2+(0.1)^2}=\sqrt{0.025625}\approx0.1601\). For C and D, the coordinate differences are 0.125 and 0.1, so \(d(C,D)\approx0.1601\) as well. For A and C, \(d(A,C)=\sqrt{(0.875)^2+(0.9)^2}=\sqrt{1.575625}\approx1.2552\). For A and D, \(d(A,D)=\sqrt{1^2+1^2}\approx1.4142\). For B and C, \(d(B,C)=\sqrt{(0.75)^2+(0.8)^2}=\sqrt{1.2025}\approx1.0966\). For B and D, \(d(B,D)=\sqrt{(0.875)^2+(0.9)^2}\approx1.2552\).
Step 3: Apply complete linkage. The two smallest distances tie, so A and B merge at approximately 0.1601 and C and D merge at approximately 0.1601. The complete-linkage distance between \(\{A,B\}\) and \(\{C,D\}\) is the largest cross-cluster distance. Those four distances are A–C 1.2552, A–D 1.4142, B–C 1.0966, and B–D 1.2552. The largest is 1.4142, so the two pairs merge at that height.
Step 4: Cut at 0.20 and check. The first two merges are below 0.20, and the merge at 1.4142 is above 0.20. The cut therefore gives \(\{A,B\}\) and \(\{C,D\}\). The calculation is internally consistent: each pair’s merge height is less than the threshold, while the merge joining the pairs is greater than it.
A common mistake is to assume both feature columns have exactly the same scaled values because they appear to follow the same pattern. Here they do not: B’s scaled amount is 0.1, not 0.125. Using a distance of \(\sqrt{2}(0.125)\) would therefore be incorrect. The calculated distance, about 0.1601, uses both actual scaled coordinates.
Why Ward Linkage Uses a Within-Cluster Increase
When clusters have sizes \(n_A\) and \(n_B\), and feature vectors have centroids \(\bar{x}_A\) and \(\bar{x}_B\), the increase in within-cluster sum of squares from merging them is \(\Delta(A,B)=\frac{n_A n_B}{n_A+n_B}\lVert\bar{x}_A-\bar{x}_B\rVert^2\). This is the quantity Ward’s rule minimizes at each merge. It includes both the centroid separation and the cluster sizes; using only the distance between centroids would not generally choose the same merge.
To see why, write each observation in \(A\) as \(x_i-\bar{x}_{A\cup B}=(x_i-\bar{x}_A)+(\bar{x}_A-\bar{x}_{A\cup B})\), and expand the squared norm. The cross terms sum to zero because \(\sum_{i\in A}(x_i-\bar{x}_A)=0\). The same argument applies to \(B\). The added terms are therefore \(n_A\lVert\bar{x}_A-\bar{x}_{A\cup B}\rVert^2+n_B\lVert\bar{x}_B-\bar{x}_{A\cup B}\rVert^2\). Since \(\bar{x}_{A\cup B}=(n_A\bar{x}_A+n_B\bar{x}_B)/(n_A+n_B)\), substituting the two centroid offsets gives the stated factor. This formula assumes squared-Euclidean within-cluster sums of squares; it is not a general formula for arbitrary dissimilarities.
Worked Example: Checking Sensitivity to Feature Units
Worked Example: Checking Sensitivity to Feature Units
A hypothetical portfolio study records feature \(x\) in index points and feature \(y\) in cents. Three portfolios have values A \((0,0)\), B \((1,100)\), and C \((2,0)\). Compare Euclidean nearest pairs before and after converting \(y\) from cents to dollars by dividing it by 100. Use the comparison to assess whether the original distance calculation is suitable. This method makes the effect of a unit conversion explicit; the trap is to treat a numerical distance as meaningful regardless of its units.
In the recorded units: \(d(A,B)=\sqrt{(1-0)^2+(100-0)^2}=\sqrt{10001}\approx100.0050\). Similarly, \(d(B,C)=\sqrt{(2-1)^2+(0-100)^2}=\sqrt{10001}\approx100.0050\). Meanwhile, \(d(A,C)=\sqrt{(2-0)^2+(0-0)^2}=2\). Thus A and C are the nearest pair. At a cut of 3, they form one cluster and B is a singleton.
After converting cents to dollars: the coordinates are A \((0,0)\), B \((1,1)\), and C \((2,0)\). Now \(d(A,B)=\sqrt{1^2+1^2}=\sqrt{2}\approx1.4142\), \(d(B,C)=\sqrt{1^2+(-1)^2}=\sqrt{2}\approx1.4142\), and \(d(A,C)=2\). The nearest-pair tie is between A–B and B–C. At a cut of 1.5, either tied pair can merge first, with the remaining portfolio as a singleton.
The nearest-pair conclusion changes after a mere unit conversion. That is evidence that unscaled Euclidean distance is not appropriate here unless the relative numerical units were deliberately intended to determine influence. Converting units alone does not settle how much each feature should matter; the analyst must choose and justify a scaling rule based on the goal of the grouping.
Inspecting the Dendrogram and Checking Unusual Observations
A dendrogram records which clusters merge and the heights associated with those merges. The height scale must be interpreted according to the chosen distance and linkage, not as a probability or a measure of cluster quality. A large increase between successive merge heights can suggest a useful separation, but it does not prove that a particular cut is correct. State the cut rule explicitly—for example, a fixed number of groups or a dissimilarity threshold—and then check whether the resulting groups are interpretable in the original features.
Worked Example: Assessing an Outlier’s Effect on a Cut
A hypothetical analyst uses one numerical feature and single linkage for observations P = 0, Q = 1, R = 4, S = 10, and T = 100. Determine the merge sequence and the groups at threshold 7. Then compare with the analysis after omitting T. This check distinguishes an outlier that affects only the final merge from one that changes earlier group formation.
Step 1: Find the single-linkage merges. The smallest pairwise distance is \(d(P,Q)=1\), so P and Q merge first. The distance from \(\{P,Q\}\) to R is \(\min(|0-4|,|1-4|)=3\), so R joins next. The distance from \(\{P,Q,R\}\) to S is \(\min(10,9,6)=6\), so S joins at height 6. Finally, the distance from \(\{P,Q,R,S\}\) to T is the minimum of 100, 99, 96, and 90, which is 90. T joins at height 90.
Step 2: Apply the threshold. At threshold 7, the merges at 1, 3, and 6 have occurred, but the merge at 90 has not. The groups are therefore \(\{P,Q,R,S\}\) and \(\{T\}\).
Step 3: Compare with T omitted. Without T, the merges remain at 1, 3, and 6, and all four remaining observations form one cluster at threshold 7. In this example T does not alter the early merges; it creates a large final merge height and remains separate at the stated cut. The correct interpretation is not automatically “remove T.” First investigate whether it is a recording error, a genuinely unusual case, or an observation that the analysis is meant to retain.
A common mistake is to infer from a very tall final merge that every earlier group is unreliable. Here the merge sequence shows otherwise: the first three merges occur below 7 both with and without T. The outlier’s influence should be described by the merges or cut it changes, not by its visual prominence alone.
Common Workflow Errors
- Scaling mechanically. Standardizing every feature is not automatically correct. It gives features comparable numerical spread, but may erase meaningful differences in importance.
- Choosing a linkage after seeing a preferred answer. Linkage should reflect the kind of group structure sought; report it and, when useful, compare a plausible alternative.
- Treating a cut as self-evident. Give the number-of-clusters rule or threshold and explain the relevant dendrogram evidence.
- Interpreting labels as rankings. Cluster labels such as 1 and 2 are arbitrary identifiers, not an order or a measure of quality.
- Ignoring sensitivity. If modest, defensible changes to scaling or linkage produce very different groups, report that dependence rather than presenting one grouping as inevitable.
Check Your Understanding
For each problem, follow the stated choices and check that your conclusion matches the merge heights or distances.
- Three observations have scaled coordinates A \((0,0)\), B \((0.3,0.4)\), and C \((1,0)\). Using Euclidean distance, find the nearest pair and say what is merged at a threshold of 0.6. (Answer: \(d(A,B)=0.5\), \(d(A,C)=1\), \(d(B,C)=\sqrt{0.65}\approx0.8062\); A and B merge, so the groups are \(\{A,B\}\) and \(\{C\}\).)
- Two features are measured in dollars and years. Their raw ranges are 10,000 dollars and 2 years. State one reason raw Euclidean distance may be unsuitable and one decision the analyst must make before standardizing. (Answer: The dollar feature may dominate because of its larger numerical range; decide whether equal influence per standardized feature is substantively appropriate.)
- Under complete linkage, clusters \(A=\{1,4\}\) and \(B=\{7\}\) have pairwise distances from B to A of 6 and 3. What is their linkage distance, and what would single linkage give? (Answer: Complete linkage is 6, the maximum; single linkage is 3, the minimum.)
- For one-feature observations at 0, 2, 5, and 30 using single linkage, list the merge heights and give the groups at threshold 4. (Answer: Merge heights are 2, 3, and 25; at threshold 4 the groups are \(\{0,2,5\}\) and \(\{30\}\).)
- A Ward analysis reports that merging clusters of sizes 2 and 4 with centroid separation 3 increases within-cluster sum of squares by 12. Verify the result using the Ward increase formula, and state whether the calculation is justified for a non-Euclidean dissimilarity. (Answer: \((2\cdot4)/(2+4)\times3^2=(8/6)\times9=12\); the stated formula uses squared-Euclidean within-cluster sums of squares and is not justified for an arbitrary non-Euclidean dissimilarity.)