Why Scaling and Distance Choices Matter
Hierarchical clustering starts from pairwise dissimilarities: a rule for how unlike two observations are. The algorithm then uses those dissimilarities, together with a linkage rule, to decide which observations or clusters to merge. Consequently, a clustering is not determined by the data table alone. It also depends on the units and scales of the features and on the chosen distance measure.
For example, a data set might record a building's floor area in square metres and its annual energy use in kilowatt-hours. If the numerical range of energy use is much larger, Euclidean distances can be driven mostly by that feature. The resulting groups may therefore reflect the units of measurement more than the intended balance between features. The remedy is not automatically to standardize everything: the appropriate scaling depends on what differences should matter in the application.
Two common choices for quantitative features are Euclidean and Manhattan distance. For observations with \(p\) features \(x_{i1},\ldots,x_{ip}\), they are
Both use the feature differences, but combine them differently. Euclidean distance gives relatively more influence to a large difference in one coordinate because that difference is squared before summation. Manhattan distance adds absolute differences. Either measure can be used in hierarchical clustering, provided it suits the features and the interpretation of difference.
Scaling also changes dissimilarities. If one feature is multiplied by a constant while the others are left alone, that feature's contribution to distances changes. By contrast, adding the same constant to every observation's value of a feature leaves pairwise differences unchanged. The worked examples make these effects concrete.
Reclustering After Changing One Feature's Scale
The first example uses complete linkage. For two clusters \(G\) and \(H\), complete-linkage dissimilarity is the largest distance between a member of \(G\) and a member of \(H\). When both clusters are single observations, it is just their pairwise distance. At each step, complete linkage merges the pair of clusters with the smallest such dissimilarity.
Worked Example: Scaling One Feature Changes the Clusters
Four hypothetical sites are recorded on features \(x\) and \(y\). Use Euclidean distance and complete linkage. First cluster the raw data; then multiply every \(y\)-value by 3, recluster, and compare the partitions after two merges (three clusters after one merge, two after two).
| Site | \(x\) | \(y\) |
|---|---|---|
| A | 0 | 0 |
| B | 0 | 4 |
| C | 10 | 1 |
| D | 10 | 6 |
Method and raw distances. Calculate each Euclidean distance from the two feature differences. For example, \(d_E(A,B)=\sqrt{0^2+4^2}=4\), and \(d_E(C,D)=\sqrt{0^2+5^2}=5\). The full set of six distances is:
- \(d_E(A,B)=4\).
- \(d_E(A,C)=\sqrt{10^2+1^2}=\sqrt{101}\approx10.050\).
- \(d_E(A,D)=\sqrt{10^2+6^2}=\sqrt{136}\approx11.662\).
- \(d_E(B,C)=\sqrt{10^2+(-3)^2}=\sqrt{109}\approx10.440\).
- \(d_E(B,D)=\sqrt{10^2+2^2}=\sqrt{104}\approx10.198\).
- \(d_E(C,D)=5\).
The smallest distance is \(d_E(A,B)=4\), so the first merge is \(A\) with \(B\). Among the remaining singleton pair \(C,D\) and the distances from the cluster \(\{A,B\}\) to singletons, the smallest available complete-linkage dissimilarity is \(d_E(C,D)=5\). Thus the second merge is \(C\) with \(D\). The two-cluster partition is \(\{A,B\},\{C,D\}\).
Rescale \(y\). The transformed coordinates are \(A=(0,0)\), \(B=(0,12)\), \(C=(10,3)\), and \(D=(10,18)\). The six distances are:
- \(d_E(A,B)=12\).
- \(d_E(A,C)=\sqrt{10^2+3^2}=\sqrt{109}\approx10.440\).
- \(d_E(A,D)=\sqrt{10^2+18^2}=\sqrt{424}\approx20.591\).
- \(d_E(B,C)=\sqrt{10^2+(-9)^2}=\sqrt{181}\approx13.454\).
- \(d_E(B,D)=\sqrt{10^2+6^2}=\sqrt{136}\approx11.662\).
- \(d_E(C,D)=15\).
The first merge is now \(A\) with \(C\), at approximately 10.440. The smallest distance between any remaining clusters is \(d_E(B,D)\approx11.662\), so the second merge is \(B\) with \(D\). The two-cluster partition is now \(\{A,C\},\{B,D\}\).
Check and interpretation. The original two groups pair sites with the same \(x\)-value; after scaling, the groups pair the lower-\(y\) sites together and the higher-\(y\) sites together. In the transformed data, the cross-feature distances \(A\)--\(C\) and \(B\)--\(D\) are smaller than the vertical distances \(A\)--\(B\) and \(C\)--\(D\). The common mistake is to treat multiplying one feature by 3 as a harmless change of units for Euclidean distance. It is not harmless here: it changes that feature's contribution relative to \(x\).
Changing the Distance Measure
The distance measure can change the first merge even when the feature values and their scales are held fixed. The next example uses single linkage, for which the dissimilarity between clusters is the smallest distance between a member of one cluster and a member of the other. In particular, the first merge is the pair of observations with the smallest distance.
Worked Example: Euclidean Versus Manhattan Distance
Four hypothetical devices have two measured features. Use single linkage and compare Euclidean with Manhattan distance. Identify the paired observations after the first merge, corresponding to a cut with three clusters.
| Device | Feature 1 | Feature 2 |
|---|---|---|
| A | 0 | 0 |
| B | 2 | 2 |
| C | 10 | 0 |
| D | 10 | 3 |
Euclidean calculation. The six distances are \(d_E(A,B)=\sqrt{2^2+2^2}=\sqrt{8}\approx2.828\), \(d_E(A,C)=10\), \(d_E(A,D)=\sqrt{109}\approx10.440\), \(d_E(B,C)=\sqrt{8^2+(-2)^2}=\sqrt{68}\approx8.246\), \(d_E(B,D)=\sqrt{8^2+1^2}=\sqrt{65}\approx8.062\), and \(d_E(C,D)=3\). The smallest is \(d_E(A,B)\approx2.828\), so the first merge pairs \(A\) and \(B\).
Manhattan calculation. The six distances are \(d_M(A,B)=|2|+|2|=4\), \(d_M(A,C)=10\), \(d_M(A,D)=13\), \(d_M(B,C)=|8|+|-2|=10\), \(d_M(B,D)=|8|+|1|=9\), and \(d_M(C,D)=3\). The smallest is \(d_M(C,D)=3\), so the first merge instead pairs \(C\) and \(D\).
Check and interpretation. Under Euclidean distance, the two-coordinate difference from \(A\) to \(B\) is slightly shorter than the separation of \(C\) and \(D\). Under Manhattan distance, the total absolute difference for \(A,B\) is 4, exceeding the distance of 3 for \(C,D\). Thus at three clusters the membership differs: \(A\) and \(B\) are together under Euclidean distance, while \(C\) and \(D\) are together under Manhattan distance. The common mistake is to assume that changing the distance formula only changes merge heights. Here it changes which pair merges first.
When Scaling Changes Heights but Not Membership
Not every scaling changes the merge order. If all features are multiplied by the same positive constant, every Euclidean distance is multiplied by that constant. The ordering of distances is preserved, and for single, complete, and average linkage the ordering of cluster dissimilarities is preserved as well. The dendrogram's merge heights change, but its nested partitions do not. This statement concerns uniform scaling of all features; it does not apply to rescaling just one feature.
Worked Example: A Common Scale Factor Changes Heights Only
Use Euclidean distance and complete linkage for \(A=(0,0)\), \(B=(1,0)\), \(C=(4,1)\), and \(D=(4,4)\). Then multiply both coordinates of every observation by 2. Compare the first two merges.
Original distances. \(d_E(A,B)=1\); \(d_E(A,C)=\sqrt{4^2+1^2}=\sqrt{17}\approx4.123\); \(d_E(A,D)=\sqrt{4^2+4^2}=\sqrt{32}\approx5.657\); \(d_E(B,C)=\sqrt{3^2+1^2}=\sqrt{10}\approx3.162\); \(d_E(B,D)=\sqrt{3^2+4^2}=5\); and \(d_E(C,D)=3\). The smallest distance is 1, so the first merge is \(A,B\). The next smallest distance between remaining clusters is \(C,D\), at 3; the complete-linkage distance from \(\{A,B\}\) to \(C\) is \(\max(\sqrt{17},\sqrt{10})=\sqrt{17}\approx4.123\), and to \(D\) it is \(\max(\sqrt{32},5)=\sqrt{32}\approx5.657\). Thus the second merge is \(C,D\).
After multiplying both features by 2. Each of the six pairwise distances doubles. In particular, \(d'_E(A,B)=2\), \(d'_E(C,D)=6\), and the complete-linkage distance between \(\{A,B\}\) and \(C\) is \(2\sqrt{17}\approx8.246\). The first two merges remain \(A,B\) and \(C,D\); their heights are now 2 and 6 rather than 1 and 3.
Check and interpretation. All distances increased in the same proportion, so their rankings and the cluster memberships are unchanged. The common mistake is to infer that any change in numerical merge heights signals a new clustering. Here the heights change, but the merge sequence and partitions do not.
A Practical Diagnosis
When comparing clustering results, state the feature transformation, distance, linkage, and cut used in each analysis. A claim that “the clusters changed” is incomplete unless it identifies the observations whose memberships differ at the same number of clusters or at a clearly stated dissimilarity threshold.
Record the original units and any transformation, such as dividing a feature by a scale or standardizing it. Do not treat different transformations as interchangeable.
For a small data set, a complete distance table is a reliable way to find the first merge and check later candidate merges.
Use the minimum, maximum, or average inter-observation distance according to the linkage rule. Do not substitute a different linkage while attributing a result to scaling or distance.
List the members of each cluster. Compare memberships, not the numerical cluster labels, since labels such as “cluster 1” have no fixed meaning.
Standardizing features is one possible response to incompatible ranges, but it is not an automatic requirement. A feature with a wide range may be substantively important, or it may simply use larger numerical units. The analyst must decide whether features should have comparable influence and whether the chosen distance represents meaningful differences. Sensitivity to plausible scaling and distance choices is useful diagnostic information: if a few small choices reverse memberships, the resulting clusters should be interpreted cautiously.
An important exam trap is to compare raw distances from different analyses as though their heights were directly comparable. A uniform scale factor changes all heights without changing the partitions; a nonuniform feature scaling can change the merge order; and a different distance formula can change which pair is closest. Separate these possibilities before explaining a result. This style of calculation is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.
Check Your Understanding
For each problem, identify the stated linkage and compare memberships at the requested cut.
- Four observations are \(P=(0,0)\), \(Q=(0,2)\), \(R=(5,0)\), and \(S=(5,2)\). Under Euclidean distance and complete linkage, which pairs form the two clusters after two merges? (Answer: \(\{P,Q\}\) and \(\{R,S\}\); each within-pair distance is 2 and every cross-pair distance is at least 5.)
- For \(A=(0,0)\), \(B=(3,1)\), and \(C=(0,4)\), calculate the Euclidean distances and identify the first single-linkage merge. (Answer: \(d_E(A,B)=\sqrt{10}\), \(d_E(A,C)=4\), \(d_E(B,C)=\sqrt{18}\); merge \(A,B\).)
- For the same three observations, calculate the Manhattan distances and identify the first single-linkage merge. (Answer: \(d_M(A,B)=4\), \(d_M(A,C)=4\), \(d_M(B,C)=6\); \(A,B\) and \(A,C\) are tied for the first merge, so the first merge is nonunique.)
- All features of a data set are multiplied by 5, and Euclidean distance with average linkage is used. What happens to pairwise distances, merge heights, and memberships? (Answer: pairwise distances and merge heights are multiplied by 5; the merge order and memberships are unchanged.)
- Two features are \(x\) and \(y\). A proposed transformation replaces each observation \((x,y)\) by \((x,4y)\). Is this a uniform rescaling, and can the merge order be assumed to remain unchanged under Euclidean distance? (Answer: it is not a uniform rescaling; the relative contribution of \(y\) increases, so the merge order and memberships can change.)