Introduction
A dendrogram represents a sequence of agglomerative merges, with the height of each merge showing the dissimilarity at which two clusters joined. A horizontal line at a chosen dissimilarity threshold turns that nested sequence into one particular clustering. Moving the line upward accepts more merges, so clusters combine and their count can only stay the same or decrease.
A threshold answers a different practical question from asking for a prespecified number of clusters: it sets an allowable dissimilarity level, then lets the dendrogram determine how many clusters result. This is useful when a meaningful scale of separation is available, but it requires careful attention to the height values, ties, and the convention used when the threshold equals a merge height.
How a Threshold Determines the Cluster Count
Suppose a dendrogram has \(n\) observations and its merge heights, in agglomeration order, are \(h_1,\ldots,h_{n-1}\). Each agglomerative step joins two existing clusters. Therefore, each accepted merge reduces the number of current clusters by exactly one. Under the convention above, the number of clusters at threshold \(\tau\) is
where \(I(h_j \leq \tau)\) equals 1 when merge \(j\) is accepted and 0 otherwise. Equivalently, subtract from \(n\) the number of merge heights at or below the threshold. This formula counts merges, not the number of plotted branch segments or the number of labels printed on a diagram.
The count is a nonincreasing step function of \(\tau\). Between consecutive distinct merge heights, changing the threshold does not change the clustering. At a merge height, the count drops by the number of merges at that height in the displayed agglomeration schedule. If several merges are tied, the count can therefore drop by more than one at a single threshold value.
The equality convention matters. Here, a merge at exactly \(\tau\) is included. Some software or diagrams may describe a cut using a line placed infinitesimally below or above a node; do not rely on the drawing alone when equality matters. State the convention and use it consistently. At a tied height, the threshold count follows the schedule, while the identities of the groups formed can depend on how tied merges were resolved.
Reading Membership and Count from a Merge Schedule
A merge schedule gives the two clusters joined at each step and the corresponding height. To identify the clusters at a threshold, start with each observation as its own cluster and apply each merge whose height is at most \(\tau\). This is also a reliable way to check the branch tracing on a dendrogram, especially when the tree is crowded.
Worked Example: Counting Clusters Across a Tied Height
A hypothetical clustering of six service locations has the following merge schedule. The labels \(AB\) and \(CD\) denote clusters formed from the listed observations.
| Merge | Clusters joined | Height |
|---|---|---|
| 1 | A and B | 0.8 |
| 2 | C and D | 1.4 |
| 3 | AB and E | 2.1 |
| 4 | CD and F | 2.1 |
| 5 | ABE and CDF | 4.0 |
Find the cluster count at thresholds \(\tau=1.9\), \(\tau=2.1\), and \(\tau=3.0\), and give the memberships at \(\tau=2.1\).
Method. There are \(n=6\) observations, so start with six singleton clusters. At \(\tau=1.9\), the accepted heights are 0.8 and 1.4. There are two accepted merges, so
The clusters are \(AB\), \(CD\), \(E\), and \(F\). At \(\tau=2.1\), the merges at 2.1 are included by the stated equality convention. Four merges have now been accepted:
Applying the merges gives clusters \(ABE\) and \(CDF\). At \(\tau=3.0\), no additional merge is accepted because the remaining height is 4.0. Thus \(K(3.0)=6-4=2\), with the same memberships. Check: the two groups at 2.1 contain three observations each, accounting for all six.
Common mistake avoided: Counting only the distinct merge heights at or below 2.1 would give two and incorrectly suggest \(6-2=4\) clusters. There are four accepted merge events: the two at 0.8 and 1.4, and both separate merges tied at 2.1.
Tracing the Branches to Recover Cluster Membership
A count alone does not identify which observations are grouped together. For membership, trace each leaf upward only as far as merges at or below the threshold. Leaves connected below or at the cut belong to the same cluster; branches that first join above the cut belong to different clusters. The threshold should be interpreted on the dissimilarity axis, not as a number of merge steps.
Read \(\tau\) on the dendrogram's dissimilarity axis and draw a horizontal guide at that level.
Use the specified rule, here \(h_j \leq \tau\), to determine which internal nodes are included.
For each connected branch group formed by included merges, list all its leaves as one cluster.
Confirm that the number of groups equals \(n\) minus the number of accepted merges.
Worked Example: Recovering Membership at a Threshold
A hypothetical portfolio analysis produces this merge sequence for seven claim-service categories:
| Merge | Clusters joined | Height |
|---|---|---|
| 1 | A and B | 0.5 |
| 2 | C and D | 1.0 |
| 3 | AB and E | 1.5 |
| 4 | CD and F | 2.0 |
| 5 | ABE and G | 3.0 |
| 6 | ABEG and CDF | 4.5 |
At \(\tau=2.4\), determine the clusters, the number of clusters, and what changes if the threshold is raised to 3.0.
At 2.4, the accepted heights are 0.5, 1.0, 1.5, and 2.0. Apply them in order: \(A\) and \(B\) form \(AB\); \(C\) and \(D\) form \(CD\); \(AB\) joins \(E\) to form \(ABE\); and \(CD\) joins \(F\) to form \(CDF\). \(G\) has not yet joined any of these groups. Since \(n=7\) and four merges are accepted,
The clusters are \(ABE\), \(CDF\), and \(G\). At \(\tau=3.0\), the merge \(ABEG\) is also accepted, because its height equals the threshold. There are now five accepted merges, so \(K(3.0)=7-5=2\), and the clusters are \(ABEG\) and \(CDF\). The merge at 4.5 remains above either threshold. Check: at 2.4, the cluster sizes are 3, 3, and 1; at 3.0, they are 4 and 3. Both sets of sizes sum to seven.
Common mistake avoided: The threshold 2.4 does not mean “make 2.4 clusters” or “perform approximately two merges.” It is a dissimilarity level; the merge heights determine the number and membership of the resulting groups.
Intervals of Constant Cluster Count
The merge heights divide the threshold axis into intervals. Within an interval that contains no merge height, the accepted merges are unchanged, so both membership and count remain unchanged. At each distinct merge height, the threshold convention determines when that level's merge events enter the clustering.
Worked Example: Finding the Threshold Range for Four Clusters
A hypothetical analysis of eight operational units has merge heights \(0.4,\ 0.7,\ 1.1,\ 1.1,\ 1.8,\ 2.6,\ 5.0\), in agglomeration order. Using the convention that merges with height at most \(\tau\) are included, find the smallest threshold giving exactly four clusters and the range of thresholds giving at least four clusters.
There are \(n=8\) units. Below 0.4, no merges are accepted, so there are eight clusters. At 0.4 and below 0.7, one merge is accepted and the count is seven. At 0.7 and below 1.1, two are accepted and the count is six. At 1.1, both tied merges are included, making four accepted merges in total:
The next merge height is 1.8. For \(1.1 \leq \tau < 1.8\), the count remains four. At \(\tau=1.8\), the fifth merge is included and the count becomes \(8-5=3\). Therefore, the smallest threshold giving exactly four clusters is 1.1, and the thresholds giving at least four clusters are \(0 \leq \tau < 1.8\), assuming nonnegative dissimilarities. Check: the two merges at 1.1 lower the count from six to four, while the merge at 1.8 lowers it from four to three.
Common mistake avoided: Treating the two tied heights of 1.1 as one merge would incorrectly give five clusters at that threshold. A tie in height does not combine separate merge events into one event.
Common Traps and Practical Interpretation
A threshold is meaningful only relative to the dissimilarity scale used to build the dendrogram. If the distance definition or feature scaling changes, the numerical merge heights can change too; a threshold chosen for one scale should not automatically be transferred to another. This makes threshold cuts useful for setting an interpretable separation rule, but not a scale-free choice.
- Mixing threshold and target count. A threshold is a height on the dissimilarity axis. It does not directly specify how many clusters there will be.
- Ignoring equality. Decide whether a merge at exactly \(\tau\) is included. Under the convention here, it is included.
- Counting heights instead of merge events. Equal heights may correspond to multiple merges and can produce a multi-cluster drop at one threshold.
- Reading only the number of groups. Trace the branches to establish membership, then use the merge-count formula as a check.
- Assuming one threshold is universally appropriate. A cut reflects a chosen dissimilarity tolerance. Its practical value depends on whether that tolerance is suitable for the data and application.
For an actuarial application, a hypothetical analyst might cluster product categories using a dissimilarity based on standardized claim features. A threshold could represent the largest dissimilarity at which categories are treated as belonging to one group. The resulting partition is a descriptive grouping, not evidence that categories have identical risk or should receive identical prices; that interpretation requires additional analysis.
The key operational facts are simple: apply every merge at or below the threshold, count each accepted merge once, and recover memberships by tracing the included branches. As the threshold rises, the count is a stepwise nonincreasing function; its jumps occur at merge heights, and tied heights can create larger jumps. These principles are aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.
Check Your Understanding
For each problem, use the convention that a merge at or below the threshold is included.
- A dendrogram has \(n=5\) observations and merge heights \(0.6,\ 1.2,\ 2.0,\ 3.5\). How many clusters remain at \(\tau=1.5\), and how many at \(\tau=2.0\)? (Answer: 3 clusters at 1.5; 2 clusters at 2.0.)
- Six observations have merge heights \(0.3,\ 0.9,\ 0.9,\ 1.7,\ 3.2\). How many clusters are present just below 0.9, at 0.9, and just below 1.7? (Answer: 5, 3, and 3 clusters, respectively.)
- A schedule begins with leaves \(J,K,L,M,N,P\) and merges \(J,K\) at 0.4; \(L,M\) at 0.8; \(JK,N\) at 1.3; \(LM,P\) at 1.6; and \(JKN,LMP\) at 2.7. At \(\tau=1.5\), list the clusters and verify the count. (Answer: \(JKN\), \(LM\), and \(P\); 3 clusters.)
- Seven observations have merge heights \(0.5,\ 1.0,\ 1.0,\ 1.4,\ 2.2,\ 4.0\). What is the smallest threshold giving exactly three clusters, and what is the largest threshold that still gives at least three clusters? (Answer: The smallest threshold is 1.4; the largest is any threshold below 2.2, with no largest attained value.)
- A dendrogram has nine leaves. At a threshold, four merge events have heights below it, two more have heights exactly equal to it, and the remaining heights are above it. Under the stated convention, how many clusters result, and by how much would the count be overstated if the two equal-height merges were incorrectly excluded? (Answer: 3 clusters; excluding them would give 5, an overstatement of 2 clusters.)