Stability Is Evidence, Not the Objective
Choosing the number of clusters, \(K\), is not simply a search for the largest resampling-agreement score. A cluster count is useful only if its subgroups are reproducible enough for the intended purpose, sufficiently large to study, and interpretable as meaningfully distinct groups. These criteria can conflict: splitting a reproducible group may reduce agreement while revealing a useful pattern, whereas a very stable partition may consist mostly of one broad group and a tiny remainder.
The earlier material on stability profiles and partition-comparison measures provides ways to compare results across perturbations. Here the task is to use those results in a decision: examine stability across candidate values of \(K\), then assess subgroup sizes and interpretability before selecting a count. No single stability threshold or universal cluster-count rule makes that decision for every data set.
What a Cluster-Count Decision Must Balance
A defensible decision starts by defining the candidate counts, resampling design, comparison measure, and any minimum subgroup-size requirement before inspecting results. Use the same design and measure across candidate \(K\) values so that differences in the profile are interpretable. If some runs cannot be compared fairly—for example, because they do not contain the same individuals—use a comparison procedure appropriate to that design rather than treating incomparable runs as equivalent evidence.
Stability summaries often use the ARI, which is 1 for identical partitions, is adjusted for agreement expected by chance, and is typically near 0 for chance-level agreement. The ARI can be negative. Its chance adjustment does not make it a measure of subgroup utility, and it does not eliminate the influence of large clusters on partition comparisons. A mean over run pairs is a descriptive summary; the individual pairwise values also matter because a mean can conceal erratic runs.
For each candidate \(K\), inspect at least three aspects:
- Reproducibility: Is agreement consistently adequate, or is the mean hiding a wide spread or a few poor runs?
- Subgroup size: Does every subgroup meet a threshold justified by the planned use, such as a minimum number or proportion of participants? A threshold is a design choice, not a property of the ARI.
- Interpretability: Do the groups have distinguishable, coherent descriptions in the measured omics data, or does the extra split merely subdivide a group without a meaningful distinction?
Interpretability should be assessed with explicit criteria appropriate to the application, not by inventing a convenient numerical score after viewing the results. For example, a team might require that a proposed subgroup have a consistent feature profile and a clear description that can be distinguished from its neighboring group. The evidence supports an analysis choice; it does not convert an exploratory cluster into a validated clinical category.
Why Agreement Can Favor Uninformative Partitions
At very low \(K\), there are few boundaries to reproduce. If every run assigns all participants to one cluster, the assignment rule is trivially constant; some implementations define the ARI for identical one-cluster partitions as 1, while others encounter a zero-denominator case. Either way, a one-cluster solution provides no subgroup distinction. For this reason, one-cluster results should not be treated as evidence that a useful subtype structure has been discovered.
A less extreme problem occurs when a partition contains one very large cluster and a very small cluster. In a pairwise comparison, the number of participant pairs inside a cluster of size \(n-m\) is \(\binom{n-m}{2}\), where \(m\) is the number outside it. When \(m\) is small relative to \(n\), that large group contributes many pairs to the comparison. Consequently, agreement about the broad majority can dominate the impression of stability even when the small subgroup is too rare to be useful or reliable.
This is not a claim that the ARI is incorrectly calculated. It is a reminder that agreement and utility are different questions. Chance adjustment addresses a particular feature of agreement measurement; it does not impose a minimum cluster size, reward interpretable distinctions, or decide whether a proposed split is worth retaining.
A Practical Decision Sequence
Specify the candidate \(K\) values, resampling design, comparison measure, and any minimum subgroup-size requirement before using the stability profile to choose.
Compare stability across \(K\), and inspect the run-to-run spread rather than relying only on the mean. Look for a region of adequate, relatively consistent stability, not automatically the highest value.
For each plausible \(K\), examine cluster counts and proportions. Identify any cluster that violates the prespecified size requirement or is unstable despite an acceptable overall summary.
Ask whether increasing \(K\) produces a coherent and distinguishable subgroup, rather than a redundant split or an unhelpfully small group. Record the trade-off and the reason for the selected count.
This sequence is a decision framework, not a theorem that selects \(K\) mechanically. A useful report explains which candidates were considered, what stability measure and resampling design were used, how the size requirement was set, and why the chosen partition was preferable for the stated purpose.
Worked Example: Choosing a Stable Count That Also Has Useful Subgroups
Suppose a hypothetical multi-omics analysis uses the same resampling design for \(K=2,3,4\), and summarizes agreement by the mean ARI over comparable run pairs. The prespecified requirement is that each subgroup contain at least 10% of the analyzed participants. The resulting summaries and cluster proportions are: \(K=2\), mean ARI 0.91, sizes 70% and 30%; \(K=3\), mean ARI 0.84, sizes 50%, 30%, and 20%; and \(K=4\), mean ARI 0.78, sizes 48%, 28%, 20%, and 4%.
All three means describe the same type of comparison, so the profile shows a decline from \(K=2\) to \(K=3\) of \(0.91-0.84=0.07\), followed by a further decline from \(K=3\) to \(K=4\) of \(0.84-0.78=0.06\). This arithmetic identifies the stability trade-off; it does not establish that either decline is unacceptable.
The \(K=4\) candidate fails the prespecified size requirement because its smallest group is 4%, which is less than 10%. Both \(K=2\) and \(K=3\) meet the requirement. Suppose the hypothetical feature summaries show that \(K=3\) separates the broad 70% group at \(K=2\) into two coherent, distinguishable profiles, each present in multiple runs. In that case, \(K=3\) is a defensible choice: its mean ARI is lower than the \(K=2\) value, but the three-group solution remains reproducible and adds an interpretable distinction. If that split were not coherent, the higher stability at \(K=2\) would be a reason to consider \(K=2\), not proof that it is automatically best.
Worked Examples: Size and Marginal Stability Gains
Worked Example: Rejecting a Stable but Too-Small Subgroup
In a hypothetical analysis of 200 participants, the candidate \(K=2\) has a mean ARI of 0.97 and produces clusters of 194 and 6 participants. A candidate \(K=3\) has a mean ARI of 0.86 and produces clusters of 110, 55, and 35 participants. Suppose the size requirement was set in advance at 10% of the 200 participants.
For \(K=2\), the smaller cluster has \(6/200=0.03\), or 3% of participants. Since \(3\%<10\%\), this candidate does not meet the requirement, despite its high mean ARI. For \(K=3\), the proportions are \(110/200=55\%\), \(55/200=27.5\%\), and \(35/200=17.5\%\); each exceeds 10%.
Under that stated requirement, \(K=3\) is eligible and \(K=2\) is not. The conclusion is not that \(K=3\) must be selected: its interpretability and run-to-run behavior still need assessment. Rather, the example shows why a stability maximum cannot override a prespecified feasibility condition. The small \(K=2\) cluster may be an important signal to investigate, but six participants do not meet this analysis's stated minimum for treating it as a usable subgroup.
Worked Example: Choosing the Simpler of Two Similar Stability Summaries
Consider a hypothetical data set with \(K=2\) and \(K=3\) candidates. For \(K=2\), four selected run-pair ARIs are 0.82, 0.88, 0.85, and 0.85. The mean is \( (0.82+0.88+0.85+0.85)/4=3.40/4=0.85 \). For \(K=3\), the values are 0.83, 0.89, 0.84, and 0.88. The mean is \( (0.83+0.89+0.84+0.88)/4=3.44/4=0.86 \).
The difference between the means is \(0.86-0.85=0.01\). These four values are only a small illustrative set, but they make an important point: a 0.01 difference in descriptive means does not, by itself, demonstrate that \(K=3\) is meaningfully more reproducible. Suppose both candidates satisfy the same size requirement, and inspecting the extra \(K=3\) group shows no coherent distinction from its neighboring group. Then choosing \(K=2\) is reasonable: the slight increase in mean ARI at \(K=3\) does not compensate for adding an uninformative split.
This is not a formal test that the two means are equal. Nor does it establish that \(K=2\) is universally preferable. The decision depends on the purpose and the observed structure; the reported values support a measured comparison rather than a claim that the larger mean settles the choice.
Diagnostics, Alternatives, and Limits
A stability-guided decision is most credible when the candidate summaries are comparable and the conclusion is not driven by one fragile feature of the analysis. Inspect the distribution of pairwise agreement, cluster-wise stability, and cluster sizes along with the overall profile. A high overall summary paired with a consistently unstable small subgroup is not reassuring for that subgroup. Likewise, a sharp change in cluster sizes as \(K\) increases can signal that a proposed split deserves closer examination.
Check that the resampling and refitting procedure matches the question being asked. If preprocessing, feature selection, or other data-dependent steps are not repeated as required within resamples, the resulting profile may overstate reproducibility of the full analysis. Also verify that each candidate \(K\) is assessed with the same comparison rules and that the interpretation of missing or non-comparable clusters is handled consistently. A stability profile cannot repair a poorly specified perturbation design.
Other cluster-count approaches answer different questions. A silhouette-based criterion assesses how separated and compact observations are under a chosen distance; an information criterion, when tied to a specified probabilistic model, compares model fit with a complexity penalty. These can complement a stability profile, but neither directly replaces the question of whether the subgroup assignments persist under perturbation. Disagreement among criteria is a reason to explain the trade-off, not to choose whichever result supports a preferred count.
Stability also depends on the data, feature representation, distance, clustering algorithm, and perturbations. A stable result under one pipeline can change under another. The chosen \(K\) therefore remains conditional on those analysis choices, and stability alone does not establish biological meaning or clinical usefulness. Its role is narrower and valuable: to show whether the candidate partition is reproducible under a clearly described set of perturbations and to inform a transparent comparison with size and interpretability.
Common Pitfall
Wrong reasoning: “The candidate with the largest mean ARI must be the best cluster count because its agreement is highest.”
Correction: The largest mean ARI identifies the highest average agreement among the candidates under the specified design. It does not determine whether all groups meet a useful size requirement or whether a split contributes an interpretable distinction. First rule out candidates that fail prespecified requirements, then compare the remaining candidates using stability and interpretability together. If the evidence is ambiguous, report the ambiguity rather than presenting a small difference in stability as a decisive result.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A one-cluster solution is identical across every resample. What does this establish, and what does it fail to establish?
- For 150 participants, a candidate has cluster sizes 138 and 12. What percentages are these, and would the candidate pass a prespecified 10% minimum-size requirement?
- Two candidate counts have mean ARIs 0.81 and 0.82. What can and cannot be concluded from this difference alone?
- Name two checks, beyond the overall mean stability, that help assess a candidate count.
Solutions. (1) It establishes that the one-cluster assignment is reproducible under those resamples, subject to how the comparison handles a one-cluster partition. It does not establish the presence of distinct subgroups or the usefulness of a multi-subgroup result. (2) The proportions are \(138/150=92\%\) and \(12/150=8\%\). The candidate fails because \(8\%<10\%\). (3) The second candidate has a descriptive mean ARI larger by \(0.82-0.81=0.01\). That alone does not show a meaningful or decisive stability advantage, nor that its subgroups are more useful. Inspect run-to-run values, sizes, and interpretability. (4) Suitable checks include the spread of pairwise agreement, cluster-wise stability, subgroup counts and proportions, and whether each subgroup has a coherent, distinguishable interpretation.