Why the Source of Instability Matters
A subtype solution can change when the observed patients change, when the selected features change, or when an analyst changes a modeling choice. These changes may look alike in a final partition, but they point to different problems. A subtype that varies across patient samples may be weakly supported by the available cohort; a subtype that changes across initializations may reflect optimization difficulty; and a subtype that appears only under one preprocessing choice may depend on how assay scales were handled.
The practical task is therefore not only to ask whether a discovered grouping is stable, but also to ask which part of the analysis makes it unstable. The earlier material on subgroup stability measures provides ways to summarize agreement. Here the emphasis is on designing comparisons that help locate the source of disagreement. A diagnostic is informative when it changes one source at a time while holding the others fixed as far as possible.
The main sources considered here are sampling variation, feature selection, preprocessing, algorithm initialization, and model specification. They can occur together. For example, a small patient sample may make feature selection erratic, and different selected features may then make a clustering algorithm converge to different local optima. A useful assessment keeps these explanations distinct before considering their interactions.
Five Sources and Their Diagnostics
| Source | What changes? | Focused diagnostic |
|---|---|---|
| Sampling variation | The patients contributing to a fit | Repeat the analysis on perturbed patient samples while keeping the analysis choices fixed; compare assignments for patients shared between fits. |
| Feature-selection variation | The variables retained for subtype discovery | Repeat or perturb feature selection under otherwise fixed conditions; record selection frequencies and compare the resulting subtype assignments. |
| Preprocessing variation | Transformations such as scaling, filtering, or normalization | Compare defensible preprocessing choices while holding the downstream model and inputs as constant as possible; inspect transformed values, distances, and assignments. |
| Initialization variation | The algorithm's starting values or random seed | Run the same specified model from multiple starts; compare objective values, convergence, and assignments. |
| Model-specification variation | Structural choices such as the number of groups, distance, or model family | Compare a small set of scientifically defensible specifications and examine which patients or relationships change. |
These diagnostics answer different questions. Sampling diagnostics ask whether the cohort supports the result. Feature diagnostics ask whether the chosen measurements are reproducible. Preprocessing diagnostics ask whether the result depends on a defensible representation of the assays. Initialization diagnostics ask whether the same optimization problem has multiple solutions that the algorithm reaches from different starts. Model-specification diagnostics ask whether the result is robust to reasonable alternative definitions of the model.
The comparisons need not be interpreted as formal tests. In many subtype analyses, there is no single probability model that assigns a universally valid p-value to “subtype stability.” A diagnostic is evidence about sensitivity under its stated perturbation, not a probability that a subtype is true. The perturbation must also be plausible: comparing a sensible normalization with a deliberately inappropriate one establishes little about robustness.
Worked Example: Separating Sampling Sensitivity From a Label Switch
Suppose a fixed clustering pipeline is fitted to a hypothetical cohort of 10 patients. The original partition labels patients 1–5 as A and 6–10 as B. After a modest patient-sample perturbation, the pipeline is evaluated on the same 10 patients and labels 1–4 and 6 as A, with 5 and 7–10 as B. All feature selection, preprocessing, model settings, and initialization are held fixed.
A simple diagnostic is the fraction of shared patients assigned to the same label, after choosing the label correspondence that gives the greatest agreement. In this example, the labels already correspond: patients 1–4 and 7–10 agree, while patients 5 and 6 do not.
Thus 80.0% of these patients retain their labels under the specified sample perturbation, while two switch. This is a descriptive diagnostic for this comparison, not a universal stability threshold. Because the analysis choices were held fixed, the observed change is consistent with sensitivity to the patient sample rather than, for example, a different feature-selection rule. Repeating such comparisons would show whether the changes recur and which patients are most affected.
The matching step matters because cluster labels such as A and B are arbitrary: exchanging the names does not change the partition. Conversely, high agreement in this single comparison does not establish population-level reproducibility. It only describes these assignments under the stated perturbation.
Worked Example: Diagnosing Feature-Selection Instability
In a hypothetical analysis, two runs use the same patient data, preprocessing, clustering method, and model settings, but repeat the feature-selection step. The first selects features \(\{a,b,c,d,e\}\); the second selects \(\{a,b,c,f,g\}\). A direct summary is the Jaccard overlap: the size of the intersection divided by the size of the union.
The intersection is \(\{a,b,c\}\), containing 3 features, and the union contains all 7 distinct features, so the overlap is about 0.429. This indicates substantial disagreement in the selected sets for these two runs. It does not show that subtype assignments must also disagree: different features can encode similar information, and selection can vary while the resulting patient grouping remains similar. The diagnostic should therefore report feature reproducibility and subtype-assignment reproducibility as separate outcomes.
If feature selection is repeated across many plausible perturbations, selection frequencies can reveal whether a few features are consistently retained or whether selection is diffuse. Frequencies depend on the perturbation design and the selection procedure; they are not automatically posterior probabilities that a feature is biologically relevant.
Worked Example: Preprocessing Can Change a Distance-Based Assignment
Consider a hypothetical patient with measurements \(x=(4,0.8)\) and two reference subtype profiles \(A=(0,0)\) and \(B=(10,1)\). A nearest-centroid rule assigns the patient to the profile with the smaller squared Euclidean distance. In the original units, the distances are:
The patient is closer to A in the original units. Now suppose the two features are scaled by standard deviations 10 and 0.2, respectively, and the patient and both reference profiles are transformed using those same scales. The scaled squared distances are:
After this scaling, the patient is closer to B. The arithmetic shows how a change in feature scale can alter the geometry used by a distance-based method and, in turn, an assignment. This toy comparison isolates the mechanism; a full analysis would refit the pipeline under each defensible preprocessing choice rather than assume that fitted profiles remain fixed. A preprocessing diagnostic should check whether the observed assignment changes are widespread and which features or patients drive them.
Worked Example: Detecting Initialization Effects
Suppose a hypothetical clustering method is run 20 times on the same data, with the same selected features, preprocessing, number of clusters, and objective, but with different starting values. Sixteen runs converge to partition P with objective value 120, and four converge to a distinct partition Q with objective value 126. Assume the objective is minimized.
The fraction of starts reaching Q is:
Thus 20.0% of these starts reach the alternative partition. Since \(120<126\), P has the better objective among the two observed solutions. This does not make Q irrelevant: its repeated appearance indicates that the algorithm can reach a distinct solution under the same specification. The diagnostic should report both solution quality and assignment differences, rather than presenting only the best run. If all starts converge to the same partition and objective, initialization sensitivity appears limited for those starts, but this does not rule out other sources of instability.
Why Controlled Comparisons Work
A useful justification for separating sources comes from the law of total variance. Let \(C\) be an indicator for whether a particular pair of patients is assigned to the same subtype, and let \(M\) denote a model specification chosen from a set of alternatives. If initialization is also random within each specification, then:
The first term summarizes variation within a fixed specification, such as variation across initializations. The second summarizes differences in the average pairwise assignment across specifications. The identity follows by writing \(C-E(C)=[C-E(C\mid M)]+[E(C\mid M)-E(C)]\), squaring, and taking expectations. The cross-product has expectation zero because \(E[C-E(C\mid M)\mid M]=0\).
This identity does not prescribe a universal numerical decomposition for every analysis: specifications may be selected deliberately rather than sampled from a probability distribution. It does explain why mixing all runs together can obscure the cause of disagreement. Holding the model fixed examines within-specification variation; deliberately changing the model examines between-specification sensitivity. Similar controlled comparisons help distinguish the other sources.
Diagnostics, Failure Modes, and Alternatives
A diagnostic is only as useful as its comparison. If sample membership, selected features, scaling, and initialization all change together, a changed subtype cannot be attributed to any one source. Likewise, using a different distance and a different number of clusters in one comparison makes it difficult to determine which choice mattered. Begin with one-factor-at-a-time contrasts to locate sensitivity, then examine interactions when there is a substantive reason to do so.
Common signs that the analysis is going wrong include highly variable assignments across small perturbations; feature sets with little overlap; subtype membership that changes after a reasonable scaling or filtering choice; repeated convergence to different solutions; and major changes under plausible alternative model specifications. These are prompts for investigation, not automatic reasons to discard the result. Check data quality, convergence, and whether alternatives are scientifically defensible before interpreting the sensitivity.
There are several complementary approaches. Controlled perturbations are easy to interpret because the varied component is explicit, but they can miss interactions. A factorial comparison varies two or more choices systematically and can reveal interactions, but requires more fits and careful interpretation. A single best-fitting model is computationally simpler, but conceals initialization and specification sensitivity. External validation can assess whether a subtype structure recurs in independent data, but differences in cohort composition or measurement can complicate that comparison. No one diagnostic answers every question.
The main limitations are that stability depends on the perturbations examined, the definition of agreement, and the resolution of the subtype solution. A coarse comparison can conceal meaningful changes within a large subtype; a detailed comparison can overemphasize minor boundary shifts. Diagnostics can also be correlated: feature selection and preprocessing often influence one another, while sampling changes can alter both. Report the analysis choices and the scope of each comparison so readers can distinguish “stable under these conditions” from a broader claim of reproducibility.
Check Your Understanding
For each question, identify the relevant source and give a suitable interpretation or diagnostic.
- A fixed pipeline gives different subtype assignments when the patient sample is perturbed, but feature selection and all model choices are held fixed. What source is being examined?
- Two selected feature sets share 4 features and contain 10 distinct features in their union. Calculate their Jaccard overlap.
- A clustering algorithm gives different assignments across random starts while the data and model specification are unchanged. Which source is implicated, and what else should be recorded?
- Assignments change when a defensible normalization is substituted, even though the downstream method is unchanged. What source is implicated, and why is that not proof that one normalization is correct?
- In one comparison, the sample, preprocessing, feature-selection rule, and number of clusters all change. Can the resulting instability be attributed to one source? Explain.
Solutions. (1) Sampling variation is being examined, because patient composition changes while analysis choices are held fixed. Compare assignments for patients shared between fits across repeated, clearly specified perturbations. (2) The Jaccard overlap is \(4/10=0.400\), so 40.0% of the features in the union are shared. This describes feature-set overlap, not necessarily subtype agreement. (3) Initialization variation is implicated. Record convergence and objective values as well as assignments, since a distinct partition may be associated with a different solution quality. (4) Preprocessing sensitivity is implicated. A changed result shows dependence on the transformation examined; it does not establish which transformation is appropriate, which requires a defensible measurement and analysis rationale. (5) No. Several sources changed together, so the observed difference cannot be uniquely attributed. Compare choices in controlled contrasts, changing one component at a time where possible.