Influential Samples In Subtype Discovery

Why Look for Influential Samples?

A subtype solution can look stable overall while depending heavily on one or two samples. A sample near a subgroup boundary may help determine where that boundary falls; an unusual sample may pull a cluster centre or alter the structure of several groups. Removing such a sample and refitting asks a focused question: how much does the fitted partition of the other samples change when this sample is absent?

This is a sensitivity diagnostic, not a test of whether a sample is “good” or “bad.” A highly influential sample may be an assay outlier, a sample with a recording or preprocessing problem, a genuine rare biological case, or a valid observation near a boundary. The partition change identifies where to investigate; it does not identify the cause by itself.

A Direct Pairwise Influence Score

Cluster labels such as “1” and “2” are arbitrary: exchanging their names does not change a partition. A direct way to compare two partitions is therefore to compare whether each pair of samples is together or apart. For sample \(i\), consider all unordered pairs \(j,k\) among the retained samples. A pair is a disagreement if the full-data fit and the leave-one-out fit disagree about whether its members share a cluster.

Here \(\mathbf{1}(\cdot)\) equals 1 when its condition is true and 0 otherwise. The denominator is the number of pairs among the retained samples. Thus \(D_i\) is the proportion of those pairs whose co-membership status changed: 0 means no pair changed, and 1 means every pair changed. It is not a probability or a p-value, and no universal cutoff makes a value “large”; its interpretation depends on the application, subgroup sizes, and comparison with the other samples’ scores.

The pairwise form is invariant to cluster-label switching and can still be calculated if the number of clusters changes after removal. If the analysis requires a fixed number of clusters, that choice should be held to the pre-specified pipeline. If the number is re-selected, the resulting score reflects both altered assignments and the change in the selection procedure; report that choice explicitly.

An adjusted Rand index (ARI) can provide a complementary summary of agreement between the two partitions. It adjusts pair-count agreement for chance under its standard fixed-margin reference model; values near 1 indicate strong agreement, values near 0 indicate agreement near that reference, and negative values indicate less agreement than expected under it. The ARI and \(D_i\) answer related but not identical questions: \(D_i\) is a direct fraction of changed pair relationships, whereas ARI adjusts agreement for chance and the partition margins. Neither should be treated as a significance test for sample influence.

How to Run and Interpret Leave-One-Out Refits

The comparison is meaningful only when the full fit and each leave-one-out fit implement the intended analysis consistently. In particular, state the features, preprocessing, clustering procedure, and rule for choosing the number of clusters. When a preprocessing step estimates parameters from the observed samples, apply the pipeline as specified for a refit; in particular, refit data-dependent preprocessing on the retained samples rather than silently carrying over estimates from the full data. Keep other design choices consistent. If clustering uses random initialization, use a deliberate, documented strategy for initializations so that a high score is not simply an unstable run.

A global score can conceal where change occurred. For a high-\(D_i\) sample, inspect the changed pairs and the affected clusters. Did one subgroup split, did two groups merge, or did a handful of samples switch together? Cluster-wise Jaccard comparisons or a co-clustering summary can help localize changes; they complement rather than replace the sample-level leave-one-out comparison.

Localizing the Effect and Looking for Causes

Once an influential sample is flagged, examine the actual changes rather than relying on a rank alone. A sample can have a high score because it changes one important subgroup boundary, or because it causes broad rearrangement. These situations may call for different investigations. Useful questions include whether the sample has unusual missingness or assay quality, whether its features were transformed or scaled as intended, and whether its cohort, batch, or clinical context differs from most samples. Such checks generate plausible explanations; they do not make the explanation causal.

A practical comparison is with a single-sample deletion versus a broader resampling analysis. Leave-one-out refits isolate the sensitivity associated with each specific omitted sample, but they assess only those one-at-a-time deletions. Repeated train-test perturbations or feature perturbations assess different kinds of changes to the analysis and may reveal fragility that no single deletion exposes. Conversely, broad perturbations do not identify which particular sample was influential. These approaches answer complementary questions.

Diagnostics, Limitations, and a Common Pitfall

Check whether the result is interpretable before acting on it. The pairwise score is undefined when fewer than two samples remain, so it requires \(n\geq3\). With very few samples, each pair carries substantial weight and scores can change in large increments. A low global score can also conceal an important change in a small subgroup; inspect subgroup membership and not just the aggregate. If cluster labels change but the pair relationships do not, \(D_i=0\), as it should.

A high score may also arise because the clustering procedure is sensitive to its random initialization or because several analysis choices changed at once. Check that the refits followed the intended pipeline and that the reported comparison is based on the shared samples. The leave-one-out analysis is conditional on the chosen data set and pipeline: it does not quantify all forms of uncertainty, establish reproducibility in another cohort, or show that a subtype is biologically meaningful.

• Common mistake: “Sample M has the largest \(D_i\), so it is an outlier and should be removed.”
• Correction: The score says that deleting M changes many pair relationships under this pipeline. Review its quality, context, and biological plausibility, and report the sensitivity. Exclude a sample only under justified, pre-specified criteria—not solely because its removal makes the clustering look more stable.

The central comparison is with other samples’ scores and with the scientific consequences of the affected changes. A moderately high score that alters a clinically important subgroup may deserve more attention than a larger score involving only inconsequential boundary rearrangements. Conversely, a sample that appears unusual on one feature can be an informative subtype rather than a defect.