Rare Subtypes And Small-Cluster Instability

Why Rare Subtypes Are Especially Unstable

A subtype represented by only a few samples can look highly reproducible in one summary and fragile in another. For instance, a cluster may have a clear profile among its members, yet one or two samples determine whether it appears at all in a resampled data set. This is not merely a technical nuisance: disappearance in resamples may reflect limited support for estimating that subgroup, rather than evidence that the biological state does not exist.

The earlier material on bootstrap assessment of subgroup stability and cluster-wise Jaccard stability provides general tools for comparing partitions. Here the emphasis is on the special behavior of small groups: how often they are omitted, how wide prevalence uncertainty can be, and what a zero count does and does not establish.

Definition: A rare subtype is a subgroup with low prevalence in the population or target cohort. A small cluster is a subgroup represented by few observations in the analyzed sample. Rarity is a population property; smallness is a sample property. A rare subtype may be well represented in a very large sample, while a moderately prevalent subtype may be small in a small study.

For a first analysis, suppose each sampled unit either belongs or does not belong to a specified subtype. Let \(p\) denote the subtype prevalence in the target population, \(n\) the number of independently sampled units, and \(X\) the number assigned to the subtype. Under independent sampling and a fixed subtype definition, \(X\sim\operatorname{Binomial}(n,p)\). These assumptions provide a useful reference model, not a guarantee that the output of a data-driven clustering pipeline is binomial.

How Often Does a Rare Group Disappear?

Consider a nonparametric bootstrap that draws \(n\) observations with replacement from the observed sample. If \(k\) of the \(n\) observations belong to a fixed observed subgroup, each bootstrap draw selects a subgroup member with probability \(k/n\). The bootstrap count \(X^*\) therefore has a binomial distribution conditional on the observed data. In particular, the chance that no member of this subgroup appears in a bootstrap sample is:

X^* \mid \text{observed data} \sim \operatorname{Binomial}\left(n,\frac{k}{n}\right), \qquad P(X^*=0\mid \text{observed data})=\left(1-\frac{k}{n}\right)^n.

This calculation concerns omission from a resample, conditional on the original data and its labels. It does not estimate the probability that the subtype is absent from the population. The expected bootstrap count is \(k\), but its standard deviation is \(\sqrt{k(1-k/n)}\), so count fluctuations are large relative to the mean when \(k\) is small.

Worked Example: A Subgroup With Two Members

A hypothetical data set contains \(n=100\) independent samples, of which \(k=2\) are assigned to a particular subtype. In a bootstrap resample of size 100, what is the probability that this observed subtype is omitted entirely?

Each draw has probability \(2/100=0.02\) of selecting one of the two observed members. Thus the number \(X^*\) of subtype members in the bootstrap resample follows \(\operatorname{Binomial}(100,0.02)\), conditional on the observed data. The probability of drawing no such member is:

P(X^*=0\mid\text{observed data}) =(1-0.02)^{100} =0.98^{100} \approx 0.1326.

So about 13.3% of these bootstrap resamples would contain no member of the observed subgroup. The expected count is \(100(0.02)=2\), and the standard deviation is \(\sqrt{100(0.02)(0.98)}=1.400\) samples. A zero count is therefore not surprising for this tiny group. It signals limited resampling support, not proof that the subtype is nonexistent.

The same logic helps interpret stability summaries. A small cluster may have low resampling stability because its few members are frequently omitted, because their replacement draws change the cluster fit, or both. The omission calculation isolates the first mechanism under fixed labels. If the clustering algorithm is refit in each resample, the result also depends on how the cluster is defined and identified across runs; the simple binomial formula alone does not capture that additional variation.

Quantifying Uncertainty in Prevalence

The observed proportion is \(\hat{p}=X/n\). When the subtype labels are fixed and observations are independent with a common population prevalence \(p\), uncertainty about \(p\) can be summarized by a binomial confidence interval. For small counts, the Wilson interval is generally preferable to a basic normal interval, which can have poor coverage and may produce limits outside the interval from zero to one.

Definition: For \(X=x\) successes among \(n\) independent Bernoulli observations, let \(\hat{p}=x/n\). The two-sided Wilson interval with standard-normal quantile \(z\) is:

\frac{ \hat{p}+\frac{z^2}{2n} \ \pm\ z\sqrt{\frac{\hat{p}(1-\hat{p})}{n}+\frac{z^2}{4n^2}} }{ 1+\frac{z^2}{n} }.

For a nominal 95% interval, use \(z=1.96\). Under the stated independent binomial model, the Wilson procedure has approximately 95% nominal long-run coverage; its actual coverage can differ from 95% for particular values of \(p\) and \(n\).

The interval is about prevalence under the sampling model. It is not an interval for the stability score of a learned cluster, and it does not include uncertainty from choosing features, preprocessing, selecting the number of clusters, or assigning uncertain labels unless those steps are explicitly incorporated into the analysis.

Worked Example: Uncertainty After Two Subtype Observations

In a hypothetical independently sampled cohort of \(n=60\) units, \(x=2\) are assigned to a specified subtype. Calculate the 95% Wilson interval for its prevalence, treating the subtype definition and assignments as fixed.

The observed proportion is \(\hat{p}=2/60=0.03333\). With \(z=1.96\), \(z^2=3.8416\), and the denominator is:

1+\frac{3.8416}{60}=1.06403.

The interval center is \((0.03333+3.8416/120)/1.06403=0.06141\). Its half-width is:

\frac{1.96\sqrt{\frac{0.03333(1-0.03333)}{60} +\frac{3.8416}{4(60)^2}}}{1.06403} \approx 0.05222.

Therefore the interval is \(0.06141\pm0.05222\), or approximately \((0.00919,0.1136)\). Under the assumed model, these data are compatible with prevalence values from about 0.92% to 11.36%. The interval is wide because only two members were observed; reporting the point estimate of 3.33% alone would conceal that uncertainty. This is an approximate nominal 95% interval, not a statement that there is a 95% probability that the fixed \(p\) lies in this particular interval.

What Does a Zero Count Mean?

If no members are observed, it is tempting to conclude that the subtype is absent. A more informative question is: if its prevalence were \(p\), how likely would a sample of this size be to contain no members? Under independent Bernoulli sampling:

P(X=0\mid p)=(1-p)^n.

When this probability is not small, a zero count is compatible with a nonzero prevalence. Conversely, a sufficiently large sample can make a zero count unlikely under prevalence values of practical concern. Solving \((1-p_U)^n=0.05\) gives a one-sided 95% upper bound \(p_U=1-0.05^{1/n}\) after observing zero members. This bound depends on the binomial assumptions and is not evidence that prevalence equals zero.

Worked Example: No Observed Members Is Not Proof of Absence

Suppose a hypothetical study samples \(n=60\) independent units and observes no members of a subtype. First assess the probability of this result if the true prevalence were 3%. Then calculate the one-sided 95% upper bound based on the zero count.

Under \(p=0.03\), the probability of observing zero members is:

P(X=0\mid p=0.03) =(1-0.03)^{60} =0.97^{60} \approx 0.1608.

Thus, even if prevalence were 3%, a sample of this size would miss the subtype about 16.1% of the time. The zero count is not particularly surprising under that prevalence. For the one-sided 95% upper bound, solve \((1-p_U)^{60}=0.05\):

p_U=1-0.05^{1/60}\approx 0.04870.

The data are compatible, at this one-sided confidence level, with prevalence as high as about 4.87% under the model. The calculation does not establish that any particular positive prevalence is true; it shows that zero detected members in 60 independent observations cannot rule out prevalence values up to that bound using this procedure.

A zero count can be stronger evidence against a subtype when the design had high probability of detecting it. That probability depends on sample size and prevalence, and may also depend on assay sensitivity, eligibility criteria, and whether the subtype can be reliably recognized. If samples are correlated, or if selection into the cohort changes subgroup prevalence, the independent binomial calculation may be misleading. The resampling unit and sampling design must match the data-generating process.

Diagnostics, Alternatives, and Limits

Useful diagnostics make the instability visible rather than compressing it into a single stability score. Report the observed subgroup size alongside any stability estimate; inspect the distribution of subgroup counts across resamples; and record the proportion of resamples in which the subgroup is absent or cannot be matched. Averages alone can hide a mixture of zero counts and occasional large counts. A subgroup that is omitted frequently should not be summarized as though every resample provided equally informative evidence about its profile.

The simple binomial prevalence interval and the bootstrap omission probability answer different questions. The first describes sampling uncertainty about \(p\) under independent sampling and fixed labels. The second describes how often a particular observed subgroup is omitted when resampling the observed data. Refitting the full clustering procedure can address broader pipeline variability, but requires consistent handling of preprocessing and cluster matching. Subsampling without replacement is another perturbation strategy; it changes the chance of retaining rare members and must be interpreted according to its sampling fraction. None of these methods can manufacture support when the observed group contains only a handful of units.

Approach	Question answered	Main caution for rare groups
Bootstrap omission probability	How often is this observed group absent in resamples?	Conditional on the observed labels; does not estimate population absence.
Wilson prevalence interval	What prevalence values are compatible with the count under a binomial model?	Approximate coverage; assumes independent observations and fixed classification.
Exact binomial methods	What bounds or tests follow from the binomial sampling model?	Can be conservative and still rely on the binomial assumptions.
Refitting the clustering pipeline	How reproducible is the data-driven subgroup under perturbation?	Cluster matching and label changes complicate interpretation; tiny groups may vanish.

Warning signs include very wide prevalence intervals, frequent zero counts across resamples, markedly different subgroup sizes across runs, and results that change greatly when one or two samples are removed. Such findings suggest inadequate support for precise characterization. They do not, by themselves, show that the subgroup is biologically meaningless or absent. Likewise, a stable count does not establish that a cluster is a genuine biological subtype; stability and biological interpretation are distinct questions.

Mistake: “The cluster disappeared in 20% of bootstrap runs, so there is a 20% probability the subtype does not exist.” Correction: That percentage describes omission under a particular resampling scheme conditional on the observed data; it is not a probability of biological existence.
Mistake: “No members were observed, so the population prevalence is zero.” Correction: State the sample size and the detection probability under relevant prevalence values, or report an appropriate upper bound under explicit assumptions.
Mistake: “The interval is narrow enough because the point estimate is small.” Correction: Precision depends on both the count and sample size. A small estimated prevalence based on very few observations often has substantial relative uncertainty.

Key takeaway: Small-cluster instability is a support problem that can be quantified, not a direct test of whether a subtype exists. Separate conditional resampling omission from population-prevalence uncertainty, state the assumptions behind each calculation, and interpret zero counts in light of the study’s ability to detect rare members.

Check Your Understanding

Answer each question using the stated sampling assumptions. Solutions follow the questions.

A sample has \(n=80\) units and a fixed observed subgroup has \(k=3\) members. What is the probability the subgroup is omitted from an ordinary bootstrap resample of size 80?
If a subtype prevalence were 3%, what is the probability of observing zero members in an independent sample of 60?
For zero observed members among 60 independent units, calculate the one-sided 95% upper prevalence bound.
What is the correct interpretation of a Wilson interval for a subtype with two observed members? Name one important assumption.

Solutions. (1) The conditional bootstrap omission probability is \((1-3/80)^{80}=(77/80)^{80}\approx0.0470\), or about 4.70%. (2) The probability is \(0.97^{60}\approx0.1608\), about 16.1%; thus a zero count is plausible even at 3% prevalence. (3) Solve \((1-p_U)^{60}=0.05\), giving \(p_U=1-0.05^{1/60}\approx0.04870\), or 4.87%. (4) The Wilson interval gives prevalence values compatible with the observed count under the binomial sampling model; its nominal 95% coverage is approximate, not exact. It assumes independent observations with a common prevalence and fixed subtype classification.