Uncertain Membership At Subgroup Boundaries

Why Sample-Level Assignment Uncertainty Matters

A subgroup solution can appear reasonably stable overall while leaving particular samples difficult to classify. A sample may be assigned to the same subgroup in nearly every resample, or it may move repeatedly between two or more groups. Those cases are substantively different, even when a single final partition gives each sample one label. Reporting only that hard label hides how much the assignment depends on the perturbations used to assess stability.

Here the goal is narrower than evaluating whether an entire clustering pipeline is reproducible: it is to describe how consistently each sample receives each subgroup label across comparable resampled analyses. The resampling design and label alignment should be established before calculating these summaries. In particular, the same subgroup label must refer to the same aligned group across runs; otherwise a label switch can be mistaken for sample-level uncertainty.

Definition: Let $r=1,\ldots,R$ index resampled analyses and $i=1,\ldots,n$ index samples in the original data. Let $A_{ir}=1$ if sample $i$ is represented and assigned in resample $r$, and $A_{ir}=0$ otherwise. For aligned subgroup $k$, let $Z_{ir}=k$ indicate that sample $i$ is assigned to subgroup $k$ in resample $r$. The assignment frequency for sample $i$ and subgroup $k$ is $$ \hat{q}_{ik}= \frac{\sum_{r=1}^{R} I(A_{ir}=1,\ Z_{ir}=k)} {\sum_{r=1}^{R} I(A_{ir}=1)}. $$ It is defined only when the denominator $m_i=\sum_{r=1}^{R} I(A_{ir}=1)$ is positive. Frequencies are conditional on the sample being represented and assigned.

When every original sample receives an assignment in every run, $m_i=R$. In a subsampling design, or when bootstrap assignments are summarized only for samples drawn into a resample, $m_i$ can be smaller than $R$. The denominator must match the question: the frequency describes assignments among the runs in which the sample was represented, not the fraction of all runs in which it was both represented and assigned to subgroup $k$. If a procedure assigns out-of-resample samples as well, that is a different assignment rule and should be stated.

Within a run, each represented sample has one aligned assignment among the $K$ subgroups. Consequently, $\hat{q}_{ik}$ is between zero and one, and $\sum_{k=1}^{K}\hat{q}_{ik}=1$, provided the same $K$ groups are defined across the runs being summarized. These quantities describe sensitivity to the specified resampling procedure. They are not posterior probabilities that a sample truly belongs to a subgroup.

Summarizing Consistency and Ambiguity

The largest assignment frequency summarizes how often the sample receives its most frequent label. It is useful to report this frequency alongside the identity of the leading subgroup. To assess competition from another label, define the top-two margin as the largest frequency minus the second-largest frequency. A high leading frequency and a wide margin indicate a comparatively consistent assignment; a small margin indicates that the leading subgroup has a close competitor. Neither summary alone explains why a sample changes groups.

Definition: For sample $i$, order its assignment frequencies from largest to smallest as $\hat{q}_{i(1)}\geq \hat{q}_{i(2)}\geq\cdots$. The leading frequency is $\hat{q}_{i(1)}$, and the top-two margin is $\hat{q}_{i(1)}-\hat{q}_{i(2)}$. When $K\geq 2$, assignment entropy is $$ H_i=-\sum_{k=1}^{K}\hat{q}_{ik}\log(\hat{q}_{ik}), $$ where a term with $\hat{q}_{ik}=0$ is defined to be zero. Entropy is zero when one subgroup receives all assignments and is largest when the frequencies are equal across the $K$ subgroups.

The leading frequency is straightforward to interpret but can miss how the remaining assignments are distributed. For example, two samples can both have a leading frequency of 0.60, while one assigns nearly all of the remaining 0.40 to a single alternative and the other divides it among several alternatives. The margin describes the strongest competitor; entropy summarizes the spread across all labels. In practice, these summaries should complement the frequency vector rather than replace it.

Worked Example: A Consistent Assignment With Occasional Changes

Suppose a hypothetical analysis produces $R=24$ aligned resampled assignments for sample M. The sample appears in all 24 runs: 18 assignments are to subgroup 1, five to subgroup 2, and one to subgroup 3. The resampling unit and assignment rule are held fixed, and the subgroup labels have been aligned.

The frequencies are $$ \hat{q}_{M1}=\frac{18}{24}=0.750,\qquad \hat{q}_{M2}=\frac{5}{24}=0.2083,\qquad \hat{q}_{M3}=\frac{1}{24}=0.04167. $$ They sum to $1.000$, as expected. Subgroup 1 is the leading assignment, and its top-two margin is $0.750-0.2083=0.5417$.

A suitable report is: “In this hypothetical resampling analysis, sample M was assigned to subgroup 1 in 75.0% of represented runs; subgroup 2 was its next most frequent assignment, at 20.8%.” The assignments support describing M as comparatively consistent under this resampling scheme, not claiming a 75% probability that subgroup 1 is its true biological class.

Worked Example: A Sample Near an Assignment Boundary

In a second hypothetical analysis, sample N appears in 20 aligned resamples. It is assigned to subgroup 1 in nine runs, subgroup 2 in eight, and subgroup 3 in three.

The frequencies are $$ \hat{q}_{N1}=\frac{9}{20}=0.450,\qquad \hat{q}_{N2}=\frac{8}{20}=0.400,\qquad \hat{q}_{N3}=\frac{3}{20}=0.150. $$ The leading frequency is 0.450 and the top-two margin is $0.450-0.400=0.050$. The leading label alone would call N a member of subgroup 1, but nearly as many runs assign it to subgroup 2.

This is a boundary-like assignment pattern in the operational sense that resampling frequently changes the label between two groups. It does not prove that N lies near a geometric boundary in the original feature space: changes can also reflect limited sample size, perturbations to the data, or other features of the clustering procedure. Report the frequencies and describe the assignment as ambiguous under the specified resampling, rather than presenting subgroup 1 as an unqualified hard label.

What the Frequencies Estimate

For a fixed observed data set and a specified resampling mechanism, imagine drawing a new resample and applying the same aligned assignment rule. Conditional on sample $i$ being represented and assigned, let $q_{ik}$ denote the probability under that resampling mechanism that its assignment is subgroup $k$. If the resampled analyses are independent draws from this mechanism, then the indicator of assignment to subgroup $k$ has mean $q_{ik}$ and variance $q_{ik}(1-q_{ik})$. Averaging $m_i$ such indicators gives $\hat{q}_{ik}$, with $$ E(\hat{q}_{ik})=q_{ik},\qquad \operatorname{Var}(\hat{q}_{ik})=\frac{q_{ik}(1-q_{ik})}{m_i}. $$

This justification explains why a frequency is a Monte Carlo estimate of a resampling frequency, conditional on inclusion, and why finite $m_i$ makes that estimate noisy. It does not turn $q_{ik}$ into a probability of true membership: $q_{ik}$ is defined by the chosen perturbation distribution, fitting procedure, alignment, and assignment rule. If the resamples are deliberately dependent, the variance expression above need not apply.

Worked Example: Accounting for Resamples That Omit a Sample

Suppose sample P is included in 16 of 20 hypothetical subsamples. Among those 16 assignments, it receives subgroup 1 twelve times and subgroup 2 four times. The other four subsamples omit P, so they provide no assignment for P under this reporting rule.

The appropriate frequencies are $$ \hat{q}_{P1}=\frac{12}{16}=0.750,\qquad \hat{q}_{P2}=\frac{4}{16}=0.250. $$ Dividing by 20 would give 0.600 and 0.200, which sum to 0.800; those values mix assignment frequency with the chance of inclusion and do not form a conditional assignment distribution.

If the 16 included assignments are independent resampling draws conditional on inclusion, the plug-in Monte Carlo standard error for the subgroup 1 frequency is $$ \widehat{SE}(\hat{q}_{P1})= \sqrt{\frac{0.750(1-0.750)}{16}} =\sqrt{0.01172}=0.1083. $$ This is a measure of finite-resampling precision under the stated approximation, not a standard error for a biological membership probability. The small denominator also warns that the frequency estimate is relatively coarse.

Diagnostics, Alternatives, and Reporting

Before interpreting a sample-level profile, check that the calculation matches the design and that it is based on comparable assignments. Label alignment is essential: without it, a sample consistently placed in one cluster can appear to switch labels merely because cluster names were permuted. Also inspect $m_i$ for every sample. A frequency based on very few represented runs is less informative than the same value supported by many runs. If the number or meaning of subgroups changes across runs, a frequency vector over labels may not be well-defined without a justified matching rule.

The frequency profile is one stability summary, not a complete diagnosis. A small top-two margin or high entropy flags ambiguity but does not reveal its cause. Inspect which groups compete and whether the pattern is concentrated in a few samples. Co-clustering matrices summarize pairwise tendencies to appear together; adjusted Rand index, cluster-wise Jaccard stability, and variation of information summarize agreement at the partition or cluster level. Those summaries address different questions and do not replace an individual sample’s assignment profile.

Summary	Primary question	Important caution
Assignment frequencies	How often does this sample receive each aligned label?	Conditional on the specified inclusion and assignment rule
Top-two margin or entropy	How concentrated or divided is this sample’s assignment profile?	Describes ambiguity, not its cause
Co-clustering frequency	How often do two samples occur together in a group?	Does not by itself name a sample’s stable subgroup
Partition agreement measures	How similar are whole partitions or clusters across runs?	Can conceal individual boundary cases

There is no universal cutoff at which an assignment becomes “stable.” A study may prespecify a descriptive rule, such as flagging samples whose leading frequency is below a chosen threshold, but the rule should be reported as a convention, not as a general statistical law. Report the resampling design, number of resamples, inclusion denominator or denominators, label-alignment procedure, and the frequency summaries used. For flagged samples, retaining a vector of frequencies or an “uncertain” designation is more faithful to the evidence than forcing one definitive label.

Common Mistakes and Limitations

Calling a resampling frequency a probability of true membership. The frequency is conditional on a particular resampling and assignment procedure; it is not a posterior probability unless a separate probabilistic model justifies that interpretation.
Using all resamples as the denominator when some omit the sample. This confounds inclusion with assigned label. Use represented, assigned runs for the conditional frequency, and report the denominator.
Ignoring label switching. Frequencies computed before alignment can reflect arbitrary label names rather than membership changes. Align labels before aggregating assignments.
Treating a leading label as certainty. A sample assigned to subgroup 1 most often may still have a small margin and frequent assignments to subgroup 2. Report the competing frequencies.
Interpreting high ambiguity as proof of a physical boundary. Resampling instability can have several sources. The frequency profile alone does not locate a geometric boundary or establish a biological mechanism.

These summaries inherit limitations from the resampling design, fitted clustering method, and chosen alignment rule. They describe how assignments vary under the perturbations actually used; a different design can produce a different profile. Nor does a stable assignment establish that the subgroup is clinically or biologically meaningful. Stability is evidence about reproducibility under specified perturbations, not a substitute for substantive validation.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A sample is represented in 30 resamples and assigned to subgroup A in 21, B in 6, and C in 3. Calculate its assignment frequencies and top-two margin.
A sample appears in 12 of 20 resamples and is assigned to group 1 in nine of those 12. What is the conditional frequency for group 1, and why is $9/20$ not that frequency?
Two samples each have a leading frequency of 0.70. What additional summaries or information could distinguish their assignment profiles?
Explain why a frequency of 0.85 for a subgroup is not, by itself, an 85% probability that the subgroup is the sample’s true class.

Solutions. (1) The frequencies are $21/30=0.700$, $6/30=0.200$, and $3/30=0.100$; the top-two margin is $0.700-0.200=0.500$. (2) The conditional frequency is $9/12=0.750$. The quantity $9/20=0.450$ also reflects the 8 runs in which the sample was omitted, so it is not the distribution of assignments conditional on representation. (3) Compare the second-largest frequency, the top-two margin, and possibly entropy; also inspect which labels receive the remaining 0.30. (4) The value estimates assignment frequency under the specified resampling and alignment procedure, conditional on inclusion. It does not quantify probability of true membership without a separate model that supports that interpretation.