Why Rare Populations Need a Different MNN Interpretation
Mutual nearest neighbors (MNNs) provide cross-batch cell pairs when each cell is among the other cell’s nearest neighbors in the chosen feature space. For common states, many cells may support local cross-batch matches. A rare state may have few cells in one batch, none in another, or too few neighbors to produce reciprocal pairs. Consequently, “no MNNs for this state” does not by itself show that alignment failed. It may reflect sampling, a genuinely batch-specific state, or a geometry in which corresponding cells are not close enough to become mutual neighbors.
The distinction matters because an integration procedure should correct technical differences among shared states, not force every state in every batch to have a counterpart. For a rare population, indiscriminate matching can create false evidence of shared biology and distort its local structure. The central task is therefore to assess what the available cells support, while keeping the shared-state assumption and the chosen feature space in view.
Sampling Limits Before Neighborhood Matching
Suppose a population has true proportion \(p\) in the source population represented by a batch, and \(n\) cells are sampled independently with the same probability of belonging to that population. Let \(R\) be the number of sampled cells from the population. Under this binomial sampling model, \(R\sim\operatorname{Binomial}(n,p)\), so the probability of seeing no cells is
The assumptions are consequential: the sampling is treated as independent, each sampled cell has the same population probability \(p\), and \(n\) is the number of cells available for the analysis. Cell capture, quality filtering, or subsampling can make the effective number of cells smaller than the number originally collected. If inclusion probabilities vary by state, this simple binomial calculation is only an approximation.
A useful first check is the expected count \(E(R)=np\), alongside the probability of zero. An expected count below one signals that absence from a sample is unsurprising, but even a larger expected count does not guarantee enough cells to form useful local neighborhoods. Conversely, observing no cells is not a statistical demonstration that the population’s prevalence is zero.
Worked Example: Could Sampling Alone Explain No Rare Cells?
Suppose a state makes up \(0.8\%\) of cells in a hypothetical source population. A batch contributes \(n=150\) independently sampled cells to the analysis. Calculate the expected count and the probability that none belong to the state.
Here \(p=0.008\). The expected count is
The probability of observing zero is
Thus, even if the state is present at the stated prevalence, the model gives about a \(30.0\%\) chance of seeing no such cells in this batch. The probability of sampling at least one is \(1-0.2997=0.7003\), or about \(70.0\%\). This calculation addresses sampling only; it does not include additional losses from capture or filtering unless those are already reflected in the effective prevalence and sample size.
The derivation is direct: \(R=0\) means that all \(n\) sampled cells are not from the population. Each cell has probability \(1-p\) of not belonging to it. Under independent sampling, the probability that all \(n\) do not belong is the product of those probabilities, \((1-p)^n\). If \(p\) is uncertain, this is a conditional calculation at the assumed value of \(p\), not an estimate of the probability that the state is absent.
Unequal Representation and Reciprocal Neighbors
Even when both batches contain rare cells, unequal counts can limit the number and pattern of MNN pairs. With a one-nearest-neighbor rule, for example, several cells in a well-represented batch may choose the same rare cell in a sparsely represented batch. Only the cell that the sparse-batch cell itself chooses can form a reciprocal pair. With a larger neighborhood size, more pairs may be possible, but neighborhoods can also include cells from nearby states. No fixed number of cells or neighborhood size guarantees valid recovery in every geometry.
This is a property of the matching rule and the observed point cloud, not a biological statement about how many cells “should” match. The MNN procedure does not establish that a state is shared merely because some pair is reciprocal, nor does lack of a pair establish that it is batch-specific. Evaluate whether the paired cells plausibly represent the same state, using independent biological information where available, and inspect the local geometry rather than interpreting the pair count alone.
Worked Example: Unequal Counts Leave Some Cells Unmatched
Consider a hypothetical one-dimensional feature space and use one nearest neighbor in the other batch. Batch A has rare-state cells at positions \(0.00\), \(0.04\), and \(0.08\). Batch B has one rare-state cell at \(0.03\). Assume other cells are far enough away not to be nearest neighbors for these rare cells.
The Batch B cell at \(0.03\) is nearest to the Batch A cell at \(0.04\), at distance \(|0.03-0.04|=0.01\). The Batch A cell at \(0.04\) is nearest to the Batch B cell, also at distance \(0.01\), so this pair is mutual.
The Batch A cell at \(0.00\) also has the Batch B cell as its nearest neighbor, at distance \(0.03\), but the Batch B cell’s nearest neighbor is the Batch A cell at \(0.04\), not the one at \(0.00\). That pair is not mutual. Similarly, the Batch A cell at \(0.08\) is \(0.05\) from the Batch B cell, but the Batch B cell still chooses the cell at \(0.04\); this pair is not mutual either.
The result is one MNN pair and two unmatched Batch A rare cells, despite rare cells being present in both batches. This is consistent with limited representation under the stated rule. It is not, by itself, evidence that either unmatched cell is a different biological state or that the alignment failed.
Distinguishing an Unmatched State From Failed Alignment
A valid unmatched state is a state for which the available evidence does not support a cross-batch counterpart. This can occur because a state is genuinely condition-specific, because it was not sampled in one batch, or because the sample contains too few cells to establish reliable local matches. In these cases, leaving cells unmatched can be more defensible than forcing them toward a different state.
A potential alignment failure is a different concern: there is good reason to expect the same state in both batches, but the representation or correction does not bring the corresponding cells into plausible cross-batch neighborhoods. Relevant evidence might include independent state annotations or markers that support correspondence, adequate numbers of cells in both batches, and a systematic separation of those cells in the feature space used for matching. These observations strengthen a diagnosis but do not create a universal numerical test. The meaning of distance depends on the feature space and data preprocessing.
A practical diagnostic sequence is to ask: (1) Were enough cells from the state sampled in each batch to make local matching plausible? (2) Are the proposed pairs biologically credible, rather than merely close in a space where different states overlap? (3) If corresponding states are independently supported, are their cross-batch distances and local neighborhoods comparable to those of well-represented shared states? (4) Does the finding persist under reasonable analysis choices? These questions help distinguish limited evidence from a systematic mismatch.
Worked Example: Interpreting a Suspected Alignment Failure
In a hypothetical dataset, independent marker review supports the presence of the same rare state in two batches, with 24 cells in Batch A and 21 in Batch B. In the selected feature space, suppose the median nearest cross-batch distance for those state cells is \(1.80\) units. For a well-represented shared state in the same analysis, the corresponding median is \(0.30\) units. The median within-batch nearest-neighbor distance for the rare state is \(0.20\) units in Batch A and \(0.25\) units in Batch B.
The cross-batch distance is \(1.80/0.30=6\) times the comparison-state distance, while the rare state’s own within-batch neighborhoods are much tighter: \(0.20\) and \(0.25\) units. The substantial separation, combined with adequate counts and independent evidence that the state occurs in both batches, is a warning that these cells are not being aligned as expected. It is stronger evidence of a representation or alignment problem than a simple absence of MNNs in a batch with zero observed cells.
These ratios are descriptive, not a formal threshold. Distances can differ across feature spaces, and a truly shared state may still have biological variation between batches. A next step is to inspect the selected features, local neighbors, and batch composition, then assess whether the pattern remains under a reasonable alternative representation or sampling sensitivity check. The calculation alone does not establish the cause of the separation.
Checks, Alternatives, and Limitations
Rare-state assessment benefits from more than one view of the data. Useful checks include reporting the state’s observed count and proportion in each batch, calculating the zero-detection probability under clearly stated sampling assumptions when appropriate, inspecting local cross-batch distances, and checking whether candidate pairs agree with independent biological annotations. For unequal sampling, a downsampling sensitivity analysis can show whether the result depends on the larger batch’s abundance. Downsampling does not add information to the smaller batch, however, and should not be presented as proof that the remaining matches are correct.
The main alternative to forcing MNNs is to preserve the state as unmatched pending further evidence. That is appropriate when the state may be genuinely unique to a batch or the observed counts are inadequate. If the target is known to be shared and both batches have adequate representation, investigate the feature space and alignment rather than treating unmatched status as the final answer. Cluster-level summaries can help describe abundance and presence, but grouping cells does not itself demonstrate that two clusters are the same state. No single diagnostic resolves biological absence, sampling failure, and alignment failure in every dataset.
The interpretation also depends on assumptions established for MNN alignment: comparisons must be made in a feature space that meaningfully represents cell similarity, and the correction relies on sufficiently comparable shared states and local geometry. If batch and biological condition are confounded, observed differences may not be identifiable as technical rather than biological; a correction cannot recover information the design does not provide. Rare cells make this limitation especially visible because a handful of observations can determine the available local matches.
Common Pitfall
- Wrong reasoning: “There are no MNN pairs for this rare state in Batch B, so integration failed and the cells must be shifted toward the nearest cells in Batch A.”
- Correction: First establish whether the state was observed in Batch B and whether the counts support local matching. If the state is absent or too sparse, no-pair status may be valid. If independent evidence supports a shared state and both batches have adequate counts, investigate the geometry and representation before deciding that alignment failed. Do not create matches solely to make every batch look complete.
Check Your Understanding
Answer each question, then compare your reasoning with the concise solutions below.
- A state has prevalence \(0.01\), and 100 cells are independently sampled. What is the probability of observing none?
- Both batches contain cells from a rare state, but one-nearest-neighbor matching gives only one MNN because one batch has a single rare cell. Does this prove that the other rare cells are a different state?
- Name two kinds of evidence that would make “alignment failure” more plausible than “insufficient sampling.”
- Why is downsampling the larger batch not proof that the rare state has been aligned correctly?
Solutions. (1) \(P(R=0)=(1-0.01)^{100}=0.99^{100}\approx0.3660\), so the model gives about a \(36.6\%\) chance of no observed cells. (2) No. Reciprocal matching can be limited by unequal representation; the unmatched cells require biological and geometric assessment. (3) Examples include independent evidence that the state is shared, adequate counts in both batches, and systematically large cross-batch distances relative to relevant shared-state comparisons. (4) Downsampling tests sensitivity to imbalance in the larger batch but cannot increase information in the smaller batch or establish that candidate pairs represent the same biology.