Diagnosing Multimodal Neighborhoods

Why Diagnose a Joint Neighborhood?

A weighted-nearest-neighbor (WNN) graph connects each cell to nearby cells using information from more than one modality. Its joint neighbors are useful only if the graph reflects a sensible combination of those modalities for the cells being studied. A graph can look plausible overall while particular cells have neighborhoods that differ sharply from their RNA, protein, or accessibility neighborhoods. Alternatively, the joint neighborhood may be almost entirely the one favored by a single assay. These are diagnostic signals to investigate, not automatic evidence that the graph is wrong.

This tutorial treats neighborhood assessment as a comparison among three kinds of evidence: overlap with each modality-specific neighbor set, consistency of labels within a joint neighborhood when suitable labels are available, and the distribution of cell-specific modality weights. The introductory material on weighted nearest neighbors establishes the joint-graph setting; here the focus is how to evaluate its local behavior.

Definition: For cell $i$, let $N_i^{(m)}$ be its set of $k$ nearest neighbors in modality $m$, and let $N_i^{(J)}$ be its set of $k$ neighbors in the joint WNN graph. The cell $i$ itself is excluded from each set. The same cells and the same $k$ are used for the comparisons below.

The comparisons assume that neighbor identities refer to the same cells across modalities, that the modality-specific and joint neighborhoods were constructed with documented choices of representation and distance, and that the graph is not being interpreted beyond the data and preprocessing used to construct it. A fixed $k$ makes set comparisons straightforward; with different neighborhood sizes, the choice of comparison measure and its denominator would need to be stated explicitly. These diagnostics do not require modalities to be statistically independent, and they do not establish that one modality is biologically more valid than another.

Overlap With Modality-Specific Neighbors

A direct audit asks how much of the joint neighborhood is also close to cell $i$ in each modality. One symmetric measure is the Jaccard similarity: the number of shared neighbors divided by the number of distinct neighbors in the two sets. It ranges from 0 (no shared neighbors) to 1 (identical sets). Because both sets contain $k$ cells, the Jaccard denominator is $2k$ minus the number shared.

Definition: The Jaccard overlap between the joint neighborhood and modality $m$ for cell $i$ is $$ J_i^{(m)}= \frac{|N_i^{(J)}\cap N_i^{(m)}|} {|N_i^{(J)}\cup N_i^{(m)}|}. $$ When both sets have size $k$, if $r=|N_i^{(J)}\cap N_i^{(m)}|$, then $J_i^{(m)}=r/(2k-r)$.

The denominator follows because the union contains the $k$ cells in the joint set plus the $k-r$ modality-specific neighbors not already in it, for a total of $2k-r$. This makes Jaccard overlap relatively conservative: even a moderate number of shared neighbors is divided by the full set of distinct neighbors. An alternative is the overlap coefficient, $r/\min(|N_i^{(J)}|,|N_i^{(m)}|)$, which equals $r/k$ for equal-sized sets. It is larger or equal to Jaccard in this setting and answers a different question: what fraction of the smaller set is shared? Choose and report a measure rather than treating these values as interchangeable.

Worked Example: Comparing Three Neighborhoods

Suppose a hypothetical cell $i$ has $k=5$ neighbors in each of three sets: $N_i^{(\mathrm{RNA})}=\{a,b,c,d,e\}$, $N_i^{(\mathrm{ATAC})}=\{a,b,f,g,h\}$, and $N_i^{(J)}=\{a,b,c,f,g\}$.

For RNA, the intersection is $\{a,b,c\}$, so $r=3$. The union has $5+5-3=7$ cells. Thus $$ J_i^{(\mathrm{RNA})}=\frac{3}{7}\approx 0.4286. $$

For ATAC, the intersection is $\{a,b,f,g\}$, so $r=4$, and the union has $5+5-4=6$ cells. Therefore $$ J_i^{(\mathrm{ATAC})}=\frac{4}{6}\approx 0.6667. $$

The joint set shares more of its members with the ATAC set than with the RNA set by this measure. This is a local description of these sets, not a test that ATAC is the correct assay or that RNA has failed. Other cells may show the opposite pattern.

Across cells, inspect the distribution of $J_i^{(m)}$ for each modality: its median, spread, and lower tail can reveal whether the joint graph regularly departs from one modality's neighborhoods or does so only for a subset of cells. A low value is not intrinsically bad. It may reflect useful integration, modality-specific biology, a sparse or noisy representation, or a local technical issue. The pattern is most informative when interpreted alongside the weight and label diagnostics below.

Local Label Consistency

If labels are available from a reliable source independent of the graph construction, they can summarize the composition of a neighborhood. For categorical labels $y_i$, define the local consistency of cell $i$ as the proportion of its neighbors whose label matches its own. This is a descriptive purity measure, not a probability that the label is correct. Labels may be broad, uncertain, or reflect a different annotation scheme from the variation represented in the graph.

Definition: Given a categorical label $y_i$ for each cell, joint-neighborhood label consistency is $$ C_i^{(J)}= \frac{1}{k}\sum_{j\in N_i^{(J)}}\mathbf{1}(y_j=y_i), $$ where $\mathbf{1}(\cdot)$ equals 1 when its condition is true and 0 otherwise. The same calculation can be made for each modality-specific neighborhood.

A low value can indicate a mixed neighborhood, a boundary between annotated groups, a mislabeled cell, a label that is too coarse or too fine for the local structure, or an embedding that does not preserve the distinctions captured by the labels. A high value is not proof of a good representation: labels can be biased, or the graph can separate groups that should not be separated for the scientific question. Avoid using labels both to construct a graph and then as if they were independent validation.

Worked Example: Interpreting Local Label Consistency

Suppose a hypothetical cell labeled A has six joint neighbors: four labeled A, one labeled B, and one labeled C. Its RNA-specific neighborhood has five A labels and one B label. Its ATAC-specific neighborhood has two A labels, two B labels, and two C labels.

For the joint neighborhood, four of six labels match the cell: $$ C_i^{(J)}=\frac{4}{6}\approx 0.6667. $$ For RNA the corresponding value is $5/6\approx 0.8333$, and for ATAC it is $2/6\approx 0.3333$. The joint neighborhood is less label-consistent than the RNA neighborhood and more consistent than the ATAC neighborhood for this cell.

This comparison does not establish that the joint graph is inferior to RNA alone. It says that, relative to the supplied labels, the local composition is intermediate in this example. Whether that is desirable depends on the scientific role of the labels and the intended analysis.

Inspecting Cell-Specific Modality Weights

WNN assigns cell-specific weights to modalities. For $M$ modalities, write $w_i^{(m)}$ for the weight of modality $m$ at cell $i$, with nonnegative weights that sum to one. A concentrated weight distribution means that one modality receives most of the relative weight for that cell. This is evidence about the graph construction's relative weighting, not a direct measurement of biological importance or an assurance that the favored assay is accurate.

Definition: For weights $w_i^{(1)},\ldots,w_i^{(M)}$ satisfying $w_i^{(m)}\geq 0$ and $\sum_{m=1}^M w_i^{(m)}=1$, the weight entropy is $$ H_i=-\sum_{m=1}^M w_i^{(m)}\log w_i^{(m)}, $$ with $0\log 0$ defined as 0. Entropy is 0 when all weight is on one modality and is largest, $\log M$, when weights are equal. The largest individual weight, $\max_m w_i^{(m)}$, is a simpler concentration summary.

Entropy complements the maximum weight: the maximum identifies the leading modality, while entropy describes the distribution across all modalities. For comparisons across different numbers of modalities, normalized entropy $H_i/\log M$ can be used when $M>1$; it ranges from 0 to 1. Neither entropy nor a particular cutoff has a universal pass/fail threshold. Across cells, report distributions or the fraction exceeding a clearly stated descriptive threshold, and examine whether extreme values cluster by cell type, library, or other known technical features.

Worked Example: Finding a Highly Concentrated Weight Profile

For a hypothetical cell measured in three modalities, suppose the WNN weights are 0.94 for RNA, 0.04 for protein, and 0.02 for ATAC. They are nonnegative and sum to one. The maximum weight is 0.94, so RNA receives nearly all of the relative weight for this cell.

Using natural logarithms, the entropy is $$ \begin{aligned} H_i &=-\{0.94\log(0.94)+0.04\log(0.04)+0.02\log(0.02)\}\\ &\approx 0.2652. \end{aligned} $$ The maximum possible entropy for three modalities is $\log(3)\approx 1.0986$, so normalized entropy is $0.26515837/1.09861229\approx 0.2414$. This relatively concentrated profile is a reason to inspect this cell's modality-specific and joint neighbors. It does not, by itself, show that the joint graph is defective or that RNA is biologically dominant.

Combining Diagnostics and Finding Failure Modes

The most useful assessment is a joint reading of the measures rather than a single threshold. For example, a cell with a very large RNA weight, high overlap between its joint and RNA neighborhoods, low overlap with other modalities, and high agreement with an independently obtained RNA-relevant label may have a joint neighborhood that is effectively RNA-driven. That may be appropriate if other modalities are uninformative for that cell, or concerning if the analysis requires meaningful contributions from all assays. Conversely, a large weight without corresponding neighborhood overlap should prompt a check of the graph construction, distance representations, and whether the displayed weights refer to the same graph being evaluated.

Fix the comparison.
Record the modalities, neighbor count $k$, distance or representation choices, and the exact graph from which each neighbor set was obtained.

Measure local agreement.
Calculate $J_i^{(m)}$ for each cell and modality; compare distributions and inspect cells with especially low or asymmetric overlap.

Check labels and weights.
Where suitable labels exist, calculate consistency in joint and modality-specific neighborhoods. Summarize maximum weight and entropy for the same cells.

Investigate patterns, not isolated flags.
Review flagged cells in context and ask whether the pattern is biologically plausible, technically associated, or sensitive to analysis choices. Do not convert a descriptive diagnostic into an unsupported significance claim.

Several failure modes can make these diagnostics misleading. First, mismatched cell identifiers can create meaningless overlaps; confirm that each set refers to the same cells. Second, comparisons with different $k$ values can change both overlap and label consistency; hold $k$ fixed or report the precise alternative normalization. Third, rare labels and uncertain annotations can make consistency low even for a coherent continuum. Fourth, small $k$ makes each neighbor count influential, while large $k$ can blur local boundaries. Repeating the summaries at a few scientifically reasonable values of $k$ is a sensitivity check, not a search for the value that produces the preferred result.

Compare these local diagnostics with alternatives in terms of what they answer. A global graph-mixing summary compresses behavior across many cells and can hide a small set of assay-dominated neighborhoods. A two-dimensional embedding can reveal broad structure but may distort distances and does not replace calculations on the graph itself. Modality-specific graphs provide a useful reference, but their disagreements may reflect real modality-specific information rather than error. The diagnostics here are therefore complementary: overlap describes shared neighbor identities, label consistency describes local composition relative to chosen labels, and weights describe relative modality contributions in the WNN construction.

Common Pitfall and Limitations

A common mistake is to reason, “The RNA weight is 0.94, so this cell's joint neighbors must be wrong because the other assays were ignored.” The weight indicates strong relative emphasis, but it does not prove that other assays contain useful information for that cell, nor does it establish an error. Check whether the joint neighbors actually overlap with RNA neighbors, whether the pattern recurs across cells, and whether it aligns with technical or biological context. Similarly, high label consistency should not be treated as proof of correctness, especially when labels informed the analysis.

Key takeaway: A potentially assay-dominated neighborhood is best identified by converging evidence: a concentrated modality-weight profile, strong joint-neighbor overlap with that modality, weak overlap with alternatives, and a contextually relevant label pattern when independent labels are available. These are diagnostic descriptions, not universal tests or causal conclusions.

These summaries also inherit limitations from the representations, distance metrics, graph construction, and labels used to compute them. A neighbor set is a local summary, and no single $k$ captures every scale of structure. Weight magnitudes are relative within the fitted multimodal procedure; they are not directly comparable measures of biological effect across datasets unless the procedures and preprocessing are comparable. Report the choices, distributions, and representative cell-level patterns so readers can judge the evidence.

Check Your Understanding

Answer each question, then compare with the solutions.

Two size-$6$ neighbor sets share three cells. What is their Jaccard overlap?
A cell has five matching labels among eight joint neighbors. What is its local label consistency, and what does that value not establish?
For weights 0.50, 0.30, and 0.20, what is the largest weight? Is the weight profile concentrated entirely on one modality?
A cell has a 0.95 RNA weight and a joint neighborhood that overlaps strongly with its RNA-specific neighborhood. What is a justified conclusion, and what is not justified?

Solutions. (1) The union has $6+6-3=9$ cells, so Jaccard overlap is $3/9=0.3333$. (2) Consistency is $5/8=0.6250$. It describes the fraction of neighbors matching the cell's label; it does not establish that the label is correct or that the graph is valid. (3) The largest weight is 0.50. The remaining weights are positive, so weight is distributed across modalities rather than entirely assigned to one. (4) The joint neighborhood is strongly aligned with RNA for this cell, and the weight profile is concentrated on RNA. This does not prove that the graph is wrong, that RNA is biologically more important, or that other modalities contain no useful information.