Why Filter Candidate Anchors Locally?
An anchor pair is evidence that two cells from different batches may represent the same biological state. But a pair can appear plausible in a shared latent representation and still be a poor match. If such a pair is retained, its cross-batch expression difference can contribute an inappropriate correction for the cells around it. Neighborhood-consistency filtering asks a more local question before correction: do other candidate anchors connect nearby cells in the two batches?
This is a filter on a set of candidate pairs, not a test that establishes biological identity. The introductory material on anchor pairs and shared latent spaces motivates using a common representation to find candidates. Here the focus is a defined, auditable rule for removing candidates whose local cross-batch support is weak. Neighborhood consistency is useful because a genuine shared state often has nearby cells in both batches that also have plausible cross-batch matches. A false pair may instead sit alone, or its nearby cells may point to a different region in the other batch.
A Two-Sided Neighborhood-Consistency Rule
For a candidate pair \((a,b)\), first identify which neighboring cells participate in any candidate anchor. Define \(L_A(a)\) as the distinct cells in \(N_A(a)\) that occur as the A endpoint of at least one pair in \(C\). Define \(L_B(b)\) as the distinct cells in \(N_B(b)\) that occur as the B endpoint of at least one pair in \(C\). A local anchor link is a pair in \(C\) whose A endpoint belongs to \(L_A(a)\) and whose B endpoint belongs to \(L_B(b)\).
Now count how many distinct local A endpoints and local B endpoints are represented among those local links. Divide each count by the number of eligible neighboring endpoints on that side. This gives a coverage proportion in each direction. The smaller proportion is the pair's two-sided consistency value: a match cannot be considered well supported if the local links cover one side but largely miss the other. The candidate pair \((a,b)\) itself is not counted as its own support; the neighborhoods contain other cells.
The fractions are defined only when both local endpoint sets are nonempty. In practice, require a prespecified minimum number \(m\) of eligible endpoints on each side, as well as a minimum consistency threshold \(\tau\). Retain a candidate only when \(|L_A(a)|\ge m\), \(|L_B(b)|\ge m\), and \(Q(a,b)\ge\tau\). Otherwise remove it or mark it as unsupported for a separate review. The choices of \(k\), \(m\), and \(\tau\) are analysis settings, not universal constants; report them and examine whether conclusions change under reasonable alternatives.
This rule is deliberately a gate: it decides whether a candidate has enough local support to proceed. The value \(Q\) is not a probability that the anchor is correct, and the threshold is not a significance level. It is also not a final correction weight. Its interpretation depends on the candidate-anchor set, the representation, the neighborhood size, and the density of cells in the region.
Why the Two-Sided Rule Works
The local-geometry rationale is that cells from a shared state tend to have related neighborhoods in a representation that captures the relevant shared structure. If nearby A cells have candidate links to nearby B cells, and those links cover a reasonable fraction of eligible endpoints in both batches, the proposed correspondence has support beyond its own pair. The filter uses that pattern to screen candidates before expression correction.
The minimum in \(Q=\min(q_A,q_B)\) matters. Suppose nearly every local A endpoint is linked into the B neighborhood, but only one of many B endpoints is represented. A one-sided fraction could look reassuring, even though the local cross-batch correspondence is asymmetric. Taking the smaller coverage prevents the larger fraction from concealing the weaker side. It does not require every nearby cell to have an anchor: the eligible sets include only neighboring cells that appear in the candidate-anchor set.
This is a structural justification, not a guarantee. A coherent but incorrect region-to-region mapping can also have high coverage. Conversely, a true pair in a rare or sparsely sampled state may have too few neighboring anchors to pass. Neighborhood agreement is evidence to retain a candidate under the chosen rule, not proof of equivalence or a substitute for inspecting biological and geometric diagnostics.
Worked Example: Removing a Locally Unsupported Pair
Suppose a hypothetical analysis uses \(k=4\), \(m=2\), and \(\tau=0.50\). For candidate pair \((a_1,b_1)\), the eligible neighboring endpoints are \(L_A(a_1)=\{a_2,a_3,a_4\}\) and \(L_B(b_1)=\{b_2,b_3,b_4,b_5\}\). Among candidate anchors connecting these sets, the links are \((a_2,b_2)\), \((a_3,b_2)\), and \((a_4,b_3)\).
On the A side, all three eligible cells have at least one local link, so \(q_A=3/3=1.00\). On the B side, only \(b_2\) and \(b_3\) have links; \(b_4\) and \(b_5\) do not. Thus \(q_B=2/4=0.500\), and \(Q=\min(1.00,0.500)=0.500\). Both endpoint sets meet the minimum size, and the consistency equals the threshold. The rule therefore retains \((a_1,b_1)\).
Now suppose a second candidate \((a_5,b_5)\) has the same number of eligible endpoints, but its local links cover one of three A endpoints and one of four B endpoints. Then \(q_A=1/3=0.333\), \(q_B=1/4=0.250\), and \(Q=0.250\). Since \(0.250<0.50\), this candidate is removed by the stated rule. These calculations describe filtering only; they do not establish whether either pair is biologically correct.
Worked Examples With Different Local Structures
Worked Example: A Candidate With One-Sided Support
In a second hypothetical data set, a candidate pair has \(L_A(a)=\{a_1,a_2,a_3,a_4\}\) and \(L_B(b)=\{b_1,b_2,b_3,b_4\}\). The local candidate links are \((a_1,b_1)\), \((a_2,b_1)\), and \((a_3,b_1)\). Let \(m=2\) and \(\tau=0.40\).
Three of the four A endpoints are represented, so \(q_A=3/4=0.750\). Only one of the four B endpoints is represented, so \(q_B=1/4=0.250\). Therefore \(Q=\min(0.750,0.250)=0.250\). Although the A-side proportion is high, the two-sided value is below \(0.40\). The candidate is removed. The example shows why counting links or checking only one direction can hide a mismatch: several A cells all point toward the same B cell, leaving most local B endpoints unsupported.
Worked Example: Sparse Support in a Rare State
Consider a hypothetical rare population in which a candidate pair has two eligible neighboring endpoints in each batch. Only one local link connects the neighborhoods. Thus \(q_A=1/2=0.500\), \(q_B=1/2=0.500\), and \(Q=0.500\). With \(m=2\) and \(\tau=0.50\), it passes exactly.
If the same analysis instead required \(m=3\), the pair would fail the minimum-support condition because each eligible set contains only two cells, even though its observed consistency is \(0.500\). That failure does not show that the pair is false; it says that the chosen rule demands more local evidence than this sparse region supplies. A defensible analysis should distinguish “removed for low consistency” from “not evaluated because support was insufficient,” and should check whether a larger neighborhood or a separate rare-state strategy changes the result.
Implementation, Diagnostics, and Failure Modes
A careful application makes the filtering decision reproducible. Choose the representation and distance metric before examining individual anchor outcomes. Define within-batch neighborhoods in that representation, set \(k\), \(m\), and \(\tau\), then calculate the local endpoint sets and two coverage proportions for each candidate. Remove unsupported pairs before using the retained anchor set for expression correction. Keep a record of the candidates removed, their neighborhood sizes, and their two directional coverage values.
Inspect the result by batch, region of the latent space, and known or suspected cell population. A filter that removes nearly all candidates from one batch or a rare region may be responding to uneven sampling rather than to false matches. Plotting retained and removed candidates against the shared representation can reveal isolated pairs, boundary effects, or a systematic pattern of one-sided support. Check sensitivity to nearby choices of \(k\), \(m\), and \(\tau\): if a small change reverses many decisions, the filtering outcome is fragile and should not be presented as definitive.
- Too small a neighborhood: local evidence can be sparse and unstable. Genuine pairs may fail simply because few neighboring cells have candidate anchors.
- Too large a neighborhood: neighborhoods can cross cell-state boundaries. Unrelated states may then provide apparent support, weakening the filter's ability to remove false pairs.
- Unequal sampling density: one batch may have many more cells or candidate endpoints in a region. A coverage proportion can then reflect sampling and candidate generation as well as biological correspondence.
- Correlated support: multiple links can share the same cells or depend on the same local candidate structure. They are not independent confirmations of an anchor.
- Systematic wrong matches: an incorrect mapping between two coherent regions can be locally consistent and pass. External biological checks and broader integration diagnostics remain important.
The geometry used for filtering also matters. If neighborhoods are calculated in a representation that poorly preserves the relevant cell-state structure, consistency values can be misleading. If batches contain condition-specific populations, forcing local overlap can suppress valid distinctions. The rule should therefore be understood relative to the integration target and shared-state assumptions: it screens candidate correspondences where shared states are expected, rather than requiring all cells to have counterparts.
How This Filter Differs From Alternatives
| Approach | What it checks | Main limitation |
|---|---|---|
| No neighborhood filter | Retains every candidate produced by the anchor-identification step. | Locally isolated false candidates can contribute to later correction. |
| One-sided neighbor check | Checks whether nearby cells on one batch side have candidate links. | Can pass when links concentrate on one endpoint or fail to cover the other side. |
| Two-sided neighborhood consistency | Checks local candidate-link coverage on both sides and applies a threshold. | Depends on neighborhood and threshold choices; coherent errors can still pass. |
The two-sided rule is more informative than a simple count of nearby links because it accounts for the number of eligible endpoints on each side. It is also intentionally simpler than assigning a calibrated probability of correctness: \(Q\) is a coverage summary under a chosen construction, not a probability model. Later decisions about how to compare or weight retained anchors are separate from this binary screening step.
Common Mistake
Another mistake is to treat a failed filter as evidence that the cells cannot share a biological state. Failure can arise from sparse sampling, a poor representation, or settings that are too strict. The precise conclusion is narrower: under the specified neighborhood, candidate set, and thresholds, the pair lacks the required local support.
Check Your Understanding
Answer each question, then compare with the solutions.
- A candidate has \(q_A=0.80\) and \(q_B=0.35\). If \(\tau=0.40\) and both endpoint sets satisfy the minimum size, is it retained? Why?
- With \(m=3\), a candidate has two eligible endpoints on each side and \(Q=0.90\). What does the rule decide, and what does that decision mean?
- Why can a one-sided check pass a locally imbalanced match that the two-sided rule rejects?
- Name two reasons a genuine pair in a rare population could fail this filter.
Solutions. 1. No. \(Q=\min(0.80,0.35)=0.35<0.40\), so it fails the consistency threshold. 2. It fails the minimum-support condition because \(2<3\) on both sides; this means support is insufficient under the chosen rule, not that the pair is known to be false. 3. One side can have high coverage while most endpoints on the other side have no local links; the minimum of the directional proportions exposes that imbalance. 4. The population may be sparsely sampled, leaving too few eligible neighboring endpoints, or its local candidate links may be too few or unstable at the chosen neighborhood size. A representation that does not preserve that population's structure is another possible cause.