Metrics for Batch Mixing

Why Mixing Must Be Measured Within Cell States

An integration can place cells from different batches near one another. That is useful when the cells represent the same biological state and batch is a technical nuisance. But a global measure of batch diversity can also look favorable when cells from distinct biological states have been placed together. A mixing score alone does not tell these situations apart.

The target here is narrower: quantify whether cells from different batches are locally intermingled within a specified cell state. The state labels must be supplied independently of the neighborhood calculation, such as from a trusted annotation or a reference. The metric then asks whether nearby cells of that same state have a batch composition resembling the composition expected for that state.

The eligible batches and their expected proportions should be defined separately for each state. Write \(n_{sb}\) for the number of cells of state \(s\) in batch \(b\), and let \(q_{sb}=n_{sb}/\sum_b n_{sb}\). The eligible batches are those with \(n_{sb}>0\). If only one batch contains cells in state \(s\), cross-batch mixing for that state cannot be assessed: there is no within-state batch overlap to evaluate. That is not evidence that integration succeeded or failed.

For a state-restricted neighborhood, let \(n_{ib}\) be the number of its \(k\) neighbors from batch \(b\), and \(p_{ib}=n_{ib}/k\). The vector \(p_i\) is the observed local batch composition; \(q_s\) is the state-specific overall composition used as a reference. These are distinct quantities. In particular, equal proportions are not automatically the right target when batches contribute unequal numbers of cells to the state.

Entropy and Neighbor-Based Scores

One common summary is Shannon entropy of the neighborhood’s batch proportions. With natural logarithms, \(H_i=-\sum_b p_{ib}\log p_{ib}\), where terms with \(p_{ib}=0\) contribute zero. If \(B_s\) eligible batches occur in state \(s\), normalized entropy divides by the maximum possible entropy, \(\log B_s\). This puts the score between zero and one when \(B_s>1\): zero means all neighbors are from one batch; one means the neighborhood has equal proportions across all eligible batches.

That interpretation has an important qualification. Equal batch proportions are the maximum-entropy configuration, but they need not match the state’s actual batch composition \(q_s\). If one batch contributed many more cells of that state, a perfectly uniform neighborhood may not be the appropriate benchmark. Therefore, report the state-specific batch proportions and, when the intended target is the observed state composition, assess how closely local composition matches \(q_s\) as well as reporting entropy.

Another neighbor-based summary is the inverse-Simpson effective number of batches, sometimes used as a local inverse-diversity measure:

It equals one if all neighbors are from one batch and reaches \(B_s\) when the neighborhood is evenly divided among \(B_s\) batches. It can be read as the effective number of equally represented batches that would give the observed concentration. Like entropy, it is a diversity measure, not a direct test that the local proportions match \(q_s\). A useful reference for unequal batch representation is \(1/\sum_b q_{sb}^2\).

A further option is to compare local batch counts to \(q_s\) using a goodness-of-fit statistic. For example, a Pearson statistic is \(\sum_b (n_{ib}-kq_{sb})^2/(kq_{sb})\). Under a multinomial sampling model with fixed reference proportions and sufficiently large expected counts, its approximate null distribution is chi-squared with \(B_s-1\) degrees of freedom. Neighbor sets overlap, however, and the reference proportions are usually estimated from the data, so these local p-values are not independent evidence and should not be treated as if each were an independent test. Small expected counts also make the chi-squared approximation unreliable.

Why Pooled Mixing Can Be Misleading

Consider hypothetical data with two biological states and two batches. State 1 has five cells from batch A and one from batch B; state 2 has one cell from batch A and five from batch B. The pooled data contain six cells from each batch. A global batch-composition summary therefore sees a 50:50 split, even though batch and state are strongly associated.

Now suppose the representation collapses the two states so that a focal state-1 cell has three state-1 batch-A cells and three state-2 batch-B cells among its six unrestricted neighbors. The unrestricted batch proportions are 50:50, giving maximal two-batch entropy. Yet all six neighbors are not comparable biological units: half are from another state. A state-restricted neighborhood might contain only batch-A cells near that focal cell, revealing little local cross-batch mixing. Since batch B does occur in state 1 overall, its absence from that local neighborhood indicates a local discrepancy, not a lack of eligibility.

This example shows why a global score can reward a biologically incorrect alignment: the algorithm can increase batch diversity by bringing unlike states together. It also shows why the state restriction is part of the metric’s definition, not a cosmetic adjustment. The metric must use only same-state neighbors if its interpretation is within-state mixing.

Derivation, Aggregation, and Diagnostics

The entropy upper bound explains why normalized entropy lies between zero and one. Let \(u_b=1/B_s\) be the uniform distribution over the \(B_s\) eligible batches. Nonnegativity of Kullback–Leibler divergence gives

Therefore \(H_i\leq \log B_s\). Equality holds when \(p_i=u\), and the minimum \(H_i=0\) occurs when all neighbors belong to one batch. This justifies the normalized entropy \(H_i/\log B_s\) for \(B_s>1\), but does not make uniformity the right reference under unequal batch representation.

To summarize many focal cells, first calculate each cell’s score using its state-restricted neighborhood, then average within state. A pooled summary across states should be explicitly weighted: an unweighted mean across cells gives more influence to large states, while an unweighted mean of state means gives each state equal influence. Neither is universally correct. State the weighting rule and report state-specific results when the states differ substantially in size or batch composition.

Useful diagnostics include checking the number of eligible batches per state, the \(n_{sb}\) values, the distribution of neighborhood sizes and scores, and whether conclusions change across reasonable choices of \(k\). If \(k\) is too small, local proportions are coarse and can vary sharply with one cell. If \(k\) is too large, neighborhoods may cross state substructure even though they are restricted to a broad state label. Verify that every neighbor actually has the focal cell’s state label; an implementation that merely calculates a global nearest-neighbor graph and then summarizes its labels has not necessarily calculated a within-state metric.

Entropy and inverse-Simpson scores summarize concentration differently. Entropy is sensitive to the full distribution of proportions; inverse-Simpson diversity gives more weight to common batches. A reference-composition discrepancy such as \(D_i\) asks a different question again. These methods are complementary rather than interchangeable. A chi-squared goodness-of-fit approach supplies a calibrated test only under its sampling and count assumptions; it is not a direct measure of effect size, and overlapping neighborhoods complicate collections of such tests.

Limits and Common Pitfalls

• Using unrestricted neighbors. Wrong reasoning: any diverse set of nearby batch labels demonstrates within-state mixing. Correction: restrict candidate neighbors to the focal cell’s known state, or describe the result only as unrestricted local batch diversity.
• Treating maximum entropy as the only target. Wrong reasoning: equal batch proportions are always ideal. Correction: compare local proportions with the state-specific reference \(q_s\) when batch contributions are unequal.
• Scoring a state represented in one batch. With \(B_s=1\), normalized entropy has a zero denominator and cross-batch mixing is not assessable. Do not assign a perfect score.
• Interpreting a high score as biological validation. Batch-mixing metrics measure batch diversity or batch-composition agreement. They cannot establish that distinct states remain biologically distinct or that the representation is otherwise correct.
• Ignoring neighborhood and annotation choices. The score depends on the representation, distance, \(k\), state labels, and included cells. Broad or uncertain labels can conceal substructure; a single score cannot remove that dependence.

The practical conclusion is to report the metric’s definition, the state restriction, the eligible batches and their reference proportions, and the aggregation rule. A high within-state mixing score is evidence about local batch composition under those choices, not a general certificate of integration quality.