Negative Controls for Detecting Overcorrection

Why Test for Overcorrection?

An integration can reduce visible differences between batches and still be a poor result: it may also erase differences among cell states that should remain. Batch-mixing summaries do not by themselves distinguish successful batch removal from excessive removal of structure. A useful benchmark therefore asks not only whether unwanted batch signal decreased, but also whether a signal specified independently as worth preserving is still present.

This tutorial uses three forms of evidence: negative-control genes with prespecified, reproducible biological structure; cell labels withheld from fitting and tuning; and relationships among features expected to be stable across batches. These controls test different aspects of preservation. None establishes that every biological feature was retained, and none can resolve a complete confounding of batch with biology.

A gene that is constant across all cells cannot show whether a cell-state pattern has been erased. Nor is a gene a suitable control merely because it is commonly called a housekeeping gene: its expression may vary by cell type, activation, or condition. Controls must be selected for the particular biological question and supported by information independent of the integration being assessed.

A Simple Model Clarifies the Target

Let \(Y_{ig}\) denote normalized expression for cell \(i\) and gene \(g\), \(S_i\) a cell-state indicator, and \(B_i\) a batch indicator. In a simple additive model for a gene, write

Here \(\beta_g\) is the state-associated difference and \(\gamma_g\) the batch-associated shift, conditional on the model. This is a benchmark description, not a claim that real expression is always additive. It makes the desired behavior clear: an ideal correction removes the batch component while retaining the state component. If an integration also attenuates the state component, it has removed signal the benchmark says should remain.

Suppose, for simplicity, that an integration removes the batch shift and retains a fraction \(1-\lambda\) of the state contrast. The resulting state difference is \((1-\lambda)\beta_g\). When \(\beta_g\ne 0\), the contrast-retention ratio is therefore

A ratio near 1 indicates little attenuation of this specified contrast; a ratio near 0 indicates that little of it remains. This interpretation requires that the reference contrast be estimable and that its direction and scale be comparable before and after integration. The calculation is diagnostic, not a universal test with a universal cutoff.

A key assumption is overlap: the states being compared must be represented in the relevant batches. If state A appears only in batch 1 and state B only in batch 2, state and batch cannot be separated from these data alone. Also check that normalization, gene selection, and the definition of the integrated output make the before-and-after quantities meaningfully comparable. Integrated coordinates and adjusted expression values are different outputs; a gene-level contrast cannot simply be calculated from an embedding coordinate.

Three Complementary Controls

The strongest benchmark specifies its control set and preservation target before examining results. A control can be used to quantify a particular signal, not to certify all biology. The following approaches answer related but distinct questions.

For held-out labels, “held out” means labels are not supplied to the integration algorithm or used to choose its parameters. They can be used afterwards to assess whether cells with different labels remain distinguishable in the integrated representation. If label-transfer accuracy is already being reported, this control adds a particular safeguard: prevent the evaluation labels from leaking into fitting or tuning, and examine performance only where labels and batches have adequate overlap. It does not replace that accuracy measure or make it independent if the same labels influenced model selection.

A batch-invariant relationship may be a reproducible correlation between two genes, a module relationship, or another prespecified association. The target is not necessarily that every batch has exactly the same noisy estimate. Instead, ask whether the relationship’s direction and approximate strength are retained after integration, and whether this conclusion is consistent across batches. A relationship can be lost even when each gene’s marginal mean appears acceptable.

Assumptions, Diagnostics, and Alternatives

A control-based conclusion is credible only when the control has a defensible expected behavior and the evaluation is not circular. Record how controls were chosen, which labels were withheld, the exact feature relationship, the scale used, and the comparison representation. Check whether controls were selected because they looked good after integration; that selection biases the evaluation toward apparent success. Where possible, use independent evidence or data not used to fit the integration.

Useful diagnostics include comparing results by batch rather than only after pooling; checking cell counts and label overlap; examining uncertainty from resampling cells or samples; and repeating the assessment across a prespecified panel rather than relying on one favorable gene. Cell-level resampling can understate uncertainty when cells from the same donor are dependent, so resampling should respect the experimental sampling units when those units are available. A very small reference contrast makes a retention ratio unstable, while a ratio above 1 can occur through sampling variation or amplification and should not automatically be called success.

These controls complement, rather than replace, other benchmarks. Batch-mixing measures ask whether batches are less distinguishable; a high mixing score can coexist with lost state information. Label accuracy assesses recoverability of labels, while control genes provide gene-level evidence. Known relationships add a check on coordinated structure. No single summary can establish both complete batch removal and complete biological preservation. In particular, a gene-level control does not validate an entire pathway, and a held-out label does not validate unmeasured or finer-grained cell states.

A common mistake is to argue, “The batches mix well, so integration worked.” That conclusion confuses removal of batch-associated variation with preservation of biology. The correction is to report batch removal and independent preservation controls separately, then interpret their joint evidence. Another mistake is to call any decrease in a control contrast overcorrection without checking whether the contrast was reliable before integration. A weak, noisy, or confounded reference signal cannot support a strong preservation claim.