Testing Overcorrection With Known Biological Contrasts

Why Test Known Biological Contrasts?

Integration is intended to reduce unwanted technical differences without erasing biological structure. A known condition or lineage contrast provides a direct check on that trade-off: specify the biological difference before correction, then compare its direction and separation in the adjusted values. This complements negative controls, which ask whether a contrast that should be absent remains absent. Neither check alone establishes that an integration method is universally reliable.

The comparison is meaningful only when the contrast is estimable in the design and is defined consistently before and after integration. For example, if condition and batch are perfectly confounded, the data cannot distinguish a condition effect from a batch effect. A correction that removes the apparent condition difference in that setting is not, by itself, evidence that it overcorrected biology; the target was not identifiable from the observed design.

A contrast can be represented by a single signed effect, such as a difference in average expression for one gene, or by a vector of effects across genes or other prespecified features. In either case, compare the same biological quantity on both sides of integration. If effects are standardized, use the same standardization reference before and after correction; recomputing scales separately can change the apparent size of the contrast.

Direction and Separation Are Different Questions

For a vector contrast, direction can be summarized by the cosine similarity. It is the dot product divided by the product of the vector lengths. A value near 1 indicates similar directions, a value near 0 indicates near-orthogonality, and a value below 0 indicates an obtuse angle and some directional disagreement; more negative values indicate stronger opposition. The measure is undefined if either vector has zero length, and can be unstable when either length is very small.

Direction alone does not measure how much separation remains. Multiplying every post-integration effect by a small positive constant preserves direction but shrinks the contrast. A simple magnitude summary is the retention ratio \(r=\|\boldsymbol{d}^{\,\mathrm{post}}\|/\|\boldsymbol{d}^{\,\mathrm{pre}}\|\), provided the pre-integration norm is clearly nonzero. A ratio near 1 indicates similar norm, a ratio near 0 indicates strong contraction, and a ratio above 1 indicates increased norm. This is a descriptive comparison, not a causal estimate of integration’s effect.

For a scalar effect, preserve its sign as well as its absolute size. If the pre-integration contrast is positive, a negative post-integration contrast signals a reversal even if the absolute effect is large. If the baseline effect is close to zero, a ratio of magnitudes can become arbitrarily large or unstable, so report the two signed effects directly rather than relying on the ratio.

Replicate-Level Retention and a Log-Scale Test

For inferential assessment, calculate a contrast and its retention ratio within each independent biological replicate when the design supports that comparison. Let \(r_b\) be the magnitude retention ratio for replicate \(b\), for \(b=1,\ldots,n\), and define \(\ell_b=\log(r_b)\). These ratios must be positive and computed using a baseline contrast sufficiently far from zero. The biological replicate, not each cell, is the unit of replication. Cells from the same donor, animal, or sample are not independent biological replicates merely because they are separate rows in a single-cell matrix.

For a one-sided one-sample \(t\) test, suppose the replicate log ratios are independent and arise from an approximately normal distribution with mean \(\mu_\ell\) and finite variance. For testing \(H_0:\mu_\ell=0\) against \(H_a:\mu_\ell<0\), the statistic is \(t=\bar{\ell}/(s_\ell/\sqrt{n})\), with \(df=n-1\), where \(s_\ell\) is the sample standard deviation of the log ratios. A small lower-tail p-value is evidence against the null mean of zero in the direction of contraction; it is not the probability that the null hypothesis is true. The threshold should be chosen for the biological application: testing against zero asks whether geometric mean retention is below 1, whereas a threshold such as 0.8 asks whether it is below 80% retention.

This statistic compares the mean log retention with the log threshold. Its \(t\) reference distribution requires independent replicate-level observations and normally distributed log ratios under the null; with few replicates, departures from normality or a single influential replicate can matter substantially. If there is strong dependence among replicates or the log ratios are highly skewed, the usual one-sample \(t\) calibration may not be reliable. Report the replicate values and uncertainty, and do not treat a nonsignificant result as proof that the contrast was preserved.

Diagnostics, Alternatives, and Limitations

Before interpreting a low retention estimate as overcorrection, check whether the contrast is genuinely known and estimable, whether its orientation and feature set match across stages, and whether the pre-integration contrast is large enough for a ratio to be stable. Plot or tabulate replicate-level signed effects and ratios. Look for a single replicate dominating the mean, large variation in directions, and ratios driven by near-zero denominators. A vector average can also conceal opposing feature-level changes, so include feature-level summaries when the scientific question concerns particular genes.

Direction and magnitude summaries are complementary but not interchangeable. Cosine similarity is insensitive to uniform positive rescaling, while norm retention is sensitive to it but can hide a direction change. Gene-by-gene signed comparisons are more interpretable for specific features, though they create many separate summaries and may be noisy. A global conservation score can condense evidence across features, but can mask a biologically important contrast if its weighting or feature set is inappropriate. A direct known-contrast assessment is most informative when its definition is prespecified and its biological relevance is independently justified.

These checks have important limits. Integration may alter feature-level values in ways that make a simple norm an imperfect measure of biological separation. Benchmark results depend on the chosen features, scaling, contrast, replicates, and threshold; they do not certify performance in every cell state or dataset. In particular, a well-preserved common contrast does not guarantee preservation of a rare state. Finally, if uncertainty is large, the right conclusion may be inconclusive rather than “preserved” or “overcorrected.”