Why Condition Batch Effects on Cell State?
A batch may contain a different mixture of cell states from another batch. If expression differs naturally by state, the batches can have different overall expression even when cells of the same state have identical expression across batches. An integration model that ignores state can mistake this compositional difference for a technical shift. Conversely, a batch effect may itself vary by state, so one common adjustment can be inadequate even after state composition is considered.
Conditioning on cell state means estimating batch-associated shifts while comparing cells at the same state-covariate value. It separates that conditional comparison from the marginal comparison of batches as actually sampled, which averages over each batch’s own state composition. These are different quantities and need not agree.
A Mixed Model Conditional on State
For a given gene, a useful model is
Here \(\mu_g\) is an intercept, and \(\boldsymbol{\phi}(z_i)\) is a specified set of state-covariate terms. For example, with two categorical states, \(\boldsymbol{\phi}\) may be a single indicator for the non-reference state. For a continuous state score, it may contain that score, possibly with additional prespecified basis terms. The fixed coefficient \(\boldsymbol{\beta}_g\) describes the average association between state and expression. The batch-specific random intercept \(u_{g,b}\) represents a batch shift at the reference value of the state terms; the random slope vector \(\boldsymbol{w}_{g,b}\) allows that shift to change with state.
A standard formulation assumes that the batch effects have mean zero and a common covariance matrix across batches, \((u_{g,b},\boldsymbol{w}_{g,b})^\mathsf{T}\sim N(\boldsymbol{0},\boldsymbol{\Sigma}_g)\), and that the residuals satisfy \(\varepsilon_{ig}\sim N(0,\sigma_g^2)\). Conditional on the covariates and batch effects, residuals are assumed independent, with constant variance in this simple model. Batch effects are independent across batches and independent of the residuals and the included state covariates. These assumptions describe a working model; transformations, repeated-measure structure, or additional covariates may be needed when the data do not support them.
The random-effects formulation is most appropriate when batches represent a set of levels for which partial pooling is useful and the number of batches supports estimating their variation. If batch is instead treated as a fixed factor, the model estimates a separate coefficient for each observed batch rather than assuming those coefficients arise from a common distribution. In either case, state terms must be included if the target comparison is conditional on state.
Why Marginal and Conditional Comparisons Differ
Write \(\boldsymbol{m}_b=E[\boldsymbol{\phi}(z_i)\mid b_i=b]\) for the average state-covariate terms in batch \(b\). Averaging the model over cells in that batch gives the following expected expression, conditional on that batch’s random effects:
This follows by linearity of expectation: the fixed state term averages to \(\boldsymbol{\beta}_g^\mathsf{T}\boldsymbol{m}_b\), and the batch-specific slope term averages to \(\boldsymbol{w}_{g,b}^\mathsf{T}\boldsymbol{m}_b\). Thus, a difference in batch averages can arise from different \(\boldsymbol{m}_b\) values even if the conditional batch effects are zero. If state slopes are present, the batch contribution to the marginal average also depends on the state composition.
Worked Example: Composition Difference Without a Batch Shift
Suppose a gene has mean expression 2 units in state 0 and 8 units in state 1, with no batch effects. Batch A contains 80% state 0 and 20% state 1; batch B contains 20% state 0 and 80% state 1. Assume the state labels are correct and expression is measured on the same scale in both batches.
The expected batch average for A is \(0.80(2)+0.20(8)=1.6+1.6=3.2\) units. For B it is \(0.20(2)+0.80(8)=0.4+6.4=6.8\) units. The marginal difference, A minus B, is \(3.2-6.8=-3.6\) units.
Within state 0, however, the expected difference is \(2-2=0\) units; within state 1 it is \(8-8=0\) units. The marginal difference is entirely due to composition, not a conditional batch shift. A model omitting state could attribute the 3.6-unit gap to batch and remove a real state-associated expression difference.
Worked Comparisons of Conditional Adjustments
Worked Example: A Batch Shift That Varies by State
Let \(z=0\) denote state 0 and \(z=1\) denote state 1. For illustration, suppose the fitted mean expression for a gene is
Assume fitted batch effects are \(u_A=0.4,\ w_A=-0.2,\ u_B=-0.1,\ w_B=0.3\). The conditional contrast A minus B at state 0 is \(u_A-u_B=0.4-(-0.1)=0.5\) units. At state 1 it is
Thus the fitted batch contrast is 0.5 units in state 0 and 0 units in state 1. A random-intercept-only model would impose one common contrast and could not represent this pattern. The calculation is conditional on the specified state and fitted effects; it does not show that batch caused the difference.
The marginal means also depend on composition. If A has 75% state 1 and B has 25% state 1, their fitted means are \(10+4(0.75)+0.4-0.2(0.75)=13.25\) units and \(10+4(0.25)-0.1+0.3(0.25)=10.975\) units, respectively. Their marginal difference is \(13.25-10.975=2.275\) units. It includes a 2-unit difference from the fixed state term, \(4(0.75-0.25)\), as well as a 0.275-unit difference from the average batch terms. Neither marginal value is the conditional contrast at a common state.
Worked Example: State Overlap and Non-identifiability
Suppose batch A contains only state 0 cells, all with mean expression 2 units, while batch B contains only state 1 cells, all with mean expression 8 units. There are no observations of state 1 in A or state 0 in B. Consider the model \(E[Y\mid z,b]=\mu+\beta z+\delta I(b=B)\).
One explanation is \(\mu=2,\ \beta=6,\ \delta=0\): expression differs by state, with no batch contrast. A second explanation is \(\mu=2,\ \beta=0,\ \delta=6\): there is no state difference, and the entire observed gap is assigned to batch. Both produce the observed means because only the combinations (A, state 0) and (B, state 1) occur. The data alone cannot choose between these explanations.
A mixed model does not resolve this lack of overlap merely by adding random effects. A conditional batch contrast requires enough observations across batches at comparable state values, or additional defensible assumptions. Otherwise, its estimate depends heavily on model structure and extrapolation beyond the observed combinations.
Model Choice, Diagnostics, and Limitations
An intercept-only batch model omits state terms and generally estimates batch differences that combine conditional shifts with composition. A conditional random-intercept model includes state but assumes the batch shift is constant across states. The random-slope model above permits the shift to vary with the included state terms. State-stratified comparisons estimate differences separately within states and can be transparent when there are few states and adequate data in every batch-state combination, but they do not automatically pool information across states and may be imprecise in sparse groups. These approaches answer different questions and rely on different assumptions.
| Approach | What it represents | Main concern |
|---|---|---|
| Batch model without state terms | Batch differences averaged over observed compositions | Can confuse state composition with batch shift |
| Conditional random-intercept model | State-adjusted batch shift constrained to be common across states | Misses genuine state-dependent batch shifts |
| Conditional random-slope model | Batch shifts that vary with included state terms | Needs enough batches and within-batch state information |
| State-stratified comparisons | Separate batch comparisons within each state | Sparse strata can yield unstable estimates |
Before interpreting a conditional model, inspect the numbers of cells in each batch-state combination and compare the ranges of continuous state covariates across batches. Large empty regions or cells represented in only one batch indicate limited overlap. Check whether estimated slopes are very uncertain, whether fitted random-effect variances are near zero or difficult to estimate, and whether residual spread changes substantially by state or batch. These patterns may indicate inadequate data support or a model that is too simple or too elaborate.
State labels can be uncertain, and errors in labels can blur state-specific differences. A state covariate derived from the same expression values being adjusted can also create circularity: the model may use a noisy or batch-affected representation of state. State terms should have a defensible biological interpretation, and sensitivity to plausible alternative state definitions should be considered. A linear slope is not sufficient when the batch shift changes nonlinearly with a continuous covariate; adding basis terms is only useful when the data support them.
Conditioning also has a scientific limitation: it deliberately compares cells at the same state, so it does not describe the total difference between batches under their observed state compositions. In addition, state and batch may be so entangled that no statistical model can separate them without assumptions or external information. For integration, the practical consequence is that a conditional adjustment can avoid removing state-composition differences as if they were technical, but it can also be unstable or misleading when overlap is weak. The estimated conditional batch terms, not an assumed universal shift, should be compared across the relevant states.
A Short Justification of the Decomposition
For two batches \(b\) and \(b'\), subtract their marginal expectations from the model. Writing \(\boldsymbol{m}_b=E[\boldsymbol{\phi}(z)\mid b]\), the difference is
The fixed intercept cancels because it is common to both batches. The second term is the contribution of different average state composition through the fixed state association. The last two terms are the average of each batch’s state-dependent random shift under its own composition. By contrast, at a common value \(z\), the fixed state terms cancel from the conditional contrast, leaving only the difference in batch-specific intercepts and slopes. This algebra explains why a marginal gap should not automatically be interpreted as a batch effect conditional on cell state.
Check Your Understanding
Answer each question using the definitions and calculations in this tutorial. Concise solutions follow the questions.
- Two batches have the same expression within each state, but different proportions of high-expression and low-expression states. Can their marginal means differ? Why?
- In a model with a state indicator \(z\), batch A has intercept 0.2 and slope 0.1; batch B has intercept \(-0.1\) and slope 0.4. Find the A-minus-B conditional contrast at \(z=0\) and \(z=1\).
- Why can a random-slope model not identify a conditional contrast if each state occurs in only one batch?
- Name one diagnostic for poor state overlap and one modeling alternative to a random-slope model.
Solutions. (1) Yes. Different proportions change the average of state-specific expression even when every within-state batch contrast is zero. (2) At \(z=0\), the contrast is \(0.2-(-0.1)=0.3\) units. At \(z=1\), it is \(0.3+(0.1-0.4)=0\) units. (3) With no batch-state combinations in common, state and batch differences can explain the observed means in multiple ways; the data do not distinguish those explanations without additional assumptions. (4) Inspect the batch-by-state cell counts or covariate ranges for missing combinations. State-stratified comparisons or a conditional random-intercept model are alternatives, each with its own assumptions and limitations.