Why the Distinction Matters
An integration model can represent batch differences as separate coefficients for the batches observed in the data, or as realizations from a distribution of batch effects. These choices are not merely two computational ways to obtain the same correction. They encode different assumptions, produce different estimates for observed batches, and differ in whether they support prediction for a batch that was not used to fit the model.
The introductory material on mixed-effects models for batch integration establishes the basic mixed-model framework. Here the focus is the statistical distinction between fixed batch coefficients and random batch effects. To keep the calculations transparent, let \(y_{ij}\) be a transformed expression value or latent coordinate for cell \(i\) in batch \(j\), after conditioning on the same biological state and any other included covariates. The Gaussian model below is an illustrative working model for that scalar response; it is not a claim that every single-cell measurement is Gaussian.
Two Models and Their Assumptions
With \(J\) observed batches, a fixed-effects model with batch 1 as the reference can be written
Here \(I(j=k)\) equals 1 when cell \(i\) is in batch \(k\), and 0 otherwise. The coefficient \(\gamma_k\) is the conditional difference between batch \(k\) and reference batch 1, in the response's units, holding the included covariates fixed. The reference constraint is \(\gamma_1=0\), so all batch contrasts have a defined scale. The model requires a full-rank design: batch indicators cannot be perfectly redundant with the intercept or other covariates. For example, if each biological condition occurs in only one batch, a condition effect and a batch effect cannot be separately identified from these data alone.
The corresponding random-intercept model is
The usual assumptions are that batch effects are independent across batches, residuals are conditionally independent across cells given the batch effect and covariates, and the residuals are independent of the batch effects. The random-effects model also assumes the batch effects come from a common distribution with mean zero and variance \(\tau^2\). Conditional on the covariates, cells in the same batch are correlated because they share \(b_j\); cells in different batches do not share that random intercept.
The fixed model does not require the observed batch coefficients to be draws from a normal distribution. Its coefficients describe these particular batches. The random model instead describes variation across a population of comparable batches, so the meaning of “random” depends on whether the batches can reasonably be regarded as exchangeable with the batches of interest. Neither model removes confounding caused by a design in which biology and batch are inseparable.
Fixed Coefficients: Direct Estimates for Observed Batches
Worked Example: Estimating Fixed Batch Contrasts
Suppose a hypothetical experiment measures the same cell state in three batches. Each batch has four cells, and the mean transformed expression value is 2.0 units in batch A, 2.6 units in batch B, and 1.8 units in batch C. Assume for this calculation that the cells have the same relevant covariates and that the residuals have mean zero and equal variance across batches. With A as the reference, the estimated fixed contrasts are the differences between each batch mean and A's mean.
For batch B, \(\hat{\gamma}_B=2.6-2.0=0.6\) units. For batch C, \(\hat{\gamma}_C=1.8-2.0=-0.2\) units. On the reference scale, subtracting the estimated contrast from each non-reference observation gives B a correction of \(-0.6\) units and C a correction of \(+0.2\) units. The estimates are not shrunk toward zero: in this simple balanced example they are the observed mean differences.
This fit describes A, B, and C. It does not estimate a distribution of batch effects merely because it includes multiple batch indicators. Nor does the coefficient for B describe a biological effect: it is a conditional batch contrast under the model and design assumptions.
In a more general linear model, the fixed coefficients are estimated jointly with the covariate coefficients, rather than by subtracting raw means. The interpretation remains conditional on the included covariates. Precision depends on the design and residual variability, but the model does not automatically pull a noisy batch estimate toward a common mean.
Random Effects: Partial Pooling of Observed Batches
The random-effects model uses information across batches to estimate the distribution of batch deviations. Once \(\tau^2\) and \(\sigma^2\) have been estimated, the conditional estimate of an observed batch effect is generally a shrunken estimate, often called an empirical Bayes estimate or best linear unbiased predictor when variance components are estimated. In a balanced random-intercept model with \(n_j\) cells in batch \(j\), define \(\bar{y}_j\) as that batch's mean after accounting for covariates and let \(\hat{\mu}\) be the corresponding common mean. With variance components treated as known for the derivation, the estimate is
Because \(0\leq\lambda_j\leq1\), this estimate lies between zero and the unpooled deviation \(\bar{y}_j-\hat{\mu}\). More cells in a batch reduce \(\sigma^2/n_j\), increasing \(\lambda_j\) and making the estimate closer to the observed deviation. When between-batch variability is small relative to sampling variability, \(\lambda_j\) is smaller and estimates are pulled more strongly toward zero.
Why Shrinkage Appears
The formula follows from the normal model. Write \(d_j=\bar{y}_j-\hat{\mu}\). Conditional on \(b_j\), the batch mean deviation has sampling variance \(\sigma^2/n_j\), so \(d_j\mid b_j\) is normal with mean \(b_j\) and variance \(\sigma^2/n_j\). The prior distribution is \(b_j\sim N(0,\tau^2)\). Multiplying these normal densities gives a conditional distribution whose precision is the sum of the prior and data precisions:
Thus the conditional mean is the observed deviation weighted by its precision relative to the prior precision. This derivation assumes the common mean and variance components are known. In practice they are estimated, so the formula is used with fitted values; uncertainty in those estimated variance components is not represented by the simple plug-in calculation.
Worked Example: Shrinking the Same Observed Batch Differences
Return to the hypothetical mean deviations from A: B is \(+0.6\) units and C is \(-0.2\) units. Suppose variance-component estimates from a larger collection of comparable batches are \(\hat{\tau}^2=0.16\) square units and \(\hat{\sigma}^2=0.36\) square units. Each batch has \(n_j=4\) cells. These values are assumed for illustration, not estimated from just the three batch means above.
The shrinkage factor for each batch is
Therefore, \(\hat{b}_B=0.64(0.6)=0.384\) units and \(\hat{b}_C=0.64(-0.2)=-0.128\) units. Relative to the reference scale, the corresponding corrections are \(-0.384\) units for B and \(+0.128\) units for C. The fixed estimates were \(+0.6\) and \(-0.2\) units, respectively. The random-effects estimates are closer to zero because the model treats the observed deviations as noisy evidence about effects drawn from a zero-mean distribution.
Prediction for a New Batch
A fixed-effects model estimates coefficients for the batches represented by its indicator columns. For a new batch, there is no fitted batch-specific coefficient. Assigning it the reference level or setting its coefficient to zero is an additional prediction convention, not an estimate of that new batch's particular deviation. In contrast, the random-effects model defines a distribution for a new exchangeable batch. Before observing any cells from it, the predicted mean batch effect is zero, and its between-batch variance is \(\tau^2\). Zero is the population mean, not a claim that every new batch has no effect.
Worked Example: Updating a Prediction With New-Batch Pilot Data
Use the variance values from the preceding example: \(\tau^2=0.16\) and \(\sigma^2=0.36\), in squared response units. For a new batch with no observations, the predicted batch effect is 0 units, with standard deviation \(\sqrt{0.16}=0.4\) units. Under the normal model, an approximate 95% predictive interval for its batch effect is \(0\pm1.96(0.4)\), or \((-0.784,0.784)\) units. This interval describes uncertainty about the effect of a new batch, assuming the batch is exchangeable with those used to estimate the variance.
Now suppose four pilot cells from that batch have a mean deviation of \(+0.5\) units from the fitted biological and common-batch mean. The shrinkage factor is again \(0.64\), so the predicted batch effect becomes \(0.64(0.5)=0.32\) units. Its conditional variance, treating the variance components as known, is
The conditional standard deviation is \(\sqrt{0.0576}=0.24\) units. Thus the pilot data move the prediction from 0 toward the observed \(+0.5\), while retaining uncertainty. For a future individual cell, add the residual variance \(0.36\) to the batch-effect prediction variance, giving \(0.0576+0.36=0.4176\) square units for its conditional predictive variance, under the model assumptions.
Choosing a Model and Checking Its Use
A fixed-effects representation is often useful when inference is explicitly about the batches in hand, the number of batches is limited, and no population-of-batches interpretation is intended. It gives a separate coefficient for each represented batch, subject to the design's ability to distinguish batch from biology. A random-effects representation is useful when batches are viewed as a sample from a broader population and partial pooling or prediction for additional comparable batches is part of the goal. These are choices about the target and assumptions, not a universal ranking of methods.
| Question | Fixed batch coefficients | Random batch effects |
|---|---|---|
| What is estimated? | Contrasts for the observed batches | A batch-effect distribution and conditional effects for observed batches |
| Are observed batch estimates pooled? | Not by the batch-effect model itself | Yes; estimates are generally shrunk toward the population mean |
| What about a new batch? | No batch-specific estimate without new data or an extra convention | Population mean prediction is zero before new-batch data; its uncertainty reflects \(\tau^2\) |
| Key additional assumption | Identifiable design and appropriate conditional mean model | Comparable, exchangeable batches following the specified effect distribution |
Several diagnostics and failure modes matter for either choice. First, inspect the design: if condition and batch are perfectly confounded, neither coding scheme can identify separate condition and batch contributions. Second, check whether a small number of batches can support the desired random-effects inference. A variance estimate near zero may indicate little estimated between-batch variation, but it can also reflect limited information; it does not prove that all batch differences are absent. Third, the random-effects distribution assumes comparable batches. If the set of batches combines distinct protocols or populations, a single zero-mean distribution may be a poor description. Fourth, check whether residual spread and conditional mean patterns are reasonably represented. A common random intercept cannot explain a batch effect that changes systematically with the covariates; that requires a different model structure, which is outside this tutorial's scope.
Cell number alone does not determine how well the between-batch distribution is estimated. Many cells within a few batches can make each batch mean precise, but they do not create many independent batch realizations from which to learn \(\tau^2\). Conversely, more batches help characterize between-batch variation, provided their effects are plausibly exchangeable. These limitations are important when interpreting shrinkage: the amount of pooling depends on the fitted variance components and the design, not on a general rule that random effects are always more stable or more correct.
Common Mistake
- Wrong reasoning: “The random-effect prediction for an unseen batch is zero, so the model says the unseen batch has no batch effect.”
- Correction: Zero is the assumed population mean of the effects. Before data from the new batch are observed, its effect still varies with variance \(\tau^2\); pilot observations update its predicted effect toward the observed deviation.
- Wrong reasoning: “A random-effect model always improves a fixed-effect estimate.”
- Correction: Shrinkage follows from the random-effects assumptions. If batches are not exchangeable, or if the target is specifically the contrast for one observed batch, that pooling may not answer the intended question.
Check Your Understanding
Answer each question, then compare with the solutions below.
- With \(\tau^2=0.09\), \(\sigma^2=0.25\), and \(n_j=5\), calculate the shrinkage factor for an observed batch.
- If that batch's adjusted mean deviation is \(-0.4\) response units, what is its random-effects estimate?
- What is the predicted mean effect of an exchangeable new batch before any cells are observed, and what does that prediction leave uncertain?
- Why can neither fixed nor random batch coding separate a biological condition effect from batch when each condition occurs in exactly one distinct batch?
Solutions. (1) \(\lambda=0.09/(0.09+0.25/5)=0.09/0.14\approx0.643\). (2) The estimate is \(0.643(-0.4)\approx-0.257\) response units. (3) The predicted mean is 0, but a particular new batch's effect remains uncertain; under the stated model its between-batch variance is \(\tau^2\). (4) Condition and batch indicators encode the same separation in the design, so their contributions are not identifiable independently from those observations.