Sources and Structure of Batch Effects

Why Batch Effects Have Different Structures

Expression measurements can differ for technical reasons even when the underlying biological material is similar. A processing date, reagent lot, operator, instrument, or laboratory can change measured expression. The term batch effect refers to systematic technical variation associated with such differences in how samples are handled or measured. It is not a synonym for random measurement noise: a batch effect tends to produce a repeatable pattern across samples processed under the same conditions.

The pattern matters. A technical factor might shift many genes by approximately the same amount on a log-expression scale, affect low- and high-intensity signals differently, or change the apparent abundance of genes because the samples contain different mixtures of cells or RNA. These patterns have different causes and implications. A single global rescaling or shift cannot be assumed to address all of them.

The discussion uses expression values after an assay-specific measurement and normalization step. On a log scale, equal additive changes correspond to equal fold changes on the original scale. For example, an increase of one log-base-2 unit corresponds to a doubling. The model descriptions below are useful approximations: they require that the expression scale and the technical factor being discussed are defined clearly, and they do not imply that every gene responds identically.

Definition: A batch is a group of samples that share a relevant technical processing or measurement condition, such as processing date, reagent lot, operator, instrument, or laboratory. A batch effect is a systematic difference in measured expression associated with that condition, rather than a biological difference of interest. Batch is a label for a technical grouping; the effect may be gene-specific and need not have the same form for all genes.

Sources of Technical Variation

Batch labels often stand in for several operational differences. A processing date can capture changes in temperature, timing, sample storage, or instrument calibration. Reagent lots can differ in concentration, binding properties, or other manufacturing characteristics. Operators may differ in pipetting, timing, or adherence to a protocol. Instruments can vary in sensitivity, background signal, or settings. Laboratory site can combine many of these factors. If multiple technical factors change together, an observed difference associated with a batch label cannot automatically be attributed to one particular cause.

The earlier material on expression normalization concerns technical differences such as array distributions or sequencing library sizes that can often be represented as scale or distribution differences across samples. Batch effects are broader. They can remain after normalization, and normalization can itself obscure meaningful differences if its assumptions do not fit the data. For instance, bringing sample distributions into closer agreement does not establish that all remaining differences are biological, nor does it ensure that gene-specific technical response has been removed.

For a clear description, record technical metadata at the sample level: processing date, reagent lot, operator, instrument identifier, site, and relevant protocol version. Record the biological variables and sample attributes that could affect expression, too. Without such records, an observed pattern may be visible but difficult to connect to a specific process.

Three Useful Structures

Let \(Y_{gi}\) denote the measured expression for gene \(g\) in sample \(i\), on a chosen analysis scale. Let \(B_i\) identify the batch for sample \(i\), and let \(\theta_{gi}\) represent the biological expression level relevant to the study. A simple additive description on the log scale is:

Y_{gi} = \theta_{gi} + \gamma_{g,B_i} + \varepsilon_{gi}.

Here \(\gamma_{g,B_i}\) is a batch-associated shift for gene \(g\), and \(\varepsilon_{gi}\) is residual variation not represented by the biological level and batch shift. This description assumes that, for the samples being compared, the batch contribution for a given gene can be treated as a constant shift on the chosen scale. It allows different genes to have different shifts. It does not assume that all genes share one common offset.

A second structure is an intensity-dependent effect. Write \(I_{gi}\) for a measure of signal intensity or abundance, and let \(f_{g,b}(I_{gi})\) describe how the technical contribution in batch \(b\) varies with intensity. Then a more flexible description is:

Y_{gi} = \theta_{gi} + f_{g,B_i}(I_{gi}) + \varepsilon_{gi}.

A constant \(f_{g,b}\) reduces to an additive shift. If the contribution changes with intensity, the difference between batches may be small at low expression and larger at high expression, or may even change direction. This can arise when sensitivity, background, or signal response differs across measurement conditions. On a raw scale, a constant log shift corresponds to a multiplicative factor, whereas a nonconstant log-scale difference is not represented by one factor.

A third structure arises when the samples differ in composition. For example, a tissue sample may contain different proportions of cell types, or an RNA library may devote different fractions of its reads to different transcripts. The resulting expression profile can differ systematically even without a technical change in how a particular gene is measured. Composition can be biological, technical, or both. It is essential to distinguish a change in what material is present from a change in measurement of the same material.

Key distinction: An additive batch shift is approximately constant across the relevant samples for a gene on the chosen scale. An intensity-dependent effect varies with signal level. A composition effect reflects a change in the mixture contributing to the measured profile. These structures can coexist, and a single batch label may be associated with more than one of them.

Worked Examples of Batch Structure

Worked Example: A Constant Shift on the Log Scale

Suppose a hypothetical gene has log-base-2 expression values of 6.0 and 6.2 in two samples processed in Batch A, and 7.0 and 7.2 in two comparable samples processed in Batch B. Assume for this illustration that the samples have similar biological expression for this gene and that the same measurement scale was used.

The batch means are:

\bar{Y}_{A} = \frac{6.0+6.2}{2}=6.1,\qquad \bar{Y}_{B} = \frac{7.0+7.2}{2}=7.1.

The observed difference is \(7.1-6.1=1.0\) log-base-2 unit. The within-batch deviations are \(-0.1\) and \(0.1\) in each batch, so the two small sets have the same pattern around their respective means. If the biological levels are indeed comparable, this is consistent with an additive shift of about one log-base-2 unit for this gene in Batch B. On the original scale, a one-unit log-base-2 difference is a factor of \(2^1=2\); it corresponds to twice the measured expression, not an increase of one expression unit.

The data alone do not prove that the difference is technical. The interpretation depends on the stated comparability assumption. A biological difference between the samples could produce the same numerical pattern.

Worked Example: An Intensity-Dependent Difference

Consider hypothetical log-expression measurements from two otherwise comparable batches at two signal levels. At low intensity, Batch A has mean 4.0 and Batch B has mean 4.1. At high intensity, Batch A has mean 8.0 and Batch B has mean 8.8.

The batch difference at low intensity is \(4.1-4.0=0.1\) log-base-2 unit. At high intensity it is \(8.8-8.0=0.8\) log-base-2 unit. The difference therefore changes by \(0.8-0.1=0.7\) log-base-2 unit between these levels. A single constant shift cannot match both observed differences: subtracting 0.1 from Batch B would align the low-intensity means but leave a difference of \(0.7\) at high intensity.

Under the assumption that the compared samples have similar biological expression at each intensity level, this pattern is consistent with an intensity-dependent technical contribution. A diagnostic plot of batch differences against intensity would be more informative than comparing only the overall batch means. In real data, the biological comparability assumption remains important; intensity-dependent biology or sample composition can also create a pattern across expression levels.

Worked Example: A Composition Difference in a Mixture

Imagine a hypothetical tissue containing two cell types. A gene is expressed at 100 units per cell in Type A and 10 units per cell in Type B. Assume equal RNA contribution per cell, equal measurement efficiency, and no change in expression within either cell type. Sample 1 contains 80% Type A and 20% Type B; Sample 2 contains 40% Type A and 60% Type B.

The expected mixture-level expression in Sample 1 is \(0.80(100)+0.20(10)=80+2=82\) units. For Sample 2 it is \(0.40(100)+0.60(10)=40+6=46\) units. The difference is \(46-82=-36\) units, despite no change in expression within either cell type. The mixture ratio is \(46/82\approx0.561\), so the expression in Sample 2 is about 56.1% of that in Sample 1 for this gene under the stated assumptions.

This is a composition-driven expression difference. If the cell-type proportions differ because samples were collected from different biological groups, the difference may be biologically meaningful rather than a technical artifact. If a technical handling step selectively loses one cell type, it can instead induce a technical composition difference. The arithmetic shows why a shift in measured expression does not by itself identify its source.

How to Recognize and Investigate the Pattern

The first diagnostic step is to compare expression patterns with recorded technical variables, while also considering biological sample characteristics. Useful summaries include distributions by batch, gene-level batch contrasts, and plots of batch differences against overall intensity. These checks address different questions: a distribution comparison can show broad shifts, while intensity-stratified comparisons can reveal trends hidden by an overall average. Review sample-level metadata for unusual processing conditions and check whether a small number of samples or genes dominate a pattern.

A batch-wide shift in many genes is not automatically a single additive effect. Different genes may move by different amounts, and technical effects can be strongest near the detection limit or at high signal. Conversely, similar distributions do not rule out gene-specific changes that cancel in aggregate. For composition, inspect relevant sample attributes or available cell-type measurements; expression differences among marker genes can suggest a mixture change, but they are not definitive evidence on their own.

A practical investigation follows a sequence:

Define the measurement.
State the assay, expression scale, sample unit, and batch labels. A shift on the log scale has a different numerical interpretation from a shift on the raw scale.

Inspect technical records.
Compare processing dates, reagent lots, operators, instruments, and sites. Determine whether several factors changed together before attributing an observed pattern to one of them.

Compare pattern, not just averages.
Check distributions and gene-level differences, then examine whether differences vary across intensity or are concentrated in particular groups of genes.

Consider sample composition.
Ask whether cell mixture, tissue content, RNA contribution, or transcript composition could explain the expression pattern. Keep biological composition distinct from technical measurement variation.

What Batch Effects Are Not, and What Can Go Wrong

Normalization and batch investigation address related but different concerns. A normalization procedure targets a specified technical feature, such as between-sample scale or distribution, under assumptions about the data. Batch describes a grouping associated with systematic technical conditions. Normalization may reduce one form of variation without removing another, and an observed batch-associated difference can remain after normalization. Conversely, a normalization procedure that forces distributions to agree may hide genuine global biological changes when its assumptions are not appropriate.

Batch is also not automatically a nuisance to be removed. A processing date may be associated with a real shift in how samples were measured, but a difference in cell composition may be part of the biological question. A correction that treats every batch-associated difference as unwanted could erase scientifically relevant signal. The interpretation requires knowledge of the study and the measurement process, not just a statistical pattern.

A further limitation is that batch labels can bundle multiple technical causes. If one reagent lot was used only on one instrument, the data may show a difference associated with that combination, but cannot by themselves assign separate contributions to lot and instrument. Good planning records each factor separately and, where feasible, distributes biological groups across processing conditions. When technical conditions and biological groups are aligned, their observed differences are difficult to distinguish; this is a design warning, not something that a diagnostic plot alone can resolve.

Common Mistakes

Calling every sample difference a batch effect. A difference associated with a batch label may also reflect biology or composition. Check what was measured and how samples were allocated.
Assuming one offset fits all genes. Batch effects can be gene-specific or intensity-dependent. Compare patterns across genes and intensity levels before describing the effect as a constant shift.
Confusing composition with measurement. A different mixture of cells can change expression even when measurement within each cell type is unchanged. Decide whether the mixture change is biological, technical, or unresolved.
Treating normalization as proof of batch removal. A normalized data set may still contain technical structure. Evaluate the particular pattern of concern rather than assuming that one preprocessing step addresses every source.
Attributing a bundled effect to one recorded factor. If date, operator, and reagent lot change together, their separate effects are not established merely because the batch means differ.

Key takeaway: Describe a batch effect by its source, scale, and pattern. A constant shift, an intensity-dependent response, and a change in sample or RNA composition are distinct explanations for structured expression differences; they require different evidence and should not be collapsed into one generic correction.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

On a log-base-2 expression scale, a gene's mean is 5.2 in one batch and 6.2 in another. What is the difference, and what fold-change factor does it represent?
Batch differences are 0.2 log units for low-intensity genes and 1.0 log unit for high-intensity genes. Why is a single constant shift inadequate?
A tissue sample has a different proportion of two cell types across batches. Can expression differences be interpreted automatically as changes in measurement? Explain.
Processing date and reagent lot always occur together in a hypothetical data set. What can be concluded about their separate effects?
Name two checks that could help distinguish a broad shift from an intensity-dependent pattern.

Solutions. (1) The difference is \(6.2-5.2=1.0\) log-base-2 unit, corresponding to a factor of \(2^1=2\) on the original scale. (2) The difference changes by \(1.0-0.2=0.8\) log unit across intensity, so one constant offset cannot align both ranges. (3) No. The mixture change can alter measured expression even if expression within each cell type is unchanged; the mixture may be biological or may result from technical handling. (4) The data show an association with the combined processing conditions, but do not separate the date effect from the reagent-lot effect. (5) Compare distributions or gene-level batch differences, and plot differences against expression intensity; these checks reveal different aspects of the pattern.