Why Model Specification Matters
Batch adjustment depends on how the model represents technical differences and which biological covariates it preserves. A batch term that is too simple can leave systematic technical variation behind; a term that is too flexible can absorb biological variation, especially when batch and biology overlap poorly. The practical question is not whether one model can be declared correct from the data alone, but whether the scientific conclusion depends on reasonable alternative specifications.
Sensitivity analysis makes this dependence visible. Fit a prespecified set of plausible models to the same observations, then compare both the estimated batch adjustments and the biological contrasts of interest. Keep the analysis target fixed: changing the model should not silently change which population or contrast the result describes.
For a measured feature, write a general linear model as \(y_i = x_i^\mathsf{T}\beta + \epsilon_i\), where \(y_i\) is the measured abundance for sample \(i\), \(x_i\) contains the chosen model terms, and \(\epsilon_i\) is the error. The fitted batch component depends on which columns of the design matrix represent batch and how those columns combine with biological covariates. If \(\widehat{b}_i\) is the fitted batch contribution, an adjusted value may be defined as \(y_i-\widehat{b}_i\), with the precise adjustment depending on which other fitted contributions are retained. Thus, compare adjusted values only under a clearly stated convention, and compare biological coefficients separately.
What Must Be Held Fixed
Before comparing models, state the outcome scale, the biological contrast, the set of samples, and the adjustment convention. For example, the contrast might be the case-minus-control difference in log2 abundance. The same samples should be used in every model; otherwise an apparent specification effect can instead reflect a changed analysis population. If missing covariates force different samples into different fits, report that limitation rather than attributing all differences to model terms.
Candidate specifications can vary in three main ways:
- Batch terms: for example, treating a batch label as a categorical factor rather than omitting it or imposing a numeric trend.
- Covariate sets: adding or removing measured variables such as age or sex when their inclusion is scientifically justified.
- Interactions: allowing a batch effect, or a biological contrast, to differ across batches rather than imposing one common effect.
Each model requires a design matrix with estimable coefficients. In practical terms, the columns must not be redundant, and the data must contain enough overlap to distinguish the terms of interest. If all cases occur in one batch and all controls in another, a case effect and a batch effect cannot generally be separated from those observations alone. Software may still return a number after dropping a column or applying a constraint, but that number does not resolve the underlying lack of information.
Worked Example: Batch Imbalance Changes the Crude Contrast
Consider hypothetical log2-abundance measurements from two batches. Batch A has three controls with mean 10 and one case with value 14. Batch B has one control with value 13 and three cases with mean 17. Within each batch, the case-minus-control contrast is 4 log2 units. The batch means differ by 3 log2 units for either biological group.
If batch is omitted, the group means are weighted by the observed group counts:
The crude contrast is therefore \(16.25-10.75=5.50\) log2 units. Its excess over the within-batch contrast arises because cases are more concentrated in the higher-mean batch.
Now fit an additive model with a categorical batch term and a common case effect. The cell means satisfy the model exactly: starting from the Batch A control mean of 10, the Batch A case mean is 14, the Batch B control mean is 13, and the Batch B case mean is 17. The estimated common case effect is 4 log2 units, and the Batch B shift is 3 log2 units. In this hypothetical example, the additive model recovers the within-batch contrast, whereas omitting batch gives 5.50. This comparison depends on the assumption that the case contrast is common across batches; the interaction model in a later example checks that assumption.
Changing Covariates and Interactions
Covariates should be added because they represent plausible structure relevant to the target contrast, not simply because they change a coefficient. A useful comparison asks whether the covariate is associated with both batch allocation and the outcome, whether its functional form is defensible, and whether enough overlap remains to estimate the contrast. An adjusted coefficient is conditional on the covariates in its model; it need not equal a crude coefficient, and neither is automatically the preferred scientific answer.
Interactions address a different question: whether an effect is shared across groups. A model with batch and biology main effects imposes a common biological contrast across batches. Adding a batch-by-biology interaction relaxes that restriction. It can reveal heterogeneity, but requires enough observations in the relevant batch-by-biology cells. If the interaction is poorly identified, large changes or unstable estimates are a warning about limited information, not proof of a meaningful batch-specific effect.
Worked Example: Adding an Age Covariate
Suppose four hypothetical samples have the following measurements. Abundance is in log2 units; age is in years.
| Group | Age | Abundance |
|---|---|---|
| Control | 40 | 5 |
| Control | 60 | 15 |
| Case | 50 | 12 |
| Case | 70 | 22 |
The crude control mean is \((5+15)/2=10\), and the crude case mean is \((12+22)/2=17\). Thus, the crude case-minus-control contrast is \(17-10=7\) log2 units.
Now fit the specified linear model \(y_i=\beta_0+\beta_1 I(\mathrm{case})_i+\beta_2(\mathrm{age}_i-50)+\epsilon_i\). The values satisfy the equation exactly with \(\widehat{\beta}_0=10\), \(\widehat{\beta}_1=2\), and \(\widehat{\beta}_2=0.5\). For example, the predicted control value at age 40 is \(10+0+0.5(40-50)=5\), and the predicted case value at age 70 is \(10+2+0.5(70-50)=22\). The fitted case contrast at the same age is 2 log2 units.
The model matrix has independent columns for the intercept, case indicator, and centered age, so these coefficients are estimable for these data. However, four observations provide very little information: the model uses three coefficients, leaving only one residual degree of freedom. The exact fit is a property of this small constructed example, not evidence that a linear age effect is adequate in an application. The change from 7 to 2 illustrates how group differences in age can affect the estimated contrast; deciding which model is relevant requires scientific justification and adequate data.
Worked Example: Testing a Common-Contrast Assumption
Consider hypothetical cell means for two cohorts, with the same number of samples in each cell: Cohort A controls have mean 10 and cases mean 14; Cohort B controls have mean 14 and cases mean 20. All values are in log2-abundance units.
The cohort-specific contrasts are \(14-10=4\) in A and \(20-14=6\) in B. A model with cohort and biology main effects only imposes a common contrast. With equal cell sizes, its fitted common contrast is the average of the two within-cohort contrasts:
An interaction model, using Cohort A and control as reference levels, estimates a control mean of 10, a Cohort B shift of \(14-10=4\), a case contrast in A of \(14-10=4\), and an interaction of \(6-4=2\). Its case contrast is therefore 4 in A and \(4+2=6\) in B. The positive direction is consistent across cohorts, but the common-effect interpretation is not: the cohort-specific estimates differ by 2 log2 units. Whether that difference matters scientifically depends on the scale and intended use of the contrast, not on the arithmetic alone.
A Justification for Comparing Common Effects
The averaging rule in the previous example follows directly from least squares when there are two groups in each batch, equal cell sizes, and a model that imposes one common biological contrast. Let \(\widehat{\Delta}_b\) be the observed case-minus-control difference in batch \(b\), and let \(m\) be the equal number of observations per cell. For a proposed common contrast \(\tau\), the part of the residual sum of squares involving within-batch case-control differences is proportional to \(\sum_b m(\widehat{\Delta}_b-\tau)^2\). Differentiating with respect to \(\tau\) and setting the derivative to zero gives:
Here \(B\) is the number of batches. With unequal cell sizes, the least-squares common contrast is generally a weighted combination rather than this simple average. The precise weights depend on the design and model. This is why a pooled coefficient should be interpreted in light of both the specification and the sample allocation.
How to Summarize Sensitivity
A useful comparison is organized around the scientific claim rather than a list of model names. For each prespecified model, record the biological contrast, its units, the fitted batch contribution or adjusted values, and any change in the conclusion. If uncertainty estimates are reported, keep their interpretation separate from the question of whether point estimates change across specifications.
| Comparison | Question to ask |
|---|---|
| Batch terms | Does the estimated batch correction or biological contrast change when batch is represented differently? |
| Covariate sets | Does adding a justified covariate materially alter the target contrast, and is there sufficient covariate overlap? |
| Interactions | Do batch-specific contrasts differ enough to make a single common contrast misleading? |
| Residual pattern | After adjustment, does a systematic association with batch remain? |
Diagnostics should be considered alongside coefficient comparisons. Check whether the design matrix is full rank, whether cells are empty or nearly empty, and whether fitted residuals still show a batch pattern. A residual pattern can indicate that the chosen batch structure is inadequate, but adding terms mechanically is not a remedy if the new terms cannot be distinguished from biology. Residual checks after adjustment and negative-control features can provide complementary evidence about remaining technical structure.
A concise robustness summary might say that the estimated contrast stayed positive and within a specified practical range across all prespecified models, while its magnitude changed under the interaction model. Alternatively, it might say that the sign reversed when a plausible covariate was included, so the direction is specification-sensitive. Avoid declaring a result “robust” merely because several nearly identical models agree.
Common Mistakes and Limitations
- Selecting the model that gives the preferred result. Choosing terms after seeing which model produces the desired contrast makes the comparison selective. Define plausible alternatives from the design and scientific question, then report the set examined.
- Treating a small coefficient change as proof of no sensitivity. A change may matter on the scientific scale even if its sign is unchanged. State the units and practical benchmark used to judge magnitude.
- Interpreting an interaction as automatically necessary. Interactions require information in the corresponding cells. Sparse or empty cells can make batch-specific contrasts unidentifiable or highly unstable.
- Calling model agreement proof of correctness. Plausible models can share the same misspecification, and no sensitivity analysis can recover a contrast that the design does not identify.
- Comparing fits on different samples without saying so. Different analysis populations can produce different estimates for reasons unrelated to model specification.
The central limitation is identifiability: sensitivity analysis can show how conclusions respond to assumptions, but cannot separate batch from biology when the design contains no information to do so. When batch and biology are strongly entangled, report that limitation explicitly rather than interpreting agreement across a narrow model set as resolution.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In the first worked example, what are the crude and batch-adjusted case-minus-control contrasts, and why do they differ?
- In the age example, calculate the crude contrast and the fitted case contrast at a common age. What assumption makes that adjusted contrast meaningful?
- For the two-cohort interaction example, calculate the cohort-specific contrasts and the interaction estimate. Is the direction consistent?
- What does an empty batch-by-biology cell imply for a model containing their interaction?
- Why is agreement among several plausible specifications not proof that the adjustment is correct?
Solutions. (1) The crude contrast is \(16.25-10.75=5.50\) log2 units; the additive batch-adjusted contrast is 4 log2 units. Cases are more concentrated in the higher-mean batch, shifting the crude contrast upward. (2) The crude contrast is \(17-10=7\) log2 units. The fitted common-age case contrast is \(\widehat{\beta}_1=2\) log2 units. Its interpretation relies on the specified linear age model and sufficient overlap to distinguish age from group. (3) The contrasts are 4 log2 units in Cohort A and 6 in Cohort B; the interaction estimate is \(6-4=2\) log2 units. Both contrasts are positive, so direction is consistent, although magnitude differs. (4) An empty cell removes direct information for that batch-by-group combination; the corresponding interaction contrast may be unidentifiable without additional assumptions. (5) The models may share a misspecification, and agreement cannot compensate for insufficient overlap or a design that does not distinguish batch from biology.