Why Compare Identifiable Models?
When batch and biological condition are imbalanced, a condition estimate can depend on assumptions that are not fully checked by the observed data. A sensitivity analysis makes that dependence visible: it compares estimates and uncertainty under several scientifically interpretable models that are each identifiable from the available comparisons. The aim is not to find a model that produces the preferred result. It is to show how conclusions change when assumptions about batch effects or the analyzed population change.
The comparison here concerns one gene at a time, using a contrast in log-expression between two conditions. In practice, such contrasts should be based on biological replicates, often through pseudobulk summaries, rather than treating individual cells from the same sample as independent replicates. We assume that each analyzed batch has observations from both conditions, so its within-batch condition contrast is estimable. If a required comparison is absent, changing models cannot manufacture it.
The earlier material on batch-by-condition interactions and combining within-batch condition contrasts provides the ingredients for this comparison. The key is to state precisely what each estimate averages and what assumptions make that average meaningful.
Three Models, Three Interpretations
Let \(\hat{\theta}_b\) be the estimated condition contrast in batch \(b\), defined as the mean expression in the treated condition minus the mean expression in the control condition. Let \(v_b\) be the estimated variance of that contrast. The contrast is measured in log-expression units. The examples below treat the \(v_b\) values as known for transparent arithmetic; in an application they are estimated and uncertainty in those estimates must also be considered.
In the additive model, every batch has the same underlying condition effect, \(\theta\). Batch-specific intercepts may differ, but the treatment-minus-control contrast is assumed constant. If batch contrasts are independent and normally distributed with variances \(v_b\), the precision-weighted estimate is:
The interaction model does not require a shared condition effect: each batch has its own \(\theta_b\). It can report the batch-specific contrasts directly. If a single summary is needed, it must be defined separately, for example as a prespecified weighted average of the batch effects. The average is then a summary of differing effects, not evidence that the effects are equal.
A restricted-overlap analysis uses only batches or observations that satisfy a stated overlap rule. That can protect against extrapolating beyond a scientifically relevant common-support population. But restricting the data changes which samples contribute and can change both the estimate and its uncertainty. The rule should be based on design or covariate support, not chosen after seeing which restriction gives the desired answer.
Why Precision Weighting Works
Under the additive model, suppose \(\hat{\theta}_b=\theta+\varepsilon_b\), where the errors are independent, have mean zero, and have variances \(v_b\). Consider a linear estimate \(\sum_b a_b\hat{\theta}_b\) with \(\sum_b a_b=1\), so it is unbiased for \(\theta\). Independence gives its variance as \(\sum_b a_b^2v_b\). Minimizing this variance subject to the weights summing to one yields weights proportional to \(1/v_b\):
Substituting these weights gives variance \(1/\sum_b(1/v_b)\), as in the additive-model formula. This derivation depends on the common-effect assumption and independent contrasts. If contrasts share samples, donors, or other sources of error, their covariances must be included; the independent-contrast formula is then not justified.
Worked Example: Comparing Additive, Interaction, and Restricted-Overlap Estimates
Suppose three hypothetical batches yield gene-level treated-minus-control contrasts. The estimated contrast and its variance are: batch A, \(2\) with variance \(1\); batch B, \(6\) with variance \(4\); and batch C, \(-1\) with variance \(1\). Each batch contains both conditions. Assume the contrasts are independent and normally distributed with the stated variances. A prespecified sample-design rule excludes batch C from the restricted-overlap target because its donor strata are not represented in A and B.
Additive model. The inverse-variance weights are \(1\), \(1/4\), and \(1\). Thus:
The estimated variance is \(1/2.25=0.4444\), so the standard error is \(\sqrt{0.4444}=0.6667\) log-expression units. Under the stated normal model, a 95% interval is \(1.111\pm1.96(0.6667)\), or approximately \([-0.196,\,2.418]\) log-expression units. Its repeated-sampling interpretation is conditional on the additive model and the variance assumptions.
Interaction model. The batch-specific estimates remain \(2\), \(6\), and \(-1\), with standard errors \(1\), \(2\), and \(1\). They should not be collapsed into a single effect without defining the desired summary. If the inverse-variance-weighted average is chosen as that summary, its value is \(1.111\), as above; this equality of arithmetic does not impose equal batch effects.
The weighted heterogeneity statistic around that average is:
Under a common-effect model with independent normal contrasts and known variances, \(Q\) has approximately a chi-square distribution with \(3-1=2\) degrees of freedom. Here the upper-tail probability is about \(0.0037\), which is evidence of heterogeneity relative to that model. It does not prove that the interaction model is the uniquely correct explanation; unmodeled differences or variance misspecification could also matter.
Restricted overlap. Using only A and B, the weights are \(1\) and \(1/4\). The restricted estimate is:
Its variance is \(1/1.25=0.8\), and its standard error is \(\sqrt{0.8}=0.8944\) log-expression units. A normal-based 95% interval is \(2.8\pm1.96(0.8944)\), or approximately \([1.047,\,4.553]\). This interval concerns the restricted A-and-B target under its assumptions; it is not simply a more cautious interval for the same target as the all-batch estimate.
Interpreting Changes in Estimates and Uncertainty
The example shows why sensitivity analysis reports both the point estimate and its uncertainty. The all-batch additive estimate is \(1.111\), while the restricted estimate is \(2.8\). This difference can arise because the excluded batch has a different contrast and because restriction changes the target population. It is not, by itself, proof that either estimate is biased. The point estimates answer related but not identical questions.
Uncertainty can move in either direction. Removing batches usually reduces the total information if the target and model are unchanged, but here the target also changes. The restricted estimate has a larger standard error than the all-batch pooled estimate, \(0.8944\) rather than \(0.6667\), because its total precision is lower. Yet an interaction model may have less precise batch-specific effects than a common-effect estimate because it does not pool information under equality. These uncertainties answer different inferential questions.
A useful comparison table makes these distinctions explicit. In an actual analysis, identify the contrast scale, included biological replicates, weighting rule, and target population alongside each row.
| Analysis | Assumption or target | What to report |
|---|---|---|
| Additive | One common condition effect across included batches | Pooled estimate, standard error or interval, and evidence of heterogeneity |
| Interaction | Batch-specific condition effects may differ | Each batch effect and uncertainty; define any summary average explicitly |
| Restricted overlap | Effect among the batches or observations meeting a prespecified support rule | Restriction rule, changed target, estimate, and uncertainty |
Worked Example: A Common Effect Can Hide Opposing Batch Effects
Consider two hypothetical batches with independent condition contrasts \(1.0\) and \(3.0\) log-expression units, each with variance \(0.25\). Assume normal errors and known variances. The additive estimate is the equally weighted mean:
The interaction model instead reports \(1.0\) and \(3.0\), each with standard error \(\sqrt{0.25}=0.5\). The additive point estimate of \(2.0\) is their average, but it conceals a two-unit difference between batches. The heterogeneity statistic is \(Q=4(1-2)^2+4(3-2)^2=8\), with \(1\) degree of freedom under the common-effect null; its upper-tail probability is approximately \(0.0047\). The result cautions against presenting \(2.0\) as an adequate description of every batch. It does not establish why the effects differ.
If the scientific target is the equally weighted mean of these two batch effects, its estimate is \(2.0\). If the target is the effect in batch 1, the estimate is \(1.0\). Those are different targets, so the choice should follow the scientific question rather than which estimate is statistically significant.
Assumptions, Diagnostics, and Failure Modes
Before comparing results, check that the models are estimable and that their uncertainty calculations match the data structure. For the calculations above, the needed conditions are independent biological-replicate contrasts, valid standard errors, and a common scale for the contrasts. The additive model adds the assumption that the underlying condition effect is common across included batches. The interaction model relaxes that assumption but requires enough within-batch information to estimate the effects it reports. Restriction requires a defensible, explicitly stated rule for selecting the target population.
- Check support and estimability. Verify that each batch-specific contrast being reported has both conditions represented. A model matrix that is rank-deficient, or nearly so, signals a non-estimable or unstable contrast, not a reason to trust a software-generated coefficient automatically.
- Inspect the contrasts and their uncertainty. Large differences relative to their standard errors may indicate heterogeneity. Very wide intervals may show that a batch-specific estimate is too imprecise for strong interpretation.
- Check dependence and variance assumptions. Shared donors or samples can make batch contrasts correlated. If so, calculate uncertainty using the covariance structure rather than treating the contrasts as independent.
- Review the restriction rule. Confirm that the subset was chosen for support or design reasons and describe which samples or donor strata it represents. A result that changes sharply under restriction is a finding to explain, not a basis for hiding one analysis.
- Do not interpret a nonsignificant heterogeneity test as proof of equality. With few batches or imprecise contrasts, the test may have little power. Examine estimates and intervals as well as a test statistic.
These models do not resolve confounding that the design cannot identify. Nor does a similar estimate across models prove that all relevant assumptions hold. Conversely, a change in the estimate does not automatically identify which model is wrong: the models may target different populations, or their assumptions may fail in different ways. The purpose is transparent characterization of assumption sensitivity, not a definitive model-selection rule.
A Practical Reporting Sequence
State the condition contrast, expression scale, batches of interest, and whether the target is common across batches or restricted to a support-defined subset.
Estimate the common effect, the batch-specific effects, and the restricted-overlap effect using biological replicates and an appropriate variance model.
Report estimates in context, standard errors or intervals, and the amount of change from one assumption set to another.
Relate discrepancies to batch-specific patterns, precision, or the changed target. State what the data do and do not identify.
A concise report might say that the common-effect estimate was \(1.11\) log-expression units, while the prespecified A-and-B overlap estimate was \(2.80\), with the corresponding standard errors and intervals. It should then clarify that the restricted result targets the A-and-B population and that the batch contrasts showed evidence of heterogeneity. This is more informative than reporting only a preferred estimate or a corrected embedding.
Check Your Understanding
Answer each question using the definitions and calculations above.
- Two independent batch contrasts are \(4\) and \(0\), each with variance \(1\). What is the inverse-variance pooled estimate and its standard error under a common-effect model?
- In the three-batch example, why is the restricted A-and-B estimate not merely a more precise or less precise version of the all-batch estimate?
- What does a large heterogeneity statistic indicate, and what does it not establish?
- If two batch contrasts share donors, which assumption behind the displayed pooling variance may fail?
- A batch has observations from only one condition. Can its within-batch condition contrast be estimated directly?
Solutions. (1) The weights are \(1\) and \(1\), so the estimate is \([1(4)+1(0)]/2=2\). Its variance is \(1/2\), giving standard error \(\sqrt{0.5}=0.7071\). (2) Restriction changes which batches define the target, as well as the precision; it is therefore a different target population, not just a change in interval width. (3) It indicates that the observed contrasts vary more than expected under the specified common-effect model and variance assumptions. It does not prove that interaction is the unique explanation or identify the cause of variation. (4) Independence may fail; the covariance between contrasts must then be included. (5) No. Without both conditions in that batch, its within-batch contrast is not directly estimable; a model cannot supply the missing comparison without additional assumptions.