Why Cohort and Batch Structure Belong in the Simulation
A multi-omics method may appear to recover biological patterns when it is actually detecting which cohort or assay batch a sample came from. Conversely, a genuine cohort difference can be obscured by technical variation. A useful simulation therefore represents both the biological groups and the measurement process that acts on their samples. The assignment of samples to platforms and batches matters as much as the numerical size of the effects: if a cohort is processed only on one platform, its biological shift may be inseparable from a platform shift.
The goal here is to generate that structure deliberately. A scenario can represent clean separation of biological and technical effects, partial overlap, or complete confounding. These are distinct design conditions, not merely different choices of effect size.
A Generative Model for Cohort and Technical Effects
Let \(i\) index samples, \(c(i)\) denote sample \(i\)'s cohort, \(m\) its omics modality, and \(j\) a feature within that modality. Let \(Z_{ijm}\) be the sample's biological feature value before the technical shifts under study. It can contain the individual-level and cross-omics biological structure specified elsewhere in a simulation. For a simple additive model, the observed value is
Here \(\delta_{c,jm}\) is a cohort-specific biological shift, \(\pi_{p,jm}\) is a platform-specific measurement shift, \(\beta_{b,jm}\) is a batch-specific technical shift, and \(\varepsilon_{ijm}\) is residual measurement noise. The platform \(p(i)\) and batch \(b(i)\) are assignments attached to samples. Effects may vary by feature and modality; a shift of 2 assay units for one feature need not imply a shift of 2 units for another. The additive model is a useful starting point, not a claim that all assay effects are additive. For example, platform-dependent scaling would require a multiplicative or otherwise feature-specific measurement model.
Before generating values, specify whether each effect is fixed or random. A fixed-effect scenario sets explicit values, such as \(\delta_{A,jm}=0\) and \(\delta_{B,jm}=3\) assay units. A random-effect scenario draws effects from declared distributions. For instance, batch effects might be generated independently as \(B_b\sim N(0,\tau^2)\), where \(\tau^2\) is the batch-effect variance. This random-effects specification assumes the draws follow the stated distribution and are independent of the cohort assignments; those assumptions are substantive parts of the scenario. Do not silently treat a fixed effect as a random draw, or vice versa.
The assignment mechanism must also be stated. A batch is nested within cohort when each batch processes samples from only one cohort. Batches are crossed with cohort when batches process samples from more than one cohort. Partial overlap means some batches are shared across cohorts but other batches occur in only one. Platform assignments have the same distinctions. A simulation can assign platforms and batches separately or make batches platform-specific; whichever is intended, encode that relationship explicitly.
Worked Examples of Assignment Structure
Worked Example: Nested Batches with Cohort Shifts
Suppose a single feature is measured in assay units. The biological baseline is \(Z=10\) for all four hypothetical samples. Cohort A has shift \(\delta_A=0\), cohort B has \(\delta_B=3\). Cohort A has batches A1 and A2, with batch effects \(+1\) and \(-1\); cohort B has batches B1 and B2, with effects \(+2\) and \(-2\). Set residual noise to zero to make the arithmetic transparent.
| Sample | Cohort | Batch | Calculation | Observed value |
|---|---|---|---|---|
| 1 | A | A1 | 10 + 0 + 1 | 11 |
| 2 | A | A2 | 10 + 0 − 1 | 9 |
| 3 | B | B1 | 10 + 3 + 2 | 15 |
| 4 | B | B2 | 10 + 3 − 2 | 11 |
The cohort means are \((11+9)/2=10\) for A and \((15+11)/2=13\) for B, a difference of 3 assay units. The batch effects average to zero within each cohort in this construction. Nevertheless, batch and cohort are nested: no batch occurs in both cohorts. With unrestricted fixed batch and cohort effects, the cohort contrast cannot be separated from the batch effects solely from this assignment. Interpreting the cohort mean difference as biological requires additional assumptions, such as a random-batch model with independent, mean-zero batch effects and adequate replication across batches.
Worked Example: A Crossed Cohort–Batch Design
Now let both cohorts be measured in both batches. Use a biological baseline of 20 assay units, cohort shifts \(\delta_A=0\) and \(\delta_B=4\), and batch effects \(\beta_1=-1\) and \(\beta_2=1\). Suppose there is one hypothetical sample in each cohort–batch combination and residual noise is zero.
The four generated values are \(Y_{A1}=20+0-1=19\), \(Y_{A2}=20+0+1=21\), \(Y_{B1}=20+4-1=23\), and \(Y_{B2}=20+4+1=25\) assay units. The mean for cohort A is \((19+21)/2=20\); the mean for cohort B is \((23+25)/2=24\). The difference is 4 assay units. Comparing cohorts within either batch gives the same result: \(23-19=4\) in batch 1 and \(25-21=4\) in batch 2. Because both cohorts occur in both batches, cohort and batch contrasts can be distinguished under the additive model.
Worked Example: Partial Overlap and a Connected Design
Consider cohorts A and B and batches 1, 2, and 3. Cohort A is measured in batches 1 and 2; cohort B is measured in batches 2 and 3. Set the biological baseline to 5 assay units, cohort shifts to \(\delta_A=0\) and \(\delta_B=2\), batch effects to \(\beta_1=-1\), \(\beta_2=1\), and \(\beta_3=0\), and residual noise to zero.
The generated values are \(Y_{A1}=5+0-1=4\), \(Y_{A2}=5+0+1=6\), \(Y_{B2}=5+2+1=8\), and \(Y_{B3}=5+2+0=7\) assay units. The unadjusted cohort means are 5 and 7.5, giving a difference of 2.5 assay units. But batch 2 is shared: the within-batch cohort contrast is \(8-6=2\), which equals the specified biological shift. The design has partial overlap, and the shared batch connects the two cohorts. This example shows why the raw difference in cohort means need not equal a cohort contrast adjusted for batch.
Why Overlap Determines Identifiability
For one feature, temporarily combine the baseline and observed values into an additive fixed-effects model \(Y_{ic}=\mu+\delta_c+\beta_{b(i)}+\varepsilon_i\), suppressing the feature and modality indices. The design determines whether the cohort and batch contributions can be distinguished. Represent that design as a bipartite graph: one set of nodes is cohorts, the other set is batches, and a cohort–batch edge is present when at least one sample belongs to that combination.
When this graph is connected, and the model includes the usual reference constraints needed to define an intercept and effects, additive cohort and batch contrasts are identifiable. When it has separate connected components, contrasts across those components cannot in general be disentangled. The reason is that each disconnected component can have its own shift reallocated between its cohort and batch terms without changing any fitted value.
The non-identifiability in a nested design can be shown directly. If batch \(b\) belongs to cohort \(c\), choose any number \(t_c\) and replace \(\delta_c\) by \(\delta_c+t_c\) and every batch effect in that cohort by \(\beta_b-t_c\). Then
Every fitted sample value stays unchanged, although the cohort and batch effects have changed. The data therefore cannot determine which part of the sum belongs to cohort and which to batch when these are unrestricted fixed effects. In a crossed design, the same shift cannot be applied independently to one cohort without also changing fitted values for batches shared with another cohort. This is the algebraic reason overlap can identify contrasts.
This argument concerns the chosen model and assignment, not a blanket rule that nested designs are unusable. With multiple batches per cohort, a random-effects model can estimate cohort contrasts by treating batch effects as independent draws from a mean-zero distribution. That is a different source of information: the interpretation rests on the random-effects assumptions, not on direct fixed-effect separation. If a platform is also perfectly aligned with cohort, adding platform terms introduces the same identification problem. A simulation should make clear which assumption or overlap is intended to resolve it.
Constructing Scenarios and Checking Them
A robust simulation specifies assignment and effect generation separately. First create the cohort labels and the cohort–platform–batch assignment table. Then generate biological values and cohort shifts, technical effects, and residuals according to the stated model. Finally, combine the terms to obtain observed values. This order makes it easier to inspect assignments and to confirm that each numerical effect enters as intended.
Decide whether batches and platforms are nested, crossed, or partially overlapping with cohorts. Specify which cohort contrasts should be estimable.
Build the sample-level table first. Do not infer the assignment structure later from effect values or from an aggregate summary.
Set fixed effects or draw random effects from declared distributions, then combine them with biological values and residual noise using the model.
Tabulate cohort by platform and cohort by batch, inspect the assignment graph, and verify that observed values equal the specified component-wise calculations.
Useful checks include counting samples in every cohort–batch and cohort–platform combination; identifying batches used by only one cohort; and checking whether the cohort–batch graph is connected. Repeat these checks for each modality if its sample processing differs. When assignments are generated randomly, verify the realized structure for every simulated replicate: a design intended to include overlap can occasionally produce no useful overlap at a particular sample size.
Also compare the generated data with the intended effects. With residual noise set to zero, calculated cell means should equal the sums of the baseline, cohort shift, platform shift, and batch shift. With noise present, the cell means need not equal those sums exactly; the deviations should be consistent with the noise mechanism. A pattern in which every sample from a cohort has one platform, or in which a batch label is accidentally reused across unrelated batches, can silently change the scenario.
Alternatives, Limitations, and Common Pitfalls
| Approach | What it represents | Main caution |
|---|---|---|
| Crossed fixed effects | Explicit cohort and technical shifts with shared assignments | Requires sufficient overlap; a connected design may still have sparse or imbalanced cells. |
| Nested fixed effects | Distinct technical shifts for batches belonging to particular cohorts | Unrestricted cohort and batch contrasts are aliased without additional structure. |
| Random batch effects | Batch-to-batch variation drawn from a specified distribution | Inference relies on distributional and independence assumptions, not just the assignment table. |
| Balanced assignment | Similar representation of cohorts across platforms and batches | Balance improves comparisons but does not eliminate noise, model misspecification, or all confounding. |
A common mistake is to reason, “There are several batches in each cohort, so batch effects and cohort effects are separable.” That does not follow. If every batch belongs to only one cohort, the fixed-effect shift reallocation shown above leaves all fitted values unchanged, regardless of how many batches each cohort has. The correction is to inspect overlap, or to state explicitly the random-effects assumptions used to identify cohort differences from variation across batches.
Other limitations matter when interpreting a benchmark. A connected graph can be weakly connected: if only one sample links a cohort to the rest, estimates may be unstable. Assignments that are mathematically connected may still be implausible operationally. Additive effects may not represent platform-by-cohort interactions or feature-specific changes in scale. Finally, a simulated cohort shift is a specified data-generating feature, not proof that a real cohort difference is biological; it could stand for differences in composition, collection, or other factors. These scenarios describe what a method encounters under the model, not a causal explanation of observed data.
Choose scenarios to answer the benchmark question: a balanced crossed case can assess performance when technical and cohort contrasts are well separated, while nested or weakly overlapping cases can expose sensitivity to confounding. Include the assignment mechanism and effect assumptions when describing results. Do not treat one convenient design as representative of all cohort structures.
Check Your Understanding
Answer each question using the stated additive model and assignment structure.
- What distinguishes a crossed batch design from a nested batch design?
- In a fixed-effects model, why does having several batches within each cohort not by itself separate cohort and batch effects?
- Two cohorts both occur in batch 2, while cohort A also occurs in batch 1 and cohort B also occurs in batch 3. What does this overlap contribute?
- What extra assumptions allow a random-batch model to provide cohort contrasts even when batches are nested within cohort?
- Give two checks that can reveal a mismatch between the intended and realized simulated assignment.
Solutions. (1) A crossed design has batches containing samples from multiple cohorts; a nested design has each batch restricted to one cohort. (2) For any cohort-specific \(t_c\), adding \(t_c\) to its cohort effect and subtracting \(t_c\) from each nested batch effect leaves every fitted value unchanged, so the fixed effects are not separately identified. (3) The shared batch links the cohorts and permits a within-batch comparison; in a connected additive design, it helps distinguish cohort from batch contrasts. (4) The model must specify, for example, independent mean-zero batch effects drawn from a stated distribution, with assumptions about their relationship to cohort assignment; the contrast then relies on those assumptions. (5) Tabulate cohort by batch and cohort by platform, check graph connectivity, or compare generated observations with their component-wise calculations.