Why Review the Allocation Before Collecting Data?
A study plan can specify an assay, a cell target, and a sophisticated analysis while still failing to support its scientific question. The central review question is whether the planned independent biological samples and their allocation allow the intended condition effects to be distinguished from donor variation and technical batch effects. A large number of cells does not fix an allocation in which every control is processed in one batch and every treated sample in another.
A design review is not a search for perfectly equal cell counts or a universal minimum number of donors. It is a structured check that the comparison is identifiable, that the independent replication matches the population to which conclusions will apply, and that the analysis can be tied to a clearly stated contrast. Any revision should preserve the scientific question while removing avoidable ambiguity and confounding.
A Practical Audit
Start by writing down the independent experimental units, the conditions assigned to them, the batches in which they will be processed, and the exact comparison of interest. In a typical donor-level condition comparison, the donor is the biological replicate; cells provide measurements within a donor. Technical batches may include library preparation days, sequencing runs, or other processing groups. The review should distinguish these levels rather than treating every cell as an independent replicate.
For a simple additive donor-level model, let \(Z_i\) be a donor-level outcome for donor \(i\), such as a cell-type-specific pseudobulk expression summary. A schematic model is
where \(T_i\) indicates condition, \(B_i\) indicates batch, and \(\varepsilon_i\) represents remaining variation under the model. This model is only an audit aid: the appropriate outcome model and variance assumptions depend on the analysis. Its key design implication is general. To estimate a condition effect separately from a batch effect, the observed allocation must contain information that distinguishes the condition and batch indicators. When they are identical, the two effects cannot be separated by adding both terms to the model.
A useful review checks four questions in order:
State what is independently sampled or assigned. Count independent donors by condition, not cells, when the intended inference concerns variation among donors.
Make a table of donor counts by condition and batch. Check for empty cells in the table and extreme imbalance, not just the total number of samples.
Specify which condition means are compared and how they are combined. A phrase such as “treatment effect” is insufficient when there are multiple doses or active conditions.
Revisit donor counts, expected attrition, and cell recovery. Use power calculations or simulation under defensible assumptions; do not treat a convenient donor count as guaranteed adequate.
Worked Design Reviews
Worked Example: Condition Completely Confounded With Batch
Hypothetical plan. A team plans to compare a control condition with a stimulated condition using eight donors. All four control donors will be processed in batch 1, and all four stimulated donors in batch 2. The team proposes to include both condition and batch in the analysis.
Let \(T_i=0\) for control and \(T_i=1\) for stimulated; let \(B_i=0\) for batch 1 and \(B_i=1\) for batch 2. Under the proposed allocation, \(T_i=B_i\) for every donor. The model therefore reduces to
The data can inform the combined difference \(\beta_T+\beta_B\), but cannot determine how much is attributable to condition and how much to batch. Including both terms does not solve this problem: their design columns are identical, so the separate coefficients are not identifiable.
Revision. If two batches are required, process donors from both conditions in each batch. For example, allocate three control and three stimulated donors to each batch, for six donors per condition and 12 donors total. Randomize or otherwise assign samples to processing positions within the practical constraints of the assay. The condition and batch indicators now vary independently in the allocation, so the additive condition and batch terms can be distinguished. This revision does not guarantee that the model is otherwise correct, but it removes complete condition–batch confounding.
Worked Example: Unequal Batch Allocation With Some Overlap
Hypothetical plan. Eighteen donors are divided equally between control and treatment. Batch 1 contains seven controls and two treated donors; batch 2 contains two controls and seven treated donors. Each batch contains both conditions, but the allocation is strongly imbalanced.
Unlike the previous plan, condition is not identical to batch: there are controls and treated donors in both batches. Under an additive model, the condition and batch effects are therefore distinguishable. However, the small cross-condition representation in each batch means that the adjusted comparison relies heavily on relatively few donors. A batch-specific condition difference cannot be assessed precisely from only two donors in one of the condition-by-batch groups, and an additive model may be a poor approximation if the condition effect varies by batch.
Revision. Keep the same 18 donors but distribute them as five controls and four treated donors in batch 1, and four controls and five treated donors in batch 2. Each batch now has substantial representation from both conditions, while the total remains nine donors per condition. This is a more robust allocation for separating an additive batch effect from a condition effect. Exact equality is not essential; adequate overlap and a defensible analysis matter more than matching every cell count.
A review should also check whether the planned number of cells per donor differs sharply by batch or condition. Unequal cell recovery can affect precision and may signal a feasibility problem, but balancing cells cannot substitute for allocating independent donors across conditions and batches.
Worked Example: Many Cells but Too Few Donors
Hypothetical plan. One control donor and one treated donor are each expected to yield 1,000 cells in the cell population of interest. The proposed analysis treats the 2,000 cells as independent evidence for a condition effect.
There is one independent donor in each condition. Cells from the same donor share biological features, so their measurements are not 1,000 independent replications of the donor-level condition assignment. In particular, the plan provides no replication within either condition from which to characterize between-donor variation. A cell-level analysis that ignores this dependence would make the evidence appear more precise than the donor allocation supports.
Revision. Recruit multiple independent donors per condition, then distribute those donors across processing batches so condition is not confounded with batch. For illustration, a revised feasibility plan might begin with six donors per condition, with each batch containing donors from both groups. Six is not a universal adequacy threshold: the final number should be justified using the expected donor-level variability, target effect, attrition, and power or simulation assumptions. Cell targets should be planned in addition to, not in place of, donor replication.
Worked Example: Replacing an Ambiguous Contrast
Hypothetical plan. A study has vehicle, low-dose, and high-dose groups. Its objective says only “compare treatment with control.” That phrase could mean high dose versus vehicle, low dose versus vehicle, or a combined comparison of both active doses with vehicle. These are different estimands and can produce different conclusions.
Suppose the planned donor-level mean outcomes on a specified analysis scale are 1.2 units for vehicle, 1.8 units for low dose, and 2.4 units for high dose. Using vehicle as the reference, define \(\beta_L=\mu_L-\mu_V\) and \(\beta_H=\mu_H-\mu_V\), where each \(\mu\) is the mean on that scale. Then
The high-dose-versus-vehicle contrast is 1.2 units. The average of the two active-dose means versus vehicle is
The high-dose-versus-low-dose contrast is \(2.4-1.8=0.6\) units. A revised plan should name the primary contrast, state its direction and analysis scale, and specify whether the other comparisons are secondary. The chosen contrast should also be considered when planning multiplicity; it should not be selected after inspecting which comparison looks most favorable.
Why Confounding Makes a Contrast Unestimable
The identifiability problem can be seen directly in the two-condition, two-batch model. If every control is in batch 1 and every treated donor is in batch 2, then \(T_i=B_i\). Substituting this equality into the model gives
For any constant \(c\), replacing \(\beta_T\) by \(\beta_T+c\) and \(\beta_B\) by \(\beta_B-c\) leaves \(\beta_T+\beta_B\), and hence the fitted values, unchanged. Thus the observed data cannot choose among infinitely many pairs of separate effects. No statistical adjustment can recover the condition effect from this allocation without additional assumptions or external information that is not supplied by the design.
Partial imbalance is different. When both conditions occur in each batch, the condition effect may be estimable under the model, though precision can be poor and conclusions may depend on assumptions such as an additive batch effect. This distinction is important in review: complete confounding is a structural failure; imbalance is a concern whose severity depends on overlap, sample size, variability, and the intended analysis.
Diagnostics, Alternatives, and Limitations
Before data collection, a condition-by-batch table is often the most revealing diagnostic. Flag empty condition-by-batch combinations, unusually small groups, and any batch that contains only one condition. Also verify that donor identifiers represent independent donors and that each planned contrast has support in the allocation. After collection, plots of donor-level outcomes by condition and batch can reveal outliers, unexpectedly sparse groups, or patterns inconsistent with a simple additive adjustment. These checks diagnose problems; they do not turn a non-identifiable design into an identifiable one.
There are several possible responses to a detected failure, but they are not interchangeable:
- Reallocate prospectively. When feasible, place both conditions in each batch and recruit additional independent donors if donor replication is inadequate. This directly addresses the allocation problem.
- Change the target question. If the intended condition effect cannot be separated from batch, report that limitation and consider whether a narrower, supportable comparison answers a useful question. Do not label a combined condition-and-batch difference as a condition effect.
- Use an analysis appropriate to the allocation. An analysis may account for measured batch variation when the design contains overlap. It cannot estimate separate condition and batch effects when their allocation indicators are identical.
- Use external information only with explicit assumptions. Calibration data or known technical controls may sometimes inform batch effects, but their relevance and uncertainty must be justified. They are not a substitute for donor replication.
A balanced allocation is often easy to interpret, but perfect numerical balance is not a requirement for valid analysis. Conversely, an apparently large total cell count does not establish adequate donor-level replication. Power depends on the number and variability of independent donors, the effect size, the allocation, and the analysis assumptions. Simulation and sensitivity analysis can assess plausible alternatives, but their conclusions are only as credible as the inputs.
Common Review Mistakes
- “We will adjust for batch.” This is not a remedy when condition and batch are completely confounded. Adjustment requires information in the allocation that distinguishes their effects.
- “We have thousands of cells, so we have thousands of replicates.” Cells from one donor are not independent donor replicates. Report and plan donor counts separately from cell counts.
- “The groups are unequal, so the design is invalid.” Unequal sizes are not automatically fatal. Check overlap and precision; distinguish imbalance from complete confounding.
- “Treatment versus control is obvious.” With multiple doses or conditions, it is not. State the specific groups and weights in the planned contrast.
- “A revised design with more donors is necessarily powered.” Additional donors generally improve information, but adequacy requires a calculation or simulation based on plausible variance, effect, and attrition assumptions.
Check Your Understanding
Answer each question, then compare with the solutions below.
- All control donors are processed in batch A and all treated donors in batch B. Can a model containing both condition and batch estimate their separate effects? Why?
- A study has 12 donors per condition, with both conditions present in each of two batches, but the batch counts are unequal. Is the design automatically invalid? What should be checked?
- Why is a plan with one donor per condition and 800 cells per donor inadequate for estimating between-donor variation?
- For vehicle, low dose, and high dose, name two distinct interpretations of “treatment versus control.”
- What is the primary prospective remedy for condition–batch confounding?
Solutions. (1) No. The condition and batch indicators are identical, so only their combined contribution is estimable from the study. (2) No. Unequal counts do not automatically invalidate the design. Check that both conditions occur in each batch, assess how sparse any condition-by-batch groups are, and justify the model assumptions and resulting precision. (3) Cells within a donor are not independent donor replicates; one donor per condition provides no replication for characterizing between-donor variation within either group. (4) Examples include high dose versus vehicle, and the average of low and high dose versus vehicle. They are different contrasts and should be specified. (5) Allocate donors from each condition across the batches, creating overlap so condition and batch effects can be distinguished.