Decomposing Variance by Cohort and Platform

Why Decompose Feature-Level Variance?

An omics feature can vary across samples for several reasons. Measurements may differ systematically among cohorts, among measurement platforms, and among particular cohort-platform combinations; they also vary among replicate samples or through measurement noise. A variance-component model summarizes these sources on a common scale. It can show, for example, whether variation across the observed samples is mostly associated with cohort-level differences or with residual variation.

This summary is descriptive under the model, not a causal attribution. A large cohort component does not establish that cohort membership caused the feature values to differ. Cohort may also represent differences in recruitment, processing, clinical composition, or other factors. The earlier material on fixed-effect and random-effect batch adjustment distinguishes modeling group-specific shifts from treating group levels as a sample from a wider population. Here the focus is the variance partition and the design conditions needed to interpret it.

A Crossed Variance-Component Model

The standard model assumes that these four sets of random effects are mutually independent, and that effects at distinct levels within a set are independent. Residuals are independent with common variance \(\sigma_e^2\), conditional on the random effects. Normal random effects and residuals are commonly assumed for likelihood-based estimation and standard small-sample inference; the variance-component interpretation itself requires finite variances and the stated independence structure. The feature scale and any transformation must be specified because variances depend on scale.

The model is crossed when cohort and platform levels are combined in the design, rather than one factor being wholly nested within the other. In a balanced design, every cohort-platform cell has the same number \(m\) of replicate observations. Replication within cells is especially important: it helps distinguish the interaction, which is shared by observations in a cell, from residual variation among observations in that cell.

For observations in the same cohort-platform cell, the shared deviation is \(A_c+B_p+(AB)_{cp}\). Under the independence assumptions, its variance is therefore \( \sigma_C^2+\sigma_P^2+\sigma_{CP}^2 \). For a single observation, adding independent residual variation gives marginal variance \( \sigma_C^2+\sigma_P^2+\sigma_{CP}^2+\sigma_e^2 \). These sums are not generally a valid decomposition if effects are correlated or the model omits important structure.

Balanced-Design Estimates and Their Justification

Suppose there are \(C\) cohorts, \(P\) platforms, and \(m\) replicates per cell. A balanced analysis of variance partitions the total sum of squares into cohort, platform, cohort-platform interaction, and residual sums of squares. Their degrees of freedom are \(C-1\), \(P-1\), \((C-1)(P-1)\), and \(CP(m-1)\), respectively. Each mean square, \(MS\), is its sum of squares divided by its degrees of freedom.

The method-of-moments estimates follow by equating observed mean squares to their expectations under this model. In the balanced design, those expectations are:

For example, each cell mean averages \(m\) independent residuals, whose variance is \(\sigma_e^2/m\); the cell interaction remains common to all \(m\) observations. Thus cell-to-cell variation beyond residual variation contains the interaction variance. Cohort means also average over \(P\) platforms and \(m\) replicates per platform, while the cohort effect is shared across those observations. These repeated appearances yield the factor \(Pm\) in the cohort mean-square expectation. The platform expression follows analogously.

Solving these expectation equations gives the balanced-design estimates:

These are method-of-moments estimates, not guarantees that every estimated component is nonnegative. They are most straightforward when the design is balanced and the factors are identifiable. For unbalanced or incomplete designs, the same simple mean-square formulas generally do not apply; likelihood-based methods such as restricted maximum likelihood (REML) can estimate components if the design contains enough information to identify them.

Confounding Limits Attribution

Variance components can only be separated when the observed design supplies distinct information about the factors. If platform is perfectly tied to cohort—for example, every sample in cohort 1 is measured only on platform 1 and every sample in cohort 2 only on platform 2—observed differences between those groups cannot reveal whether they arise from cohort, platform, or their interaction. In a one-to-one pairing of cohort and platform levels, the observed cell deviation contains all three terms \(A_c+B_p+(AB)_{cp}\). Under the model's independence assumptions, the variance of that shared deviation is \(\sigma_C^2+\sigma_P^2+\sigma_{CP}^2\), not merely \(\sigma_C^2+\sigma_P^2\).

If there is only one observation in each observed cell, interaction and residual variation may also be inseparable. Repeated observations in a cell can provide information about residual variation, but they do not by themselves separate cohort from platform when those factors remain perfectly aliased. The earlier material on identifiability and design connectivity gives the corresponding design-matrix perspective. A variance-component model does not repair a design that lacks information.

Negative Estimates, Diagnostics, and Alternatives

Useful checks include confirming that all relevant cohort-platform combinations are represented, checking replication and missingness patterns, and verifying that the component formulas match the actual design. Investigate whether residual spread differs markedly across groups or whether a few observations dominate sums of squares. Such patterns challenge the common residual variance or independence assumptions. A large component is also sensitive to the scale of measurement: variance on a log-transformed abundance scale is not directly comparable to variance on the original scale.

The main alternative is to treat cohort or platform levels as fixed effects when the goal is to estimate differences among these particular levels. Fixed-effect contrasts answer questions about specified level differences; variance components summarize variability among levels under a random-effects interpretation. Either approach can be inappropriate if the design is confounded. In unbalanced designs, REML is often preferred to simple moment formulas, but it still depends on an identifiable design and a defensible model. Neither method determines whether an observed component is technical or biological without appropriate design information.

Why the Decomposition Matters

A feature-level variance partition can help prioritize investigation: a substantial platform component may motivate checks of measurement comparability, while substantial cohort variation may indicate heterogeneity that matters for pooled analyses. The partition is most useful when reported with the feature scale, design, component estimates, and uncertainty where available. Comparing proportions across features also requires care: features with different total variances can have similar proportions but very different absolute variability.

The central limitation is attribution. The estimated components describe variation under a specified model and sampling structure. When cohort and platform are confounded, the data may support only a combined contrast or variance, not a unique division between sources. Avoid reading a software output as evidence that every printed component was independently identified.