Worked Example: Estimation With Partial Batch Overlap

Why Partial Overlap Requires Care

Across cohorts, a feature may be measured on different platforms, while only some cohorts or reference samples appear on more than one platform. This partial overlap can be enough to separate a cohort contrast from a platform offset, but only under an explicit model and only when the observed cohort-platform combinations connect the relevant effects. A difference between two observed groups is not automatically a biological difference: it may combine cohort and platform shifts.

The earlier material on bridge samples and confounding motivates the design question. Here the goal is to work through the estimation: represent the observed combinations, identify the contrasts supported by them, and calculate the uncertainty of those contrasts.

Model and Conditions for Estimation

Definition: Let \(Y_{icp}\) be the measured value of a feature for sample \(i\) from cohort \(c\), measured on platform \(p\). The additive cohort-platform model is \(Y_{icp}=\mu+\alpha_c+\gamma_p+\varepsilon_{icp}\), where \(\mu\) is an overall level, \(\alpha_c\) is a cohort effect, and \(\gamma_p\) is a platform effect. A reference constraint, such as \(\alpha_1=0\) and \(\gamma_1=0\), fixes the parameterization. The model assumes no cohort-by-platform interaction for the feature.

For the calculations below, assume that, conditional on cohort and platform, errors have mean zero, common variance \(\sigma^2\), and are independent across samples. For exact small-sample \(t\) intervals, also assume normally distributed errors. The observed design must contain enough overlap to estimate the contrast of interest. In particular, the cohort-platform combinations form a bipartite graph: one set of nodes represents cohorts, the other platforms, and an edge represents an observed combination. Effects can be compared within a connected component, subject to the model and its reference constraint. A disconnected design cannot separate shifts across its components.

Let \(\bar{Y}_{cp}\) denote the sample mean in an observed cohort-platform cell, \(n_{cp}\) its sample size, and \(s^2\) the pooled residual variance from the fitted model. Under the additive model, a difference of two cell means from the same platform estimates the corresponding cohort difference. More generally, adjusted differences can be assembled along an overlap path; the platform shifts cancel when the path is combined. The contrast is estimable only if that combination is supported by the design.

A Direct Calculation in a Partially Overlapped Design

Worked Example: A Cohort Contrast Linked by a Bridge Cell

Suppose a hypothetical feature is measured in cohort A on platforms 1 and 2, and in cohort B on platform 2. Platform 1 is not represented for cohort B. The observed values are:

Cohort A, platform 1: 9, 10, 10, 11; \(n=4\), \(\bar{Y}_{A1}=10\).
Cohort A, platform 2: 11, 12, 13; \(n=3\), \(\bar{Y}_{A2}=12\).
Cohort B, platform 2: 14, 15, 15, 16; \(n=4\), \(\bar{Y}_{B2}=15\).

The observed cells form a connected design: cohort A links the two platforms. The additive model therefore estimates the cohort contrast. Because A and B are both observed on platform 2, the adjusted cohort B-minus-A contrast is

\widehat{\alpha_B-\alpha_A} =\bar{Y}_{B2}-\bar{Y}_{A2} =15-12=3.

In feature units, the fitted cohort B level is 3 units higher than cohort A after accounting for the additive platform shift. The platform 2-minus-platform 1 offset is also estimable from cohort A: \(12-10=2\) feature units. Thus the model predicts a value of \(15-2=13\) for cohort B on platform 1, although no B/platform 1 samples were observed. This prediction depends on the no-interaction assumption; it is not a directly observed cell mean.

The residual sums of squares within the three cells are \(2\), \(2\), and \(2\), respectively, so the total residual sum of squares is \(6\). The additive model has rank 3 for these three observed cells, and there are \(n=11\) samples. Hence \(df=11-3=8\), and

s^2=\frac{6}{8}=0.75.

The contrast is the difference of two independent cell means. Its estimated variance and standard error are

\widehat{\operatorname{Var}}(\bar{Y}_{B2}-\bar{Y}_{A2}) =s^2\left(\frac{1}{4}+\frac{1}{3}\right) =0.75\left(\frac{7}{12}\right) =0.4375, \qquad SE=\sqrt{0.4375}=0.6614.

With normal errors and the common-variance model, a 95% interval uses \(t_{0.975,8}=2.306\): \(3\mathbin{\pm}2.306(0.6614)=3\mathbin{\pm}1.525\), giving approximately \([1.475,4.525]\) feature units. Under the model, this interval procedure has 95% repeated-sampling coverage for the cohort contrast. The observed data support a positive adjusted contrast, but the result is conditional on additivity and the error assumptions.

Why the Overlap Determines What Can Be Estimated

A short derivation makes the role of overlap explicit. In the preceding two-cohort, two-platform design, take cohort A/platform 1 as the reference. Write the parameter vector as \(\boldsymbol{\beta}=(\mu,\alpha_B,\gamma_2)^T\). The three observed cell means satisfy

\begin{pmatrix} \bar{Y}_{A1}\\ \bar{Y}_{A2}\\ \bar{Y}_{B2} \end{pmatrix} = \begin{pmatrix} 1&0&0\\ 1&0&1\\ 1&1&1 \end{pmatrix} \begin{pmatrix} \mu\\ \alpha_B\\ \gamma_2 \end{pmatrix} +\text{sampling error}.

The rows of the design matrix are linearly independent, so its rank is 3 and all three parameters in this reference parameterization are estimable. Subtracting the second expected cell mean from the third gives \(\alpha_B\); subtracting the first from the second gives \(\gamma_2\). The bridge cell is what makes those two subtractions possible.

For a contrast of observed cell means, write \(\mathbf{a}^T\bar{\mathbf{Y}}\). Its estimated variance, when the cell means are independent and the pooled variance assumption holds, is \(s^2\sum_j a_j^2/n_j\). This is distinct from a contrast of model coefficients. For an estimable coefficient contrast \(\boldsymbol{\ell}^T\boldsymbol{\beta}\), the corresponding fitted linear-model variance is

\widehat{\operatorname{Var}}(\boldsymbol{\ell}^T\hat{\boldsymbol{\beta}}) =s^2\boldsymbol{\ell}^T(X^TX)^{-}\boldsymbol{\ell},

where \(X\) is the sample-level design matrix and \((X^TX)^{-}\) is a generalized inverse. Here \(\boldsymbol{\ell}=(0,1,0)^T\) represents \(\alpha_B\) in the stated parameterization. It is not the cell-mean contrast vector. In the example, the coefficient variance factor is \(1/4+1/3=7/12\), giving the same variance \(s^2(7/12)=0.4375\) as the direct cell-mean calculation. The two formulas agree when applied to their respective contrasts.

Worked Example: Three Cohorts Connected Through One Platform

Consider a second hypothetical feature measured in these cells: A/platform 1 has values 7, 8, 9; A/platform 2 has 9.5, 10.5; B/platform 2 has 12, 13, 14; and C/platform 1 has 5.5, 6.5. The design is connected through cohort A, which was measured on both platforms. The absent B/platform 1 and C/platform 2 cells do not prevent estimating B-minus-A or C-minus-A under the additive model.

The cell means are 8, 10, 13, and 6. Thus B-minus-A is estimated on platform 2 as \(13-10=3\) feature units, while C-minus-A is estimated on platform 1 as \(6-8=-2\) feature units. The pooled within-cell residual sum of squares is \(2+0.5+2+0.5=5\). There are \(n=10\) samples and four independent observed cell means; the additive model has rank 4, so \(df=10-4=6\) and \(s^2=5/6=0.8333\).

For B-minus-A, the standard error and 95% interval are

SE=\sqrt{\frac{5}{6}\left(\frac{1}{3}+\frac{1}{2}\right)} =\sqrt{\frac{25}{36}}=0.8333, \qquad 3\mathbin{\pm}t_{0.975,6}(0.8333) =3\mathbin{\pm}2.447(0.8333) =[0.9608,5.039].

This interval describes the B-minus-A contrast in feature units under the additive model. The C-minus-A estimate is likewise \(-2\) units, with the same standard error because its two cell sizes are also 3 and 2. The shared A cohort does not make the two estimates identical or change either marginal variance; their joint covariance would matter if a joint comparison were being made.

When the Design Does Not Identify a Cohort Contrast

Worked Example: A Difference That Mixes Biology and Platform

Suppose a hypothetical feature has mean 10 in cohort A measured only on platform 1, and mean 15 in cohort B measured only on platform 2. The observed difference is \(15-10=5\) feature units, but the design has no bridge or other connection between the two cohort-platform combinations.

Under the additive model, the difference equals \((\alpha_B-\alpha_A)+(\gamma_2-\gamma_1)\). The single observed difference of 5 therefore supplies only one equation for two unknown contrasts. For example, a cohort difference of 5 and platform difference of 0, or a cohort difference of 0 and platform difference of 5, both reproduce the observed difference. The cohort contrast is not estimable from these data alone. No standard error or confidence interval for that biological contrast can repair the missing identification; a value can be produced only by imposing additional information or assumptions, such as a known platform offset.

Diagnostics, Alternatives, and Limitations

Before interpreting an adjusted contrast, inspect the cell layout and check that the relevant cohorts and platforms are connected. Then examine residuals from the fitted model for substantial heteroscedasticity, outliers, or systematic patterns. Replication within cells provides information about residual variation; replication alone does not reveal a cohort-by-platform interaction if the design lacks the combinations needed to compare it. With only one bridge cohort, additivity may be essential but difficult to assess. A platform shift that differs by cohort can bias the adjusted contrast.

For a single feature, a transparent alternative is to compare cohort means within a shared platform, as in the worked examples. The additive linear model is more flexible for combining a path through several observed cells and for expressing multiple covariates, but it does not create information absent from the design. If a cell is unobserved, its fitted value is an extrapolation under the model, not a measured mean. The introductory material on ComBat and covariate-preserving correction addresses broader correction strategies; the calculations here concern an explicit feature-level contrast.

Common limitations include the following:

Unbalanced precision: small cell sizes increase the variance contribution \(s^2/n_{cp}\), even when the contrast is estimable.
Unequal residual variances: a single pooled \(s^2\) may be inappropriate if variability differs strongly by cell; the displayed \(t\) intervals then need not have their stated coverage.
Unmodeled interaction: if platform effects vary across cohorts, a single additive platform correction may not represent the data.
Disconnected overlap: cohort differences across separate design components remain confounded with platform differences.
Scope of inference: an adjusted association between cohort and feature is not, by itself, a causal effect of cohort membership.

A common mistake is to subtract the overall observed cohort means and call the result batch-adjusted. If cohort A has observations on two platforms but cohort B has observations only on one, the pooled A mean combines a different platform composition with B's measurements. The correction is to state the model, use the supported within-platform or overlap-path contrast, and report the assumptions and uncertainty. If the design is disconnected, report that the contrast is not identified rather than assigning the raw difference a biological interpretation.

Key takeaway: Partial overlap can identify adjusted contrasts when it connects the cohort-platform design and the additive model is defensible. Estimate contrasts from the observed overlap, use a variance formula matched to the contrast representation, and do not interpret disconnected cohort-platform differences as separable effects.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Cohort A is measured on platforms 1 and 2, while cohort B is measured only on platform 2. Under the additive model, which observed cell means give the B-minus-A contrast?
In the three-cohort example, why is B-minus-A estimable even though B/platform 1 is missing?
A cell-mean contrast uses coefficients \((-1,1)\) on two cell means. State its estimated variance when both cells have sample size \(m\).
If cohort A/platform 1 and cohort B/platform 2 are the only observed cells, can their difference identify a cohort effect separately from a platform effect? Explain.

Solutions. (1) Use \(\bar{Y}_{B2}-\bar{Y}_{A2}\); both means are on platform 2, so the additive platform shift cancels. (2) Cohort A links platforms 1 and 2, making the observed design connected; B-minus-A can be estimated by comparing B and A on platform 2. The result relies on the additive assumption. (3) The estimated variance is \(s^2(1/m+1/m)=2s^2/m\), assuming independent cell means and a common residual variance. (4) No. The observed difference is the sum of the cohort contrast and the platform contrast, so these effects are confounded without further information or assumptions.