Pooling Cross-Cohort Estimates With Meta-Analysis

Why Pool Cohort Estimates?

Independent cohorts can each provide an estimate of the same scientific effect, but their estimates may have different precision. A small cohort or a noisy assay often yields a larger standard error than a larger, more precise cohort. Pooling summarizes the information across cohorts while giving more influence to estimates with greater precision. The aim is not to make different scientific questions equivalent: the estimates must first refer to a sufficiently common target and effect scale.

For example, a validation analysis might estimate the association between a molecular feature and an outcome separately in several cohorts. If the effect is expressed on the same scale in each cohort, inverse-variance weighting provides a transparent way to combine those estimates and quantify uncertainty. A pooled estimate is a summary; it does not by itself establish that every cohort has the same effect.

Conditions for a Meaningful Pool

The formula is straightforward, but its assumptions determine whether its output answers a meaningful question. Cohort estimates should target the same parameter, or parameters close enough that a common-effect summary is the intended target. They must be expressed on the same scale, with compatible coding and direction. For instance, an odds ratio and a log odds ratio cannot be averaged directly; the odds ratios must first be expressed on the log scale, or the estimates must otherwise be put on a common scale.

Each \(\hat{\theta}_i\) should be approximately unbiased for the target effect, and its reported \(SE_i\) should reasonably quantify its sampling uncertainty. For the standard error formula above, the cohort estimates must also be statistically independent, or their dependence must be accounted for. Independent participant samples often support this assumption, but shared participants, overlapping controls, or correlated data can violate it. The usual confidence interval additionally relies on an approximately normal sampling distribution for the pooled estimate; this is often reasonable when estimates are based on adequate information, but can be poor with sparse outcomes or very small samples.

The inverse-variance formula is commonly introduced under a common-effect model: each cohort estimates one shared value \(\theta\), and its sampling error has variance \(SE_i^2\). This is a working model for pooling, not proof that effects really are identical across populations. If cohorts target different populations or effects vary substantially, the pooled number may be a difficult-to-interpret average. Assessing between-cohort heterogeneity requires separate attention.

How the Weighting Formula Arises

Suppose the cohort estimates are unbiased for a common \(\theta\), independent, and have variances \(v_i=SE_i^2\). Consider a linear summary \(\sum_i a_i\hat{\theta}_i\) whose coefficients sum to one, so it remains unbiased for \(\theta\). Independence gives its variance as \(\sum_i a_i^2v_i\). The coefficients that minimize this variance are proportional to \(1/v_i\).

To see this, minimize \(\sum_i a_i^2v_i\) subject to \(\sum_i a_i=1\). The Lagrangian is \(L=\sum_i a_i^2v_i-\lambda(\sum_i a_i-1)\). Differentiating with respect to \(a_i\) gives \(2a_iv_i-\lambda=0\), so \(a_i=\lambda/(2v_i)\). Enforcing the sum constraint gives \(\lambda/2=1/\sum_i(1/v_i)\), and therefore \(a_i=(1/v_i)/\sum_j(1/v_j)\). Substituting these coefficients into the variance yields \(1/\sum_i(1/v_i)\). Thus the normalized inverse-variance weights produce the stated pooled estimate and standard error.

A confidence interval can then be formed as \(\hat{\theta}_{\mathrm{pool}}\pm z^*SE_{\mathrm{pool}}\), where \(z^*\) is the appropriate standard-normal critical value under the normal approximation. For a two-sided 95% interval, \(z^*\) is approximately 1.96. The interval describes uncertainty about the pooled target under the model and assumptions; it is not a range guaranteed to contain every cohort-specific effect.

Reading the Pooled Estimate and Its Uncertainty

Weights depend on standard errors, not on the direction or size of the estimates. A cohort with a standard error half as large has four times the inverse-variance weight, because \(1/(SE/2)^2=4/SE^2\). This can make the pooled result close to a highly precise cohort estimate even when several less precise cohorts point elsewhere. The weighted average should therefore be reported alongside the cohort-specific estimates and their uncertainty, rather than replacing them.

The pooled confidence interval quantifies uncertainty in the pooled estimate under the assumed model. If a 95% interval excludes zero on a difference or log-ratio scale, it indicates incompatibility with zero at the corresponding two-sided level under the model; it does not state the probability that the null hypothesis is true. If an interval for a ratio scale is produced by exponentiation, the null value is 1 rather than 0.

When Pooling Goes Wrong

Several checks should precede a pooled calculation. Confirm that the effect definitions, contrast directions, outcome coding, and measurement units agree. Check that each standard error refers to the estimate being pooled and that the cohort estimates are independent. Finally, review the cohort estimates and intervals themselves: a pooled result can conceal meaningful differences, and a narrow pooled interval does not demonstrate that effects are homogeneous.

A frequent mistake is to treat inverse-variance weighting as a repair for incompatible estimates. It is not. If one cohort reports a difference of treatment minus control and another reports control minus treatment, their estimates have opposite sign conventions and must be harmonized before pooling. Similarly, a per-standard-deviation effect and a per-unit effect are not directly interchangeable. A numerical average of unlike quantities has no coherent interpretation, regardless of how precisely the weights are calculated.

Another mistake is to weight by sample size alone. Sample size can be a rough proxy for precision in some settings, but precision also depends on outcome variability, event frequency, design, and estimation method. When standard errors are available and valid, inverse-variance weights directly use the estimated sampling variance. Equal weighting is also an alternative, but it generally sacrifices precision when cohort estimates have substantially different standard errors. Pooling individual participant data is a different approach that can permit common modeling and covariate adjustment, but it requires compatible data and a suitable analysis plan; it is not interchangeable with combining reported summary estimates.

If participants overlap across cohorts, the estimates may be correlated. Then the variance of a weighted sum includes covariance terms, not just the sum of weighted variances, and the independence-based standard error can be too small or too large. Likewise, if cohort effects differ, a single common-effect estimate may not describe the distribution of effects. In these circumstances, report the cohort-specific results and use methods that explicitly address dependence or between-cohort variation rather than presenting the simple pooled interval as if its assumptions held.