Meta-Analysis With Overlapping Cohorts

Why Overlapping Cohorts Change Meta-Analysis

A meta-analysis combines estimates from several cohorts. The usual inverse-variance calculation treats the estimates as independent. That assumption fails when some of the same participants contribute to more than one estimate: a participant’s genotype and outcome information can affect both estimates, inducing sampling covariance. The estimates can therefore contain partly duplicated information, even when they are reported as separate cohort results.

The key adjustment is to represent the estimates’ sampling variances and covariances together in a covariance matrix. The resulting generalized least-squares calculation uses the dependence when choosing weights and estimating uncertainty. Positive covariance usually means that an independence-based calculation overstates the amount of information and understates the standard error. The direction and size of the error depend on the covariance and on the weights; overlap alone does not specify either.

The calculations below assume the effects have already been defined on a common scale and for a common estimand. This is necessary for combining them, but overlap adjustment does not itself harmonize effect scales or alleles.

Model and Generalized Least Squares

Definition: Let $y=(\hat{\beta}_1,\ldots,\hat{\beta}_K)^\mathsf{T}$ be $K$ cohort-specific estimates of a common fixed effect $\theta$. Write $y=\theta\mathbf{1}+\varepsilon$, where $E(\varepsilon)=0$ and $\operatorname{Cov}(\varepsilon)=\Sigma$. The diagonal entries of $\Sigma$ are the estimates’ sampling variances, and its off-diagonal entries are their sampling covariances. If $\Sigma$ is known and positive definite, the generalized least-squares estimator is $$ \hat{\theta}_{\mathrm{GLS}}= \frac{\mathbf{1}^\mathsf{T}\Sigma^{-1}y} {\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1}}, \qquad \operatorname{Var}(\hat{\theta}_{\mathrm{GLS}})= \frac{1}{\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1}}. $$

The fixed-effect model assumes that each cohort estimates the same underlying effect and that the estimates are unbiased for it. The covariance matrix must correctly describe their sampling dependence. A multivariate normal approximation for $y$ supports normal-based confidence intervals and tests; for large association studies this is often a useful approximation, but it does not repair a misspecified covariance matrix. In practice, $\Sigma$ is usually estimated, so the displayed variance treats that estimate as if it were known.

When cohorts are independent, all off-diagonal entries are zero. The formula then reduces to the familiar inverse-variance weighted estimate. With overlap, the off-diagonal entries carry information that cannot be recovered from the standard errors alone. For two estimates with variances $v_1,v_2$ and covariance $c$, the GLS expression simplifies to:

\hat{\theta}_{\mathrm{GLS}}= \frac{(v_2-c)\hat{\beta}_1+(v_1-c)\hat{\beta}_2} {v_1+v_2-2c}, \qquad \operatorname{Var}(\hat{\theta}_{\mathrm{GLS}})= \frac{v_1v_2-c^2}{v_1+v_2-2c}.

These expressions require a valid covariance matrix: $v_1>0$, $v_2>0$, and $c^2<v_1v_2$ for a positive-definite two-estimate matrix. The weights are not necessarily both positive. If the estimates have very different precisions or the covariance is large, a GLS weight can be negative; this is a mathematical consequence of the dependence structure, not automatically a calculation error. It does call for careful checking of the covariance inputs and model.

Deriving the GLS Formula

For any linear estimator $a^\mathsf{T}y$ that is unbiased for $\theta$, the weights must satisfy $a^\mathsf{T}\mathbf{1}=1$. Its variance is $a^\mathsf{T}\Sigma a$. Minimize this variance subject to the unbiasedness constraint. The Lagrangian is $a^\mathsf{T}\Sigma a-2\lambda(a^\mathsf{T}\mathbf{1}-1)$. Differentiating with respect to $a$ gives $\Sigma a=\lambda\mathbf{1}$, so $a=\lambda\Sigma^{-1}\mathbf{1}$. Applying the constraint yields $\lambda=1/(\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1})$. Therefore,

a=\frac{\Sigma^{-1}\mathbf{1}} {\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1}}, \qquad a^\mathsf{T}\Sigma a= \frac{1}{\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1}}.

Thus GLS is the minimum-variance linear unbiased combination under the stated model. This derivation also shows why it is not enough to insert an “effective sample size” into an independent-study formula: dependence changes the weights as well as the variance.

Worked Example: Two Overlapping Cohorts

Suppose two hypothetical cohorts estimate a genetic effect in log-odds-ratio units. Their estimates are $\hat{\beta}_1=0.20$ and $\hat{\beta}_2=0.16$, with standard errors $0.050$ and $0.040$. A joint covariance calculation gives correlation $0.50$. Find the overlap-adjusted estimate and standard error, then compare them with the independence calculation.

The variances are $v_1=0.050^2=0.0025$ and $v_2=0.040^2=0.0016$. The covariance is $c=0.50(0.050)(0.040)=0.0010$. The covariance matrix is valid because $c^2=0.000001$ is less than $v_1v_2=0.000004$.

The denominator in the two-estimate formula is $v_1+v_2-2c=0.0025+0.0016-0.0020=0.0021$. Thus the weights are $(v_2-c)/0.0021=0.0006/0.0021=0.2857$ and $(v_1-c)/0.0021=0.0015/0.0021=0.7143$. They sum to one, and

\hat{\theta}_{\mathrm{GLS}} =(0.2857)(0.20)+(0.7143)(0.16)=0.1714.

The variance is $(v_1v_2-c^2)/(v_1+v_2-2c)=(0.000004-0.000001)/0.0021=0.001429$, giving $SE=\sqrt{0.001429}=0.03780$ log-odds-ratio units.

If the cohorts were incorrectly treated as independent, the inverse-variance weights would be $400/(400+625)=0.3902$ and $625/(400+625)=0.6098$. The reported standard error would be $1/\sqrt{400+625}=0.03123$, smaller than the overlap-adjusted value. In this example the independence calculation is too confident because it counts correlated information as if it were independent.

Obtaining and Checking the Covariance

The covariance should be estimated from the mechanism that generated the estimates, not guessed from the overlap count alone. One general approach uses influence functions. If an estimate can be approximated by its target plus the sum of individual influence contributions, then the cross-covariance of two estimates is estimated by summing the products of their contributions over participants shared by the two analyses. Contributions must be defined on the scale and normalization appropriate to the estimators. For regression estimates, a joint sandwich calculation from the estimating equations is another way to obtain the cross-covariance.

A simple overlap fraction is not a universal correlation estimate. The relationship depends on the outcome, genotype distribution, model, covariate adjustment, and the way each estimate is calculated. Two analyses can share participants but have a modest covariance; conversely, dependence can also arise from shared controls, related samples, or other shared data structures even if the nominal cohort labels differ.

Worked Example: Covariance From Shared-Participant Contributions

Suppose two hypothetical association estimates share three participants. For illustration, their already normalized influence contributions to the two estimates are as follows, in log-odds-ratio units: $(0.020,0.010)$, $(-0.010,-0.030)$, and $(0.010,0.020)$. Assume the remaining participants are distinct across cohorts, and that the influence-based variance estimates for the two effects are $0.0025$ and $0.0036$.

The estimated covariance contribution from the shared participants is the sum of the paired products:

\hat{c}=(0.020)(0.010)+(-0.010)(-0.030)+(0.010)(0.020) =0.0002+0.0003+0.0002=0.0007.

This produces the covariance matrix with diagonal entries $0.0025$ and $0.0036$, and off-diagonal entries $0.0007$. It is admissible because $0.0007^2=0.00000049<0.0025(0.0036)=0.000009$. Its implied correlation is $0.0007/\sqrt{0.0025(0.0036)}=0.2333$. The paired influence products illustrate the calculation; in an actual analysis the influence contributions and their variance estimates must come from the fitted models and observed data.

For estimates $0.12$ and $0.08$, the GLS denominator is $0.0025+0.0036-2(0.0007)=0.0047$. The weights are $(0.0036-0.0007)/0.0047=0.6170$ and $(0.0025-0.0007)/0.0047=0.3830$, giving $0.6170(0.12)+0.3830(0.08)=0.1047$. The variance is $(0.0025(0.0036)-0.0007^2)/0.0047=0.001811$, so the standard error is $0.04255$ log-odds-ratio units.

Worked Example: Three Equally Precise, Correlated Estimates

Consider three hypothetical estimates of a common effect, all in standardized effect units: $0.30$, $0.10$, and $0.20$. Each has standard error $0.20$, and each pair has correlation $0.25$. Thus each variance is $0.04$ and each covariance is $0.25(0.20)(0.20)=0.01$. The covariance matrix is positive definite: its diagonal is $0.04$, its common off-diagonal is $0.01$, and its eigenvalues are $0.03, 0.03,$ and $0.06$, all positive.

By symmetry, GLS assigns equal weights, so the pooled estimate is $(0.30+0.10+0.20)/3=0.20$. Its variance is the variance of the mean, including all three covariance terms for distinct pairs:

\operatorname{Var}(\hat{\theta})= \frac{3(0.04)+6(0.01)}{3^2} =\frac{0.18}{9}=0.020, \qquad SE=\sqrt{0.020}=0.1414.

An independence calculation would instead report variance $0.04/3=0.01333$ and standard error $0.1155$. Using a normal critical value of $1.96$, the covariance-adjusted 95% interval is $0.20\mathbin{\pm}1.96(0.1414)=[-0.0772,0.4772]$. Under the stated model, this procedure has 95% coverage in repeated sampling; it does not mean there is a 95% probability that the fixed effect lies in this particular interval.

Random Effects, Alternatives, and Diagnostics

Overlap adjustment addresses sampling dependence; it does not decide whether a fixed-effect or random-effects model is appropriate. That choice concerns how cohort effects vary, as discussed in the earlier material on choosing fixed- and random-effects models. If cohort-specific effects are modeled as independent deviations with variance $\tau^2$, a common covariance specification is $\Sigma_{\text{sampling}}+\tau^2 I$: the estimated sampling covariance remains, while the random-effects variance is added to each diagonal. The form of this addition depends on the random-effects assumptions. It is not correct to erase the sampling covariances when moving to a random-effects model.

When reliable covariance estimates are unavailable, possible alternatives include removing duplicated participants and re-estimating effects, selecting one estimate from the overlapping analyses, or performing a joint individual-level analysis if appropriate data and a suitable model are available. These approaches have different costs: removing participants may reduce precision, selecting one result discards some data, and joint analysis requires harmonized inputs and modeling assumptions. Treating the estimates as independent is not a neutral substitute for missing covariance information.

Before interpreting an overlap-adjusted result, check that the covariance matrix is symmetric, has the intended marginal variances, and is positive semidefinite; a non-positive eigenvalue can make inversion unstable or impossible. Also check that the reported overlap and covariance calculation match the actual sample inclusion, that all estimates target the same effect, and that the estimates are not nearly redundant. Near-singularity can produce very large or unstable weights, making small covariance errors consequential. Inspect residuals against the fitted common effect for evidence of study-specific discrepancies; covariance adjustment does not eliminate genuine heterogeneity or model misspecification.

Common mistake: “The cohorts overlap, but each estimate has its own standard error, so inverse-variance weighting is still valid.” Separate standard errors give only the diagonal of $\Sigma$. If the off-diagonal covariance is positive, the independent-study standard error can be too small, as in the examples above. Estimate or otherwise justify the covariance, use it in the combination, and report how it was obtained.

Check Your Understanding

Use the covariance formulas and assumptions from this tutorial. Solutions follow the questions.

Two estimates have variances $0.04$ and $0.09$, and covariance $0.03$. Is their covariance matrix positive definite?
For the estimates in Question 1, with values $0.10$ and $0.20$, calculate the two GLS weights and pooled estimate.
Three equal-variance estimates have variance $0.01$ each and pairwise covariance $0.002$. What is the variance of their equally weighted mean?
Why can the fraction of participants shared by two cohorts not, by itself, determine the covariance of their genetic association estimates?

Solutions. (1) Yes: $0.03^2=0.0009<0.04(0.09)=0.0036$, so the two-by-two covariance matrix is positive definite. (2) The denominator is $0.04+0.09-2(0.03)=0.07$. The weights are $(0.09-0.03)/0.07=0.8571$ and $(0.04-0.03)/0.07=0.1429$, giving $0.8571(0.10)+0.1429(0.20)=0.1143$. (3) The variance is $\{3(0.01)+6(0.002)\}/9=0.042/9=0.004667$. (4) Covariance also depends on the estimators, outcomes, genotypes, covariate adjustments, and shared data structure; overlap proportion alone does not encode these quantities.