Random-Effects Meta-Analysis and Between-Cohort Variance

Why Allow Effects to Vary Across Cohorts?

A genetic association meta-analysis combines estimates from multiple cohorts. A fixed-effect model treats all cohorts as estimating one common true effect, with observed differences arising from sampling error alone. A random-effects model allows the true association effect itself to differ across cohorts. It summarizes those cohort-specific effects through a distribution, whose mean is the overall effect of interest and whose variance describes between-cohort variation.

This distinction matters when cohorts differ in ways that could change the association effect, even after their results have been harmonized to the same allele, phenotype, and effect scale. Random effects do not remove the need for those checks: they provide a model for remaining variation in true effects. In this tutorial, the between-cohort variance is treated as specified, so the focus is on the model and the resulting weights rather than on estimating that variance.

Definition: In a random-effects meta-analysis, cohort \(i\) has a true effect \(\beta_i\), modeled as a draw from a distribution with mean \(\mu\) and between-cohort variance \(\tau^2\). A common model is \(\beta_i\sim N(\mu,\tau^2)\). The quantity \(\mu\) is the mean of the distribution of cohort-specific true effects, and \(\tau^2\) measures their variance around that mean. The observed estimate \(\hat{\beta}_i\) also has sampling error, summarized by its standard error \(SE_i\).

The model separates two sources of variation. The sampling variance \(SE_i^2\) reflects uncertainty in the estimate from cohort \(i\); the between-cohort variance \(\tau^2\) reflects variation among cohort-specific true effects. These are different quantities, even though both contribute to the spread of observed estimates.

From the Model to the Random-Effects Weight

Suppose cohort \(i\) provides an approximately unbiased estimate of its true effect, with sampling variance \(SE_i^2\). A standard working model is \(\hat{\beta}_i\mid\beta_i\sim N(\beta_i,SE_i^2)\). Assume that the cohort estimates are independent, that their standard errors are appropriately estimated, and that the true effects follow the distribution above. Conditional on a true effect, the estimate varies because of sampling; across cohorts, the true effect varies as well.

The marginal variance follows from the law of total variance: the variance of an estimate is its expected conditional variance plus the variance of its conditional expectation. Here, the conditional variance is \(SE_i^2\), and the conditional expectation is \(\beta_i\). Therefore:

\begin{aligned} E(\hat{\beta}_i) &= E\{E(\hat{\beta}_i\mid\beta_i)\}=E(\beta_i)=\mu,\\ \operatorname{Var}(\hat{\beta}_i) &=E\{\operatorname{Var}(\hat{\beta}_i\mid\beta_i)\} +\operatorname{Var}\{E(\hat{\beta}_i\mid\beta_i)\}\\ &=SE_i^2+\tau^2. \end{aligned}

Thus, under the normal model, the marginal estimate has distribution \(\hat{\beta}_i\sim N(\mu,SE_i^2+\tau^2)\). Its random-effects inverse-variance weight is the reciprocal of this marginal variance. For a specified \(\tau^2\), the pooled estimate of \(\mu\) is the weighted mean of the cohort estimates:

w_i^*=\frac{1}{SE_i^2+\tau^2}, \qquad \hat{\mu}_{RE}=\frac{\sum_{i=1}^{k}w_i^*\hat{\beta}_i} {\sum_{i=1}^{k}w_i^*}, \qquad SE(\hat{\mu}_{RE})=\sqrt{\frac{1}{\sum_{i=1}^{k}w_i^*}}.

The standard error formula uses independence of the cohort estimates and treats the variance components used in the weights as fixed. In practice, \(\tau^2\) is usually estimated, which adds a layer of uncertainty and raises method-specific issues. The calculations here condition on a stated value of \(\tau^2\).

Key distinction: A fixed-effect weight is \(1/SE_i^2\). A random-effects weight is \(1/(SE_i^2+\tau^2)\). If \(\tau^2=0\), the random-effects formula reduces to the fixed-effect formula. If \(\tau^2\) is positive, every weight is smaller, and the most precise cohorts generally lose relative weight compared with less precise cohorts.

Worked Examples

Worked Example: Separating the Two Sources of Variance

Suppose a hypothetical cohort has a standard error of \(0.06\) log-odds units per effect allele. Assume the cohort effects have a normal distribution with between-cohort standard deviation \(0.08\) log-odds units per allele. Find the marginal standard deviation of the cohort estimate under the random-effects model.

The sampling variance is \(SE_i^2=0.06^2=0.0036\), in squared log-odds units per allele. The between-cohort variance is \(\tau^2=0.08^2=0.0064\), in the same squared units. Add them to obtain the marginal variance:

SE_i^2+\tau^2=0.0036+0.0064=0.0100.

Taking the square root gives a marginal standard deviation of \(\sqrt{0.0100}=0.10\) log-odds units per allele. This is larger than the sampling standard error alone because observed cohort estimates vary both from sampling error and from differences in the underlying cohort-specific effects. The calculation assumes the stated standard error is valid and that the cohort effect belongs to the specified distribution.

Worked Example: Pooling Three Cohorts With a Stated Variance

Consider three hypothetical, independent cohorts studying the same harmonized genetic variant and phenotype on the log-odds-per-allele scale. Their estimates and standard errors are \((0.20,0.10)\), \((0.10,0.20)\), and \((0.30,0.15)\). Assume a between-cohort variance of \(\tau^2=0.0100\) squared log-odds units per allele. Calculate the random-effects pooled estimate and its standard error.

The model conditions are that the cohort estimates target effects on the same scale, are approximately unbiased with the reported standard errors, and are independent. We take the specified \(\tau^2\) as given. The three random-effects weights are:

\begin{aligned} w_1^*&=\frac{1}{0.10^2+0.0100}=\frac{1}{0.0200}=50.00,\\ w_2^*&=\frac{1}{0.20^2+0.0100}=\frac{1}{0.0500}=20.00,\\ w_3^*&=\frac{1}{0.15^2+0.0100}=\frac{1}{0.0325}=30.77. \end{aligned}

Substitute the weights and estimates into the weighted mean:

\hat{\mu}_{RE} =\frac{50.00(0.20)+20.00(0.10)+30.77(0.30)} {50.00+20.00+30.77} =\frac{21.231}{100.77} \approx 0.2107.

The standard error is:

SE(\hat{\mu}_{RE})=\sqrt{\frac{1}{100.77}} \approx 0.09962 \quad\text{log-odds units per allele}.

Under the usual normal approximation, a 95% confidence interval for the mean effect is \(0.2107\pm1.96(0.09962)\), approximately \((0.01544,\,0.40594)\) log-odds units per allele. In repeated applications of this procedure under its assumptions, 95% of intervals constructed this way would cover the mean effect \(\mu\). This interval concerns the mean of the distribution of cohort effects, not the range of effects expected in a new cohort.

Worked Example: How a Larger Variance Changes Relative Weights

Two hypothetical independent cohorts report genetic effects of \(0.12\) and \(0.32\) log-odds units per allele, with standard errors \(0.08\) and \(0.12\), respectively. Suppose \(\tau^2=0.0400\) squared log-odds units per allele. Find the pooled estimate and compare the relative weights with those from a fixed-effect calculation.

Assume both estimates target the same effect scale, their standard errors correctly describe sampling uncertainty, and the cohorts are independent. Under the stated random-effects variance, the weights are:

\begin{aligned} w_1^*&=\frac{1}{0.08^2+0.0400} =\frac{1}{0.0464}=21.55,\\ w_2^*&=\frac{1}{0.12^2+0.0400} =\frac{1}{0.0544}=18.38. \end{aligned}

The pooled estimate is:

\hat{\mu}_{RE} =\frac{21.55(0.12)+18.38(0.32)}{21.55+18.38} \approx \frac{8.469}{39.93} \approx 0.2121 \quad\text{log-odds units per allele}.

The first cohort receives \(21.55/39.93\approx0.5397\), or about 54.0%, of the total random-effects weight; the second receives about 46.0%. For comparison, fixed-effect weights would be \(1/0.08^2=156.25\) and \(1/0.12^2=69.44\). Their shares are about 69.2% and 30.8%. The positive between-cohort variance therefore makes the weights more similar: it reduces the first cohort’s relative dominance because its greater precision matters less when between-cohort variation is substantial.

For completeness, the random-effects standard error is \(\sqrt{1/39.93}\approx0.1582\) log-odds units per allele. This is larger than the standard error obtained from the fixed-effect weights, because the random-effects calculation includes the stated between-cohort variance in the marginal variances.

What the Pooled Estimate Means

The random-effects pooled estimate targets \(\mu\), the mean of the modeled distribution of cohort-specific true effects. It is not necessarily the true effect in every cohort, nor does it assert that every cohort has the same effect. The value of \(\tau^2\) describes how much the true cohort effects vary around that mean on the chosen effect scale.

The random-effects weights show how the model allocates influence across cohorts. When \(\tau^2=0\), relative weight depends only on sampling precision. As \(\tau^2\) increases, the added variance becomes more important relative to differences in \(SE_i^2\), and the weights tend to become more alike. A cohort with a very small standard error can still be informative, but its precision does not give it as much relative dominance as under a fixed-effect model.

The variance is scale-dependent. If effects are reported in log-odds units per allele, \(\tau^2\) is in squared log-odds units per allele. A variance stated on a different scale cannot be inserted directly into these weights. Likewise, harmonizing alleles and effect scales is essential: variation caused by incompatible coding or estimands is not a substitute for modeled between-cohort variation.

Common Mistakes

Adding standard errors instead of variances. The marginal variance is \(SE_i^2+\tau^2\), not \(SE_i+\tau\). Add the variances, then take a square root if a standard deviation is needed.
Using fixed-effect weights while claiming a random-effects analysis. With positive \(\tau^2\), each weight uses \(SE_i^2+\tau^2\) in its denominator. The fixed-effect formula is recovered only when \(\tau^2=0\).
Calling \(\tau^2\) sampling error. It represents variation among cohort-specific true effects. Sampling uncertainty for a particular estimate is represented by \(SE_i^2\).
Interpreting the pooled estimate as every cohort’s effect. The pooled estimate targets the mean \(\mu\); cohort-specific true effects may differ from it.
Using incompatible effect scales or dependent cohorts without adjustment. The displayed formulas require comparable estimates and, for the stated standard error, independence. If cohorts overlap, the estimates may be correlated and the simple independent-weight calculation is not appropriate.
Treating an assumed variance as an observed fact. A calculation conditional on a stated \(\tau^2\) illustrates the model; it does not establish that this is the correct value for a real analysis.

Key takeaway: Random-effects meta-analysis adds the between-cohort variance to each cohort’s sampling variance, giving weights \(1/(SE_i^2+\tau^2)\). The resulting weighted mean estimates the mean cohort effect, while the size of \(\tau^2\) controls how strongly sampling precision determines each cohort’s relative influence.

Check Your Understanding

Answer each question using the random-effects model and its assumptions.

In words, distinguish \(SE_i^2\) from \(\tau^2\).
If a cohort has \(SE_i=0.10\) and the specified \(\tau^2=0.03\), what is its random-effects weight?
What does the pooled estimate \(\hat{\mu}_{RE}\) target: a common effect in every cohort or the mean of cohort-specific true effects?
Why do relative weights tend to become more similar as \(\tau^2\) increases?
Name two conditions needed for the displayed pooled standard error formula to be appropriate.