Random-Effects Meta-Analysis With Estimated Tau-Squared

Why Random-Effects Weights Change the Pooling

Cohort estimates of a genetic association can differ because of sampling error and because the underlying effects may vary across cohorts. A fixed-effect analysis accounts for sampling error but assumes one common true effect. A random-effects analysis allows true cohort effects to vary around a mean effect. Its weights therefore include both each cohort’s sampling variance and an estimated between-cohort variance.

The earlier material on I-squared and tau-squared introduced measures of heterogeneity. Here the focus is using an estimate of tau-squared to calculate random-effects weights and pool the effects. The calculation is useful only when the cohort estimates are independent and reported on a comparable effect scale.

Definition: Let $y_i$ be the estimated effect in cohort $i$, with standard error $SE_i$ and estimated sampling variance $v_i=SE_i^2$, for $i=1,\ldots,k$. A random-effects model assumes $$ y_i=\theta_i+\varepsilon_i,\qquad \theta_i\sim N(\mu,\tau^2),\qquad \varepsilon_i\sim N(0,v_i), $$ where $\mu$ is the mean of the distribution of true cohort effects and $\tau^2$ is their between-cohort variance. The sampling errors and true effects are assumed independent, and cohorts are assumed independent of one another. Consequently, $y_i$ has mean $\mu$ and variance $v_i+\tau^2$.

The normal model is often an approximation for association estimates, such as log odds ratios or regression coefficients. It assumes a common mean effect across the distribution of cohorts, while allowing cohort-specific true effects to differ. The variance $\tau^2$ is on the squared effect scale: if effects are log odds ratios, for example, $\tau^2$ is in squared log-odds-ratio units.

Estimating Between-Cohort Variance

The DerSimonian–Laird (DL) method estimates $\tau^2$ by a method-of-moments calculation based on Cochran’s Q. First calculate the fixed-effect weights $w_i=1/v_i$, the fixed-effect weighted mean $\bar{y}_{FE}$, and Q:

\bar{y}_{FE}=\frac{\sum_{i=1}^k w_i y_i}{\sum_{i=1}^k w_i}, \qquad Q=\sum_{i=1}^k w_i(y_i-\bar{y}_{FE})^2.

Define $C=\sum_i w_i-\frac{\sum_i w_i^2}{\sum_i w_i}$. The untruncated DL estimate is $\{Q-(k-1)\}/C$. Because a variance cannot be negative, the estimate used in the ordinary DL procedure is truncated at zero:

\hat{\tau}^{\,2}_{DL}=\max\left\{0,\frac{Q-(k-1)}{C}\right\}.

The resulting random-effects weight for cohort $i$ is $w_i^*=1/(v_i+\hat{\tau}^{\,2}_{DL})$. The pooled estimate and its conventional plug-in standard error are

\bar{y}_{RE}=\frac{\sum_{i=1}^k w_i^*y_i}{\sum_{i=1}^k w_i^*}, \qquad SE(\bar{y}_{RE})=\sqrt{\frac{1}{\sum_{i=1}^k w_i^*}}.

These formulas use the estimated between-cohort variance as though it were known. The conventional standard error describes uncertainty conditional on that plug-in estimate; it does not fully account for the fact that $\tau^2$ was estimated, which can matter especially when there are few cohorts.

Justification: Under the model, the expectation of Q is $(k-1)+\tau^2 C$. To see why, let $W$ be the diagonal matrix of fixed-effect weights, let $w$ be the vector of those weights, and set $P=W-ww^\mathsf{T}/\sum_i w_i$. Then $Q=y^\mathsf{T}Py$, and $P$ removes a common mean, so $P\mathbf{1}=0$. Since the covariance matrix of $y$ is $V+\tau^2 I$, with $V=\operatorname{diag}(v_i)$, it follows that $E(Q)=\operatorname{tr}(PV)+\tau^2\operatorname{tr}(P)$. The first trace is $k-1$: $\operatorname{tr}(WV)=k$, and the subtracted term has trace 1. The second trace is $C$. Equating observed Q to this expectation and solving for $\tau^2$ gives the untruncated DL expression. Truncating at zero enforces the variance constraint.

Worked Calculations

Worked Example: Equal Sampling Variances and Clear Heterogeneity

Suppose three hypothetical cohorts estimate the same genetic association on the log-odds-ratio scale. Their estimates are $0.10$, $0.30$, and $0.50$, each with $SE_i=0.10$. Assume the estimates are independent and comparable, and use the normal approximation described above.

Each sampling variance is $v_i=0.10^2=0.01$, so each fixed-effect weight is $w_i=100$. The fixed-effect mean is $$ \bar{y}_{FE}=\frac{100(0.10)+100(0.30)+100(0.50)}{300}=0.30. $$ The heterogeneity statistic is $$ Q=100(0.10-0.30)^2+100(0.30-0.30)^2+100(0.50-0.30)^2=8. $$ Here $k-1=2$, and $C=300-(100^2+100^2+100^2)/300=200$. Therefore, $$ \hat{\tau}^{\,2}_{DL}=\max\left\{0,\frac{8-2}{200}\right\}=0.0300. $$

Each random-effects weight is $1/(0.01+0.03)=25$. Thus $\bar{y}_{RE}=0.30$, and $SE(\bar{y}_{RE})=\sqrt{1/75}=0.1155$. A 95% normal-approximation confidence interval is $0.30\pm1.96(0.1155)=(0.0737,0.5263)$ log-odds-ratio units. It estimates the mean effect across the modeled distribution of cohorts; under repeated sampling and the model, this interval procedure has approximately 95% coverage of that mean.

The fixed-effect standard error is $\sqrt{1/300}=0.0577$, giving a narrower 95% interval of $0.30\pm1.96(0.0577)=(0.1868,0.4132)$. In this example the pooled point estimates happen to agree, but allowing for between-cohort variance reduces the weights and increases the estimated uncertainty.

Worked Example: Unequal Standard Errors and Changed Weights

Consider hypothetical log-odds-ratio estimates $y_1=0.20$, $y_2=0.50$, and $y_3=0.10$, with standard errors $0.10$, $0.20$, and $0.10$. Assume independent, comparable estimates. The sampling variances are $0.01$, $0.04$, and $0.01$, giving fixed weights $100$, $25$, and $100$.

The fixed-effect mean is $$ \bar{y}_{FE}=\frac{100(0.20)+25(0.50)+100(0.10)}{225}=0.1889. $$ The deviations from this mean are $1/90$, $28/90$, and $-8/90$. Hence $$ Q=100(1/90)^2+25(28/90)^2+100(-8/90)^2=\frac{29}{9}=3.2222. $$ Also, $C=225-(100^2+25^2+100^2)/225=133.3333$. The DL estimate is $$ \hat{\tau}^{\,2}_{DL}=\frac{3.2222-2}{133.3333}=0.009167. $$

The random-effects weights are $1/(0.01+0.009167)=52.1739$ for cohorts 1 and 3, and $1/(0.04+0.009167)=20.3390$ for cohort 2. Substitution gives $$ \bar{y}_{RE}=\frac{52.1739(0.20)+20.3390(0.50)+52.1739(0.10)}{52.1739+20.3390+52.1739}=0.2071. $$ The standard error is $\sqrt{1/124.6868}=0.08956$, and the 95% normal-approximation interval is $0.2071\pm1.96(0.08956)=(0.0316,0.3826)$ log-odds-ratio units. By comparison, the fixed-effect standard error is $\sqrt{1/225}=0.06667$, and its interval is $0.1889\pm1.96(0.06667)=(0.0582,0.3196)$. The more imprecise second cohort retains less weight, but less than it would receive under fixed effects because the random-effects adjustment adds $\hat{\tau}^{\,2}_{DL}$ to every sampling variance.

Worked Example: A Zero Estimate at the Boundary

Suppose three hypothetical cohorts have equal fixed-effect weights of $37.5$, so each has sampling variance $1/37.5=0.0266667$. Let their estimates be $0.20-a$, $0.20$, and $0.20+a$, where $a=\sqrt{0.02/75}=0.0163299$. Assume again that estimates are independent and on the same effect scale.

Their fixed-effect mean is $0.20$. Because the deviations are $-a,0,a$, $$ Q=37.5\{(-a)^2+0^2+a^2\}=75a^2=0.02. $$ For equal weights, $C=112.5-(3\times37.5^2)/112.5=75$. The untruncated calculation is $$ \frac{Q-(k-1)}{C}=\frac{0.02-2}{75}=\frac{-1.98}{75}=-0.0264. $$ Therefore $\hat{\tau}^{\,2}_{DL}=\max(0,-0.0264)=0$. The random-effects weights then equal the fixed-effect weights; the pooled estimate is $0.20$, and its conventional standard error is $\sqrt{1/112.5}=0.09428$. This boundary result means the DL procedure estimates no additional variance for this data set. It does not establish that the true between-cohort variance is exactly zero.

Interpretation, Diagnostics, and Alternatives

Random-effects weights are $1/(v_i+\hat{\tau}^{\,2}_{DL})$, so a positive estimated $\tau^2$ generally makes weights more similar than fixed-effect weights. As $\tau^2$ grows, the most precise cohorts lose relative dominance. The random-effects pooled estimate can consequently differ from the fixed-effect estimate, particularly when cohort estimates and standard errors differ.

Check the calculation and its inputs before interpreting the result. A negative untruncated estimate should be truncated, not reported as a negative variance. A very large estimated variance, a cohort estimate far from the others, or major changes in the pooled estimate when one cohort is removed can indicate sensitivity to the data or model. Q is a measure of weighted disagreement, not a diagnosis of its cause. Also verify that effects are coded in the same direction, use the same scale, and come from non-overlapping cohorts; dependence or inconsistent scales invalidate the stated variance calculations.

A fixed-effect analysis is the main comparison: it uses weights $1/v_i$ and targets a common true effect, whereas the random-effects analysis allows a distribution of true cohort effects and targets its mean. If the random-effects assumption is unsuitable, changing the weights does not repair the model. Other variance estimators, including restricted maximum likelihood and Paule–Mandel estimation, are alternatives to DL, especially when the method-of-moments estimate is unstable. They can yield different variance estimates and pooled results. With few cohorts, estimating $\tau^2$ is intrinsically difficult, and the ordinary plug-in interval may understate uncertainty because it treats that estimate as fixed.

Common pitfall: “The random-effects interval is wider, so the cohort effects must all be different.” This is not justified. A positive estimated $\tau^2$ represents estimated dispersion under a model; it does not show that every cohort has a distinct effect. Conversely, a DL estimate of zero does not prove identical true effects. Interpret the pooled estimate and its uncertainty alongside the model assumptions and the observed cohort estimates.

Check Your Understanding

Try the questions first, then compare your reasoning with the solutions.

In the random-effects model, what two sources of variance contribute to the marginal variance of a cohort estimate?
For four cohorts, suppose $Q=5.0$ and $C=30$. Calculate the DL estimate of $\tau^2$.
If $\hat{\tau}^{\,2}_{DL}>0$, how does the random-effects weight compare with the fixed-effect weight for a cohort with sampling variance $v_i$?
Why does a DL estimate of zero not demonstrate that all cohorts have the same true effect?

Solutions. (1) The sampling variance $v_i$ and between-cohort variance $\tau^2$, so the marginal variance is $v_i+\tau^2$. (2) Since $k-1=3$, $\hat{\tau}^{\,2}_{DL}=\max\{0,(5.0-3)/30\}=0.06667$. (3) The random-effects weight $1/(v_i+\hat{\tau}^{\,2}_{DL})$ is smaller than the fixed-effect weight $1/v_i$, because its denominator is larger. (4) The estimate is a nonnegative, data-based method-of-moments estimate and can land on the zero boundary. That outcome indicates no additional variance estimated by this procedure for the observed data; it does not establish that the population variance is exactly zero.

Random-Effects Meta-Analysis With Estimated Tau-Squared

What You'll Learn