Choose the Model by Choosing the Target
A meta-analysis can answer different scientific questions even when it combines the same cohort estimates. One question is whether cohorts estimate a single shared genetic association effect. Another is what the average association effect is across a population of cohorts whose effects may differ. These questions correspond to different estimands, so the model choice should begin with the scientific target rather than a test result.
For a genetic association, the effect might be a log odds ratio per allele, a regression coefficient for gene expression, or a difference in a quantitative trait per allele. The effect scale and outcome must be aligned across cohorts before estimates can be meaningfully combined. This tutorial assumes estimates are on a common scale, with standard errors that adequately describe their sampling uncertainty.
These targets are not interchangeable. A common effect is appropriate when the scientific claim concerns a shared association under a defensible common-effect assumption. A mean effect is appropriate when cohorts represent a broader set of settings and their true effects may vary, but the average across that set is scientifically meaningful. Neither model automatically estimates a universal biological effect: the target depends on which cohorts and populations the analysis represents.
Models and Their Assumptions
Let \(y_i\) be the estimated genetic association effect in cohort \(i\), with standard error \(SE_i\), for \(i=1,\ldots,k\). Treat the sampling variance \(SE_i^2\) as known or estimated sufficiently accurately for the meta-analysis. The usual fixed-effect model is
Here \(\theta\) is the common true effect, and sampling errors are assumed independent across cohorts. This model requires more than similar-looking estimates: it assumes the underlying true effect is the same in all cohorts, on the specified effect scale and for the specified outcome and genetic model. If cohorts overlap in participants, independence may fail; the standard formulas below then need adjustment.
The random-effects model instead separates sampling error from true-effect variation:
The cohort-specific true effect is \(\theta_i\); \(\mu\) is the mean of the distribution of true effects, and \(\tau^2\) is their between-cohort variance. The model assumes that effects are drawn from a distribution represented by this mean and variance, and that sampling errors are independent of the true effects and of one another. Marginally, \(y_i\) has variance \(SE_i^2+\tau^2\). A random-effects model does not explain why effects differ, and its mean is meaningful only relative to the distribution of cohorts the analysis is intended to represent.
Under the fixed-effect model, inverse-variance weighting gives
For random effects, if \(\tau^2\) were known, the corresponding weights would use total variance:
In practice, \(\tau^2\) is estimated, and different standard methods can produce different estimates, especially when there are few cohorts. The formulas show the central distinction: when estimated heterogeneity is positive, random-effects weights are less dominated by the most precise cohort. The uncertainty calculation must account for the chosen estimator and the estimation of \(\tau^2\), rather than treating an estimated variance as known without qualification.
A Short Justification for the Weights
The inverse-variance weights follow from estimating a common mean from independent normal observations with known variances. For the fixed-effect model, the log likelihood, apart from constants, is proportional to the negative of
Differentiating this expression with respect to \(\theta\), setting the derivative to zero, and solving gives
Because independent estimates have variances \(SE_i^2\), the variance of a weighted average with weights proportional to \(1/SE_i^2\) is \(1/\sum_i1/SE_i^2\). Under the random-effects model, the marginal variance of \(y_i\) is \(SE_i^2+\tau^2\), which yields the corresponding total-variance weights when \(\tau^2\) is known.
Worked Examples: Matching Model to the Scientific Question
Worked Example: A Shared Effect Is the Target
Suppose three hypothetical, independent cohorts estimate the same per-allele log odds ratio for a prespecified outcome. Their estimates are \(0.10\), \(0.12\), and \(0.08\), each with standard error \(0.05\). Assume the effect scale and outcome are aligned, and the scientific target is a single association effect shared by these cohorts. The fixed-effect model matches that target, subject to its common-effect assumption.
Each cohort has weight \(1/0.05^2=400\). Therefore,
The estimated standard error is \(\sqrt{1/1200}=0.0289\). On the log odds ratio scale this is \(0.100\) with standard error \(0.0289\); exponentiating the estimate gives an odds ratio of \(e^{0.100}=1.105\) per allele. The calculation does not prove that all cohorts share the same true effect. The model is justified here by the stated target and assumptions, not by the numerical closeness of three estimates alone.
Worked Example: Cohort Effects Vary Around a Mean
Now suppose three hypothetical, independent cohorts estimate a per-allele log odds ratio in distinct populations: \(0.05\), \(0.25\), and \(0.45\), each with standard error \(0.10\). Assume the aim is the mean effect across a distribution of comparable target populations, and that variation among their true effects is plausible. The random-effects target is then appropriate to the question.
For illustration, estimate \(\tau^2\) using the DerSimonian–Laird method. The fixed-effect estimate is \(0.25\), and the heterogeneity statistic is
With \(k=3\), \(df=k-1=2\). The denominator in the DerSimonian–Laird estimate is \(C=\sum_iw_i-\sum_iw_i^2/\sum_iw_i=300-30000/300=200\), where each fixed-effect weight is \(100\). Thus \(\hat{\tau}^2=\max\{0,(Q-df)/C\}=(8-2)/200=0.0300\), on the squared log odds ratio scale.
Using this estimate, each random-effects weight is \(1/(0.10^2+0.0300)=25\). The random-effects mean is \(\{25(0.05)+25(0.25)+25(0.45)\}/75=0.250\), and the plug-in standard error is \(\sqrt{1/75}=0.1155\). The mean equals the fixed-effect estimate because the three standard errors and resulting random-effects weights are equal. Its larger standard error reflects the estimated between-cohort variation. This plug-in calculation illustrates weighting; it does not include every adjustment used by all inferential procedures for estimating \(\tau^2\).
Worked Example: Heterogeneity Changes the Influence of a Precise Cohort
Consider two hypothetical cohort estimates, \(0.00\) and \(0.40\), for the same genetic effect, with standard errors \(0.05\) and \(0.20\), respectively. Suppose the scientific target is a mean across settings and, for illustration, take \(\tau^2=0.040\) as given. This value is used to show the weighting consequence, not claimed to be estimated from these two observations.
The fixed-effect weights are \(400\) and \(25\), so the fixed-effect estimate is
With random effects, the weights are \(1/(0.0025+0.040)=23.529\) and \(1/(0.040+0.040)=12.500\). Hence
The random-effects estimate is closer to the less precise cohort because the assumed between-cohort variance reduces the relative dominance of the most precise cohort. This is not evidence that the second cohort is more reliable. It is a consequence of estimating a different target under a model that allows true effects to vary.
Assess Comparability and Heterogeneity
Before choosing either model, check whether the cohort estimates answer sufficiently comparable questions. In genetic association analyses, relevant differences include the phenotype definition, ancestry and other population characteristics, allele coding, genetic model, covariate adjustment, and effect scale. Some differences may be harmonized or explained by design; others may imply that the estimates do not represent one common quantity. Neither fixed nor random effects repairs a mismatch in the underlying estimand.
Cochran’s \(Q\) statistic measures dispersion of the cohort estimates around the fixed-effect pooled estimate relative to their stated sampling variances. Under a common-effect model and its assumptions, it is commonly compared with a chi-squared distribution with \(k-1\) degrees of freedom. \(I^2\) summarizes the share of observed dispersion attributed to between-cohort variation under its usual interpretation; \(\tau^2\) expresses estimated between-cohort variance on the squared effect scale. These quantities, introduced in earlier material on heterogeneity, describe evidence and magnitude of variation; none independently determines the scientific target.
A heterogeneity-test p-value is particularly poor as a model-selection switch. With few cohorts, \(Q\) can have low power, so a large p-value does not establish that effects are identical. With many cohorts or very precise estimates, small departures from a common effect can produce a small p-value even when the variation has little practical consequence. Also, statistical evidence of variation does not make the mean effect the relevant scientific target.
A useful decision sequence is:
State whether the goal is one shared effect for the included cohorts or the mean of effects across a defined distribution of settings.
Confirm that outcomes, effect definitions, scales, and genetic contrasts align, and assess whether cohort dependence or design differences undermine the usual assumptions.
Use cohort estimates, uncertainty, \(Q\), \(I^2\), and \(\tau^2\) as evidence about heterogeneity, while considering plausible reasons for differences and the number of cohorts.
Explain the estimand and assumptions, report the pooled estimate and uncertainty, and describe heterogeneity rather than presenting a p-value as the model-selection rule.
Comparison, Diagnostics, and Limitations
| Question | Fixed-effect approach | Random-effects approach |
|---|---|---|
| Target | One common true effect in the included cohorts | Mean of a distribution of cohort-specific true effects |
| Variance used for weighting | \(SE_i^2\) | \(SE_i^2+\tau^2\) |
| Key assumption | All cohorts share the same true effect | True effects vary around a meaningful distributional mean |
| When useful | A shared effect is scientifically justified and relevant | Variation is plausible and the mean across a target set of settings is relevant |
If the scientific question concerns a particular cohort or population rather than a shared effect or a distributional mean, neither pooled target may be sufficient; a cohort-specific estimate or a model that represents the relevant population differences may be needed. A random-effects model also should not be treated as a universal safeguard against heterogeneity. A single mean and variance can obscure structured differences, and a normal distribution of true effects may be a poor approximation.
Diagnostics include examining the direction and magnitude of cohort estimates, their standard errors, and whether a small number of cohorts dominate the result. Look for inconsistencies in outcome definitions, allele alignment, effect scales, and study design. The forthcoming material on forest plots and influence analysis provides visual and sensitivity tools for these checks. If apparent heterogeneity tracks a systematic design or population difference, simply adding \(\tau^2\) does not explain that pattern. With few cohorts, both \(\tau^2\) and the random-effects mean can be imprecise; inference can depend materially on the variance estimator and uncertainty method.
Common Mistake
Wrong reasoning: “The \(Q\)-test p-value is greater than 0.05, so there is no heterogeneity and I must use a fixed-effect model.” The test’s failure to reject is not evidence that all true effects are equal, particularly when there are few cohorts. Nor does a small p-value by itself establish that a random-effects mean answers the scientific question.
Correction: Choose the model according to the estimand and the assumptions justified by the study context. Use heterogeneity statistics to assess consistency and uncertainty about variation, not as an automatic decision threshold. State what population of effects the pooled estimate represents and what limitations remain.
Check Your Understanding
Answer each question, then compare with the solutions below.
- Two cohorts have estimates \(0.20\) and \(0.30\), each with standard error \(0.10\). What is their fixed-effect estimate?
- Why does a nonsignificant \(Q\)-test not prove that the common-effect assumption is true?
- A study aims to estimate the average effect across a defined range of populations and expects true effects to vary. Which model target is aligned with that aim?
- In a random-effects model with \(\tau^2=0.020\), what is the weight for an estimate with \(SE_i=0.10\)?
Solutions. (1) Both weights are \(1/0.10^2=100\), so \(\hat{\theta}_{FE}=\{100(0.20)+100(0.30)\}/200=0.250\). (2) A nonsignificant result means the data did not provide sufficient evidence against the common-effect model at the chosen threshold; with few cohorts, the test may have low power to detect real variation. (3) The random-effects mean \(\mu\) is aligned with the target, provided the cohort effects can reasonably be treated as a distribution representing those populations. (4) The weight is \(1/(0.10^2+0.020)=1/0.030=33.33\), in inverse squared effect units.