Why Reporting Is Part of the Analysis
A meta-analysis result is only interpretable if readers can tell what was combined. A pooled effect can conceal different phenotype definitions, allele codings, adjustment models, or cohort participation. A small p-value does not resolve those differences. Reporting should therefore make the chain from cohort-level estimates to the pooled conclusion visible, including decisions that affect which estimates enter the analysis.
This tutorial focuses on reporting, not on re-deriving inverse-variance meta-analysis or the heterogeneity statistics. The earlier material on fixed- and random-effects meta-analysis, Cochran’s Q, and I-squared supplies those methods. Here, the central task is to present their inputs, assumptions, and implications clearly enough that readers can reconstruct the analysis and judge its scope.
State the Estimand and the Cohort-Level Inputs
Begin by stating the outcome, effect scale, genetic model, and target effect. For example, an additive linear-regression coefficient might be the mean change in a quantitative phenotype, in mmol/L, per additional copy of the effect allele. A logistic-regression coefficient is commonly on the log-odds scale; if it is exponentiated, report the odds ratio and its scale. These are different estimands, so a report must not describe one as the other.
For every contributing cohort, report the sample size relevant to the analysis, the phenotype definition and units, the model and covariates, and the estimate with a standard error or confidence interval. In case-control analyses, distinguish the numbers of cases and controls. Also state relevant cohort features, such as ancestry composition, genotyping or imputation source, and the variant’s imputation quality when applicable. These details help readers assess whether the cohort estimates plausibly target a sufficiently comparable effect.
For a common-variant association, allele labels need particular care. Name the variant using a build-specific genomic position and reference/alternate alleles, or another unambiguous identifier with the genome build stated. Identify the effect allele and the other allele, and define the dosage whose coefficient is reported. A label such as “the risk allele” is not a substitute: it presumes a direction and can become misleading when effects differ by phenotype or cohort.
Worked Example: Harmonizing a Cohort Effect Allele
Suppose three hypothetical cohorts analyze the same A/G variant and the same quantitative phenotype. The target effect is the mean phenotype difference per additional G allele. Cohort 1 reports \(\hat{\beta}=0.08\) mmol/L per A allele, with \(SE=0.03\). Cohort 2 reports \(0.05\) mmol/L per G allele, with \(SE=0.025\). Cohort 3 reports \(0.06\) mmol/L per G allele, with \(SE=0.04\). Assume each cohort has used the stated phenotype scale and comparable additive dosage models.
Cohort 1 must be recoded because its reported effect allele is A, not G. For a biallelic genotype, the G-allele dosage equals \(2\) minus the A-allele dosage. Replacing one dosage by its complement reverses the regression slope: the same fitted mean relationship is written with the opposite coefficient. Thus the harmonized estimate for Cohort 1 is \(-0.08\) mmol/L per G allele, and its standard error remains \(0.03\) mmol/L per allele.
The harmonized cohort table should therefore give Cohort 1 as \(-0.08\) (\(SE=0.03\)), Cohort 2 as \(0.05\) (\(SE=0.025\)), and Cohort 3 as \(0.06\) (\(SE=0.04\)), all per G allele. The negative estimate in Cohort 1 is not a change in the data; it is the same estimate expressed on the common allele scale. The disagreement in direction is scientifically important and should not be hidden by allele recoding.
This recoding requires confidence that the cohorts refer to the same variant and allele orientation. Complementary strand alleles can be difficult to distinguish for A/T and C/G variants, particularly when strand and allele-frequency information are unavailable or uninformative. State how such cases were resolved. If orientation cannot be established reliably, excluding the estimate or variant is safer than silently choosing a direction. Report exclusions and their reasons, including variants unavailable in some cohorts and any proxy-variant decisions.
Report Fixed- and Random-Effects Results Precisely
For \(K\) independent cohorts with estimates \(\hat{\beta}_i\) and standard errors \(SE_i\), inverse-variance fixed-effect meta-analysis assigns weight \(w_i=1/SE_i^2\). It estimates one common underlying effect, with observed differences attributed to sampling error. The pooled estimate is the weighted mean, and its standard error is based on the sum of the weights. The fixed-effect interpretation requires a defensible common-effect target; it does not mean that cohort estimates are literally identical.
A random-effects model allows underlying cohort effects to vary, commonly representing them as draws from a distribution with mean \(\mu\) and between-cohort variance \(\tau^2\). Its weights are typically \(w_i^*=1/(SE_i^2+\hat{\tau}^2)\). The report must name the estimator used for \(\tau^2\), because different estimators can yield different results, particularly with few cohorts. Report the random-effects mean and uncertainty as an average across the modeled distribution of cohort effects, not as proof that every cohort has that effect.
For either model, give the pooled estimate, standard error, confidence interval, and p-value, with units or effect scale. State the confidence-interval method and whether the p-value is one- or two-sided when that distinction matters. A normal-approximation interval is \(\hat{\beta}\pm1.96SE\) for a 95% interval; if this approximation is used, say so. Do not present more decimal places than the inputs and analysis support.
Worked Example: Reporting Fixed- and Random-Effects Results
Consider three hypothetical independent cohorts estimating the effect of one allele on a quantitative phenotype, in mmol/L per allele: \(0.10\) with \(SE=0.02\), \(0.02\) with \(SE=0.02\), and \(0.14\) with \(SE=0.02\). Assume allele harmonization, phenotype scale, and additive model are consistent across cohorts. Each fixed-effect weight is \(1/0.02^2=2500\).
The fixed-effect estimate is
Its standard error is \(\sqrt{1/7500}=0.01155\) mmol/L per allele. The 95% normal-approximation interval is \(0.08667\pm1.96(0.01155)\), or approximately \(0.06404\) to \(0.1093\) mmol/L per allele. The test statistic is \(Z=0.08667/0.01155=7.506\), giving a two-sided p-value of approximately \(6.1\times10^{-14}\). This is strong evidence against a zero common effect under the fixed-effect model, but the cohort estimates also show substantial dispersion.
For these data, Cochran’s statistic is
With \(df=K-1=2\), the chi-square reference gives \(p\approx0.0000884\). The conventional estimate \(I^2=\max\{0,(Q-df)/Q\}\times100\%\) is approximately \(89.3\%\). This describes a high estimated proportion of the observed dispersion attributable to between-cohort variation rather than sampling error; it is not a measure of the size or direction of that variation.
As an illustration, the DerSimonian–Laird estimate here is \(\hat{\tau}^2=(Q-df)/C\), where \(C=\sum w_i-\sum w_i^2/\sum w_i=5000\). Thus \(\hat{\tau}^2=(18.67-2)/5000\approx0.003333\) \((\text{mmol/L per allele})^2\). Each random-effects weight is then \(1/(0.0004+0.003333)\approx267.9\), so the random-effects mean remains \(0.08667\) mmol/L per allele because the standard errors are equal. Its standard error is \(\sqrt{1/(3\times267.9)}\approx0.03528\), and its approximate 95% interval is \(0.01752\) to \(0.1558\) mmol/L per allele. The corresponding normal test gives \(Z\approx2.457\) and a two-sided p-value of about \(0.0140\). Report the estimator and these modeling choices; do not treat the random-effects result as interchangeable with the fixed-effect result.
Make Heterogeneity and Analysis Choices Legible
Report the number of cohorts contributing to each variant and the heterogeneity information alongside the pooled estimate. Commonly reported quantities are \(Q\), its degrees of freedom and p-value, \(I^2\), and \(\hat{\tau}^2\) for a random-effects analysis. \(Q\) tests whether the observed dispersion is greater than expected from sampling error under a common-effect model. A non-significant \(Q\) does not establish homogeneity, especially when few cohorts contribute. Conversely, a significant \(Q\) indicates evidence of dispersion, not its cause.
\(I^2\) is a relative summary of the fraction of observed variability associated with heterogeneity rather than sampling error. It depends on study precision and can be uncertain, particularly with a small number of cohorts. It does not tell readers how large the differences are in the phenotype’s units. \(\tau^2\), by contrast, is on the squared effect scale and estimates between-cohort variance under the chosen random-effects model. A prediction interval, when reported, addresses the range of underlying effects anticipated for a new comparable setting under that model; it is distinct from a confidence interval for the mean effect.
Also report choices that determine the analysis set: variant and sample quality-control criteria, how missing variants were handled, whether proxy variants were used, and whether any cohorts overlap. Overlapping samples can make cohort estimates dependent, violating the ordinary inverse-variance calculations that assume independence unless their covariance is accounted for. State whether analyses were prespecified, how multiple testing was handled, and whether results are primary or sensitivity analyses. A sensitivity analysis is most informative when the changed assumption or inclusion rule is explicit.
Define Replication Before Interpreting It
Replication is not simply observing another small p-value. A useful replication claim specifies the variant, phenotype, effect allele, direction, analysis model, eligible cohort or cohorts, and statistical criterion in advance. It also states whether the replication data are independent of discovery data. If a replication set overlaps discovery participants, the resulting evidence is not an independent confirmation in the usual sense.
The criterion may require a prespecified direction of effect and a one-sided significance threshold, or it may require a two-sided threshold. There is no universal rule that every replication must use one particular p-value cutoff. The choice should reflect the protocol and the number of variants or hypotheses being tested. If many candidates are tested in replication, the report should explain the multiplicity procedure. Distinguish “directionally consistent,” “nominally significant under the stated criterion,” and “replicated under the prespecified definition”; they are not synonyms.
Worked Example: Evaluating a Prespecified Replication
Suppose a hypothetical discovery analysis reports a positive effect for a specified allele on a stated quantitative phenotype. Before examining new data, investigators define replication as an independent cohort, the same phenotype scale and allele, a positive direction, and a one-sided test at \(\alpha=0.05\). In the independent replication cohort, the estimate is \(0.045\) mmol/L per allele with \(SE=0.020\) mmol/L per allele. Assume the cohort’s model and allele coding match the protocol.
The test statistic is \(Z=0.045/0.020=2.25\). The one-sided standard-normal p-value is approximately \(0.0122\), which is below the prespecified \(0.05\) threshold. The result is directionally consistent and meets this stated nominal replication criterion. The two-sided p-value is approximately \(0.0244\); reporting it as two-sided would not change the estimate but would answer a different testing question.
The 95% normal-approximation confidence interval is \(0.045\pm1.96(0.020)\), or \(0.0058\) to \(0.0842\) mmol/L per allele. This interval is compatible with a small positive effect as well as a larger one; it does not establish that the replication effect equals the discovery estimate. The conclusion should be limited to meeting the prespecified one-sided, nominal criterion in this independent cohort. It should not be described as genome-wide significant or as proof of a causal effect.
A Practical Reporting Checklist
A compact table can carry most cohort-level information, while the methods and results text explains the decisions and interpretation. For each cohort, include the harmonized effect allele, estimate, standard error, p-value, sample size, and key analysis details. For each pooled result, include the model, number of contributing cohorts, estimate, uncertainty, heterogeneity statistics, and any relevant exclusion or sensitivity result.
Name the phenotype, its units or scale, the effect allele, genetic model, and covariate-adjusted analysis.
Report cohort estimates and standard errors, cohort sample sizes, and the allele-orientation and variant-availability decisions.
State fixed or random effects, the between-cohort variance estimator if used, and the method for intervals and tests.
Report heterogeneity measures, cohort dependence or overlap, sensitivity analyses, and the scope of any replication claim.
Common Reporting Pitfalls
- Reporting only a pooled p-value. A p-value alone hides the estimated effect, its uncertainty, the cohort contributions, and possible heterogeneity. Give the estimate and interval in context.
- Assuming that the same variant label means the same allele. Cohorts may report opposite effect alleles. Harmonize explicitly and document strand ambiguity or exclusions.
- Calling a non-significant heterogeneity test proof of consistency. The test can have low power. Report the statistic and acknowledge uncertainty rather than claiming that all effects are identical.
- Treating a random-effects mean as a universal effect. It summarizes a modeled distribution of cohort effects. Variation in populations, measurements, or designs can limit its generalizability.
- Calling any same-direction result a replication. A strong claim requires a prespecified definition, suitable independent data, and the stated statistical criterion. Describe weaker evidence more narrowly.
- Reporting rounded inputs that do not reproduce the displayed result. Calculate from full-precision values, round consistently, and verify that displayed estimates, standard errors, intervals, and p-values agree at their reported precision.
Check Your Understanding
Use the reporting principles above to answer these questions.
- A cohort reports an estimate per A allele while the meta-analysis uses G as the effect allele. What must be checked and changed before combining the estimates?
- Why should a report give both \(I^2\) and an effect-scale measure such as \(\hat{\tau}^2\), when available?
- What assumption about cohort estimates underlies ordinary inverse-variance meta-analysis, and why does sample overlap matter?
- A replication estimate has the discovery direction but misses the prespecified significance threshold. What can be concluded, and what should not be claimed?
- Why is a confidence interval for a random-effects mean not the same as a prediction interval for a new cohort?