Forest Plots and Cohort-Level Diagnostics

Why Examine Cohorts Before the Pooled Result?

A meta-analysis can produce a precise pooled estimate even when cohort results differ substantially. A forest plot makes those cohort-level estimates visible alongside their uncertainty, so the reader can ask whether the pooled result represents a broadly consistent pattern or is being pulled by one cohort. This is a diagnostic step: a striking cohort result may reflect a genuine difference, random variation, or a data or modeling problem. The plot alone cannot determine which explanation is correct.

This tutorial focuses on reading the plot and checking the cohort estimates that feed into it. It assumes that the effect alleles and phenotype scales have been aligned, and that the estimates and standard errors are suitable for the meta-analysis being displayed. Those checks matter: a forest plot cannot repair incompatible inputs.

Read the Scale, Direction, and Precision

Start with the axis label and the effect measure. For a regression coefficient or a log odds ratio, the null value is zero: positive estimates lie to the right of zero and negative estimates to the left. If the effect is instead shown as an odds ratio, the null is one. A positive log odds ratio corresponds to an odds ratio greater than one because exponentiation preserves the direction: \(OR_i=e^{\hat{\beta}_i}\). The plot’s legend or caption should make clear which allele is the effect allele and what one unit of effect means, such as one additional effect-allele copy.

A confidence interval’s width conveys precision: narrower intervals indicate more precise estimates, usually because their standard errors are smaller. A conventional approximate 95% interval for a coefficient is \(\hat{\beta}_i \pm 1.96SE_i\), under the usual large-sample normal approximation. Whether it crosses the null is not a measure of effect direction by itself. An interval crossing zero indicates that the data do not distinguish the estimate from zero at the corresponding two-sided 5% level under that approximation; it does not establish that the effect is absent.

Many plots draw larger squares for more precise cohorts, often using a size proportional to inverse variance weight \(w_i=1/SE_i^2\). This convention is common, not universal, so check the legend. A large marker does not mean a larger genetic effect. Nor does a narrow interval imply that the estimate is unbiased: precision describes sampling uncertainty under the model, not protection from a cohort-specific error.

Compare Cohort Estimates With Heterogeneity in View

A forest plot is useful for spotting disagreement, but visual separation is not enough to quantify it. For independent cohort estimates with standard errors \(SE_i\), the fixed-effect estimate weights each estimate by its inverse variance. Cochran’s \(Q\) measures the weighted squared departures of cohort estimates from that pooled value. In this setting, \(Q\) is compared with a chi-square distribution with \(k-1\) degrees of freedom under the null model of one common effect, where \(k\) is the number of cohorts. This is a model-based diagnostic, not a test for whether any cohort is erroneous.

Under independent, approximately normal estimates with a common underlying effect and correctly specified standard errors, \(Q\) has approximately a \(\chi^2_{k-1}\) distribution. A large \(Q\) relative to that distribution signals more dispersion than expected under this common-effect model. It does not identify the cause, and it can be affected by the number and precision of cohorts. As covered in earlier material on heterogeneity, the \(Q\) test also has limited power when there are few cohorts.

Use Influence Checks, Not Visual Suspicion Alone

A cohort may look influential because its estimate is far from the others, because it is precise, or both. A straightforward diagnostic is to recalculate the pooled estimate after omitting one cohort at a time. This is called a leave-one-cohort-out analysis. A large change shows that the pooled estimate depends strongly on that cohort. It does not establish that the cohort should be excluded; exclusion requires a defensible reason independent of whether its result is inconvenient.

Influence should be interpreted alongside the cohort’s interval, sample size, allele frequency, imputation quality, phenotype definition, and analysis model. A small cohort with a wide interval may have an extreme point estimate but little influence on an inverse-variance pooled result. Conversely, a precise estimate that differs from the others can strongly affect the pooled result. Examine both the plotted estimate and its weight, rather than judging influence by distance from the center alone.

Translate Intervals Carefully

When a coefficient is on the log-odds scale, it can be exponentiated to report an odds ratio, but the interval endpoints must also be exponentiated. The interval remains an interval for the underlying parameter under the model; it is not a probability statement about where the fixed parameter lies. The calculation below also illustrates why unrounded endpoints should be retained until the final presentation.

From a Diagnostic Signal to a Cohort Review

When cohort estimates disagree or one cohort is highly influential, treat the plot as a prompt for targeted checks. Begin with the inputs and then examine the analyses that produced them. A cohort-level review can include:

• Allele and effect coding: confirm that estimates refer to the same effect allele and that strand, allele frequency, and effect direction were harmonized.
• Phenotype and effect scale: check that phenotype definitions, transformations, units, and reported effect measures are comparable across cohorts.
• Uncertainty and variant quality: verify that standard errors are plausible and inspect variant availability, imputation quality, and allele frequency where appropriate.
• Model and sample structure: review covariate adjustment, relatedness handling, ancestry composition, and possible sample overlap, because these can affect estimates or their uncertainty.
• Influence and robustness: compare the full result with leave-one-cohort-out estimates and inspect whether the discrepancy is isolated or shared by several cohorts.

A coherent difference across cohorts may be scientifically meaningful, while a discrepancy caused by mismatched coding or an unstable model requires correction before interpretation. The distinction cannot be made from \(Q\), an interval, or an influence calculation alone. Document the concern, investigate it using cohort-level information, and avoid selecting a pooled result solely because it is statistically stronger or more convenient.

Common Mistakes

• Wrong: “The cohort interval crosses zero, so there is no genetic effect.” Correction: It indicates uncertainty compatible with zero at that confidence level; it does not establish absence of an effect.
• Wrong: “The most distant point is an outlier and should be removed.” Correction: Distance is a signal to investigate. Check its precision, data, coding, and model, and do not exclude a cohort without a defensible reason.
• Wrong: “A significant \(Q\) test proves that one cohort has a data error.” Correction: It indicates excess variation relative to the common-effect model under its assumptions, not the source of that variation.
• Wrong: “Overlapping confidence intervals mean there is no heterogeneity.” Correction: Interval overlap is not a formal heterogeneity test. Compare estimates with an appropriate heterogeneity assessment and inspect cohort-level context.
• Wrong: “The pooled estimate is the best estimate because its interval is narrow.” Correction: A narrow interval reflects modeled precision. It does not guarantee comparable inputs or rule out influential cohort-specific problems.