Why Read the Cohort Estimates, Not Just the Pooled Result?
A meta-analysis condenses evidence from multiple cohorts into a pooled estimate, but that summary can hide important differences. Cohorts may have effects in different directions, estimates with very different precision, or confidence intervals that suggest a pooled result is not representative of every setting. A forest plot places the cohort-specific estimates and intervals alongside the pooled result so these patterns can be inspected directly.
A forest plot is a diagnostic display, not a test by itself. It helps the reader check whether estimates are aligned, whether the pooled result is driven mainly by precise cohorts, and whether there are features that warrant checking the analysis. It cannot establish why cohort results differ, and visual inspection does not replace the model assumptions or formal heterogeneity assessment.
Read the Plot in Its Analysis Scale
Before interpreting a forest plot, check what the horizontal axis measures and how the outcome and effect have been defined. For a regression coefficient or mean difference, the null value is usually zero: positive values indicate one direction of association and negative values the other, according to the stated coding. For a ratio measure such as an odds ratio (OR), the null is one on the original scale. Many analyses instead plot the logarithm of the ratio, for which the null is zero. The axis label must make clear which scale is shown.
A cohort's point marks its estimated effect. Its horizontal line marks a confidence interval, often a 95% interval. A confidence interval describes uncertainty in that cohort's estimate under the model used to calculate it; it is not a range that contains every cohort's true effect. Under a normal approximation, an estimate \(\hat\beta_i\) with standard error \(SE_i\) has an approximate 95% confidence interval \(\hat\beta_i \pm 1.96SE_i\). This interval is centered at the estimate, and its width reflects its standard error.
Some plots use larger squares for cohorts with greater meta-analysis weight. Under fixed-effect inverse-variance weighting, cohort \(i\)'s weight is \(w_i=1/SE_i^2\): a smaller standard error means a larger weight. A diamond often marks the pooled estimate, with its width representing the pooled confidence interval. These conventions are common, not universal, so read the legend rather than infer meaning from the symbols alone.
Assumptions Behind the Display and the Pooled Marker
A meaningful forest plot requires comparable cohort estimates. The effect must refer to the same outcome, contrast, and direction in each cohort, and estimates must be on a common scale. Harmonizing alleles and effect scales is especially important for genetic association results: an allele reversal can flip a coefficient's sign without any biological change. The variance estimates must also be appropriate for the cohort designs and analyses.
The usual inverse-variance fixed-effect calculation assumes cohort estimates are unbiased for a common effect and are independent, with correctly estimated sampling variances. Under those assumptions, the pooled estimate is a weighted average, and its standard error follows from the sum of weights. If cohort samples overlap, treating their estimates as independent can produce an incorrect pooled standard error and interval; the dependence must be accounted for in the analysis, not merely noted in the plot.
When the scientific model allows cohort-specific true effects, a random-effects calculation adds between-cohort variance \(\tau^2\) to the sampling variance. Its weights are \(1/(SE_i^2+\tau^2)\). The pooled marker therefore depends on the model and variance estimate used. It summarizes the model's center; it does not imply that every cohort has that effect. A prediction interval, when reported, addresses a different question by representing uncertainty about effects in a new cohort under the random-effects model.
Worked Example: Connecting Interval Width to Weight
Suppose three hypothetical, independent cohorts estimate the same harmonized genetic association coefficient, measured in phenotype units per effect allele. Their estimates and standard errors are \(0.20\) with \(SE=0.10\), \(0.45\) with \(SE=0.15\), and \(0.10\) with \(SE=0.08\). Assume normal-approximation intervals are appropriate. The corresponding 95% intervals are:
Cohort A: \(0.20 \pm 1.96(0.10)=0.20\pm0.196=[0.004,0.396]\).
Cohort B: \(0.45 \pm 1.96(0.15)=0.45\pm0.294=[0.156,0.744]\).
Cohort C: \(0.10 \pm 1.96(0.08)=0.10\pm0.1568=[-0.0568,0.2568]\).
Cohort C has the smallest standard error and therefore the narrowest interval; Cohort B has the largest standard error and widest interval. Under fixed-effect inverse-variance weighting, the weights are \(100\), \(44.44\), and \(156.25\), respectively. Thus Cohort C contributes the most weight, even though its point estimate is not the largest.
The plot should therefore be read in two ways: Cohort B has the largest estimated coefficient, while Cohort C is the most precise of the three. Interval width and point-estimate size answer different questions. A large point estimate does not automatically have a large influence on the pooled result.
A Short Derivation: Why the Pooled Marker Is a Weighted Average
For independent cohort estimates \(\hat\beta_i\) with sampling variances \(SE_i^2\), suppose each estimates the same common effect \(\beta\) and is approximately normal: \(\hat\beta_i\sim N(\beta,SE_i^2)\). The inverse-variance estimate minimizes the weighted sum of squared deviations, \(\sum_i w_i(\hat\beta_i-b)^2\), over candidate pooled values \(b\), using \(w_i=1/SE_i^2\). Differentiating with respect to \(b\) and setting the result to zero gives:
Because the cohort estimates are independent, the variance of this weighted average is \(\sum_i w_i^2SE_i^2/(\sum_i w_i)^2=1/\sum_i w_i\). Hence \(SE(\hat\beta_{\mathrm{FE}})=1/\sqrt{\sum_i w_i}\). The plotted fixed-effect confidence interval is typically \(\hat\beta_{\mathrm{FE}}\pm1.96SE(\hat\beta_{\mathrm{FE}})\). These formulas rely on the common-effect and independence assumptions; a forest plot cannot make those assumptions true.
Worked Example: A Pooled Result Can Conceal Cohort Differences
Consider four hypothetical independent cohorts estimating one harmonized coefficient in the same phenotype units per allele. Their estimates and standard errors are \(0.30\) and \(0.10\), \(0.25\) and \(0.12\), \(-0.05\) and \(0.10\), and \(-0.10\) and \(0.12\). Suppose the fixed-effect model is used for illustration. The weights are \(100\), \(69.44\), \(100\), and \(69.44\), giving:
The pooled standard error is \(1/\sqrt{338.88}\approx0.0543\), so its approximate 95% interval is \(0.1045\pm1.96(0.0543)=[-0.0019,0.2109]\). The interval just includes zero. The pooled estimate is positive, but two cohort estimates are negative and two are positive. The pooled interval does not show that all cohort effects are positive, nor that the effects are identical.
For a formal heterogeneity summary, Cochran's \(Q\) compares cohort estimates with the fixed-effect pooled estimate: \(Q=\sum_i w_i(\hat\beta_i-\hat\beta_{\mathrm{FE}})^2\). Substitution gives approximately \(3.82+1.47+2.39+2.90=10.58\), with \(df=4-1=3\). The conventional \(I^2\) summary is \(\max\{0,(Q-df)/Q\}\times100\%\), which here is approximately \(71.7\%\). These calculations suggest substantial inconsistency relative to sampling variation in this example; neither statistic explains its cause. Visually, the opposing point-estimate directions are a diagnostic signal that the single pooled estimate conceals.
Visual Patterns and What They Do—and Do Not—Tell You
A useful reading begins with the cohort rows, before the diamond. Check the direction of each estimate relative to the null, the spread of the point estimates, and how much the intervals overlap. Then compare the pooled marker with the cohort estimates and inspect the model and weighting information. This prevents the visually prominent pooled result from eclipsing important cohort-level patterns.
Interval overlap is not a heterogeneity test. Two 95% intervals can overlap even when the estimates differ more than expected under a common-effect model, and intervals that do not overlap are a strong visual warning but still do not replace a formal assessment. The intervals are calculated separately; they are not a direct confidence interval for the difference between two cohort effects. Use the formal heterogeneity methods, such as \(Q\), \(I^2\), and \(\tau^2\), alongside—not instead of—the plot.
Also distinguish uncertainty about the pooled mean from variation among true effects. A narrow pooled confidence interval can coexist with substantial heterogeneity, especially when many cohorts contribute information. If the scientific question concerns what effects might occur in a new cohort, a prediction interval is more relevant than the pooled confidence interval. Neither interval removes the need to inspect the cohort estimates.
Worked Example: Reading a Ratio-Scale Forest Plot
Suppose a hypothetical plot reports cohort odds ratios for a genetic variant and disease outcome, with the horizontal axis labeled “OR” and a vertical null line at \(1\). One cohort has \(OR=1.20\) and a 95% interval from \(0.95\) to \(1.52\). The estimate indicates 20% higher odds per stated allele under the fitted model, compared with the reference allele; its interval includes \(1\), so this cohort's interval is compatible with the null value at the 95% confidence level.
A second cohort reports \(OR=0.80\) with a 95% interval from \(0.65\) to \(0.99\). This estimate is in the opposite direction: the estimated odds are 20% lower per allele. The interval lies below \(1\), which is evidence against the null at the corresponding two-sided 5% level under the interval's model. It does not by itself establish why the cohorts differ.
If the plot instead uses log odds ratios, the two point estimates are \(\log(1.20)\approx0.182\) and \(\log(0.80)\approx-0.223\), and the null line is at zero. The direction is unchanged by taking logs, but the axis value and null location change. A reader who treats a log-OR axis as an OR axis may misread both the effect magnitude and the null. The plot's scale label and reference line must be read together.
Diagnostics, Alternatives, and Limitations
Unexpected visual patterns should prompt checks rather than immediate biological explanations. A single cohort far from the others may reflect genuine context-specific effects, but it can also signal a coding reversal, allele or outcome harmonization error, incompatible effect scale, unusual standard error, or an analysis difference. Confirm that estimates refer to the same contrast and direction, inspect the reported units, and verify whether cohort samples are independent. If plotted intervals are unexpectedly asymmetric, determine whether they were constructed on a transformed scale and then back-transformed.
A forest plot is particularly useful alongside formal heterogeneity summaries: \(Q\) assesses incompatibility with a common-effect model, while \(I^2\) describes the proportion of observed variation attributed to heterogeneity rather than sampling error under its conventional interpretation. These summaries have limitations, including low power with few cohorts and sensitivity to precision. A random-effects model is an alternative when cohort effects are treated as varying; it changes the weights and interpretation of the pooled result but does not make heterogeneity disappear. A fixed-effect model targets a common effect under its assumptions, whereas a random-effects model summarizes a distribution of effects. The choice should reflect the scientific question and design, not which plot looks more appealing.
The plot itself has limits. Axis range can visually exaggerate or compress differences; study order can influence what patterns stand out; and small cohorts may have wide intervals that dominate the display despite low weight. Marker size can help show weights, but size is not a measure of effect magnitude. A forest plot also cannot identify causal mechanisms, correct selection bias, or show all relevant design differences. It is a compact diagnostic, not a substitute for checking the underlying analyses.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A coefficient estimate is \(0.12\) with \(SE=0.05\). Calculate its approximate 95% confidence interval and state whether it includes the null value.
- Why can a cohort with the largest point estimate contribute less to a fixed-effect pooled result than another cohort?
- A forest plot uses a log-OR axis. What is the null line, and what does a negative plotted value indicate?
- Two cohort 95% confidence intervals overlap. Does this establish that the effects are homogeneous? Explain.
- A pooled confidence interval is narrow, while cohort estimates vary in direction. What should be examined before interpreting the pooled result as representative of every cohort?
Solutions. (1) \(0.12\pm1.96(0.05)=0.12\pm0.098=[0.022,0.218]\). The interval excludes zero. (2) Fixed-effect weight is \(1/SE^2\), so a less precise estimate has less weight even if its point estimate is larger. (3) The null is zero on a log-OR scale; a negative value corresponds to an OR below one, in the direction of lower odds for the specified allele contrast. (4) No. Overlap is not a test of homogeneity; assess heterogeneity formally and inspect the estimates and assumptions. (5) Examine cohort directions and precision, heterogeneity summaries, model choice, harmonization, and independence. A narrow pooled interval alone does not show that each cohort shares the pooled effect.