Why Forest Plots Need More Than a Row of Estimates
A subgroup forest plot places treatment-effect estimates and their uncertainty on a shared scale. It can make a pattern of subgroup results easier to scan, but the plot is a summary of estimates—not a test of whether effects differ. A sound display therefore makes clear what each effect means, how uncertain it is, how many participants contributed to it, and what formal test addresses treatment-by-subgroup interaction.
The target should be settled before drawing the plot. Each row should represent a prespecified subgroup and the same kind of treatment-effect contrast, measured on the same scale and for the same outcome and follow-up definition. Otherwise, apparent differences may reflect different estimands or scales rather than effect modification.
The causal interpretation of an estimate depends on the study design and analysis, not on the plot. In a randomized trial, randomization within the prespecified baseline subgroups supports causal comparisons, subject to consistency, no interference, and adequate treatment assignment in the groups being analyzed. In an observational study, identification additionally requires appropriate control of confounding and other design assumptions. A forest plot cannot repair a failure of those assumptions.
What to Put in the Plot
For each subgroup, show the estimated effect, its uncertainty interval, and the subgroup sample size. When possible, report the number assigned to each treatment arm, not just the total subgroup size: unequal allocation, missing outcomes, and different arm-specific information can affect precision. If the analysis uses a model or standardization, state what estimate is being plotted and how its uncertainty was obtained.
The horizontal axis must match the effect measure. For a mean difference or risk difference, zero represents no effect. For a risk ratio or odds ratio, one represents no effect; if the ratios are plotted on a logarithmic scale, the null is still marked at one. Estimates and intervals should be displayed on a scale that preserves the intended interpretation. Label the outcome, units, treatment contrast, and direction that favors each treatment.
A point marker represents the subgroup estimate, and a horizontal line represents its confidence interval. Marker size is sometimes scaled to precision, but the convention should be stated; a larger marker must not be mistaken for a larger treatment effect. Subgroup rows should be named clearly and kept in a clinically interpretable, prespecified order. If an overall estimate is also shown, identify its target population and method. It is not automatically an average of subgroup effects, nor is it a test of whether they differ.
| Plot element | What it communicates | What it does not establish |
|---|---|---|
| Point estimate | The estimated treatment contrast for that subgroup | That the estimate is precise or causally identified |
| Confidence interval | Sampling uncertainty under the analysis and its assumptions | The probability that the fixed effect lies inside this particular interval |
| Subgroup sizes | How many participants contributed, ideally by treatment arm | Precision by themselves; outcome variability and analysis also matter |
| Interaction test | Evidence against equal subgroup effects on the tested scale | Which subgroup differences matter or why they arise |
Worked Example: Reading Estimates and an Omnibus Test
Suppose a hypothetical randomized trial estimates mean blood-pressure effects in three prespecified baseline subgroups. Effects are treatment minus control, in mmHg, so negative values favor treatment. Assume subgroup estimates are independent, approximately normally distributed, and have the standard errors shown. The trial has 80 treated and 80 control participants in subgroup A, 100 and 100 in subgroup B, and 70 and 70 in subgroup C.
| Subgroup | Estimate (mmHg) | SE (mmHg) | 95% CI (mmHg) | Treated / control |
|---|---|---|---|---|
| A | -2.0 | 0.8 | -3.568 to -0.432 | 80 / 80 |
| B | -4.0 | 1.0 | -5.960 to -2.040 | 100 / 100 |
| C | -1.0 | 0.7 | -2.372 to 0.372 | 70 / 70 |
The intervals use estimate \(\pm 1.96SE\). For subgroup A, for example, \(-2.0\pm1.96(0.8)=-2.0\pm1.568\), giving \([-3.568,-0.432]\) mmHg. Subgroup C's interval includes zero, so its individual estimate is not distinguishable from zero at the corresponding two-sided 5% level under this approximation. That statement is not an interaction test: it does not compare subgroup C with A or B.
For an omnibus test of equal effects, use the inverse-variance weighted mean under the common-effect null. The weights are \(w_g=1/SE_g^2\), giving \(w_A=1.5625\), \(w_B=1.0000\), and \(w_C=2.0408\). Thus,
The heterogeneity statistic is
Under the null of equal subgroup effects and the stated independent, approximately normal model, \(Q\) is compared with a chi-square distribution with \(3-1=2\) degrees of freedom. Its upper-tail p-value is approximately \(0.0488\). This is evidence against equal effects on the mean-difference scale at the 5% level, under the test assumptions. It does not prove that a particular subgroup truly benefits more, identify a mechanism, or rule out chance as an explanation.
How the Interaction Test Relates to the Plot
The formal question for an omnibus interaction test is whether the subgroup effects are all equal on a specified scale. With \(G\) mutually exclusive subgroups, the null is \(H_0:\tau_1=\tau_2=\cdots=\tau_G\). The alternative is that at least one effect differs. For an estimate vector with covariance matrix \(V\), this can be tested with a Wald statistic: choose a contrast matrix \(C\) whose rows encode \(G-1\) independent subgroup comparisons, then calculate \(W=(C\hat{\boldsymbol{\tau}})^\mathsf{T}(CVC^\mathsf{T})^{-1}(C\hat{\boldsymbol{\tau}})\). Under the null and suitable large-sample conditions, \(W\) is approximately chi-square with \(G-1\) degrees of freedom.
When the subgroup estimates are independent and approximately normal with variances \(SE_g^2\), the heterogeneity statistic \(Q\) used in the example is a convenient form of this test. The weighted mean is the value common to all subgroups that minimizes the weighted sum of squared deviations. To see this, define \(S(t)=\sum_g w_g(\hat{\tau}_g-t)^2\). Differentiating gives \(S'(t)=-2\sum_g w_g(\hat{\tau}_g-t)\); setting this to zero yields \(t=\sum_gw_g\hat{\tau}_g/\sum_gw_g\). Substituting that fitted common value into \(S(t)\) gives \(Q\). Under the common-effect null, the remaining deviations have \(G-1\) independent dimensions, which explains the chi-square reference degrees of freedom.
If subgroup estimates are correlated—for example, because the same participants contribute to multiple overlapping categories—the independence-based \(Q\) calculation is not appropriate as written. Use the covariance matrix from the analysis in a suitable interaction test. Likewise, the test must match the effect scale: equality of risk differences is not the same null hypothesis as equality of risk ratios. Report the scale and the test rather than presenting an unexplained interaction p-value.
Worked Example: Two Subgroups and a Direct Contrast
In a hypothetical randomized trial, an analyst estimates mean treatment effects of \(-3.0\) mmHg (SE \(0.9\)) in subgroup 1 and \(-1.0\) mmHg (SE \(0.8\)) in subgroup 2. The groups are disjoint, so suppose the estimates are independent and approximately normal. The direct interaction contrast is the difference between subgroup effects:
The Wald statistic is \(z=-2.0/1.204\approx-1.661\), giving a two-sided p-value of approximately \(0.0967\). Equivalently, \(W=z^2\approx2.759\), compared with a chi-square distribution with 1 degree of freedom. A 95% confidence interval for the difference is
The interval includes zero, and the interaction test does not reject equal effects at the 5% level. The result is not proof of identical subgroup effects: the interval also permits a difference as large as about 4.36 mmHg in the negative direction. The plotted subgroup estimates may look separated, but the contrast's uncertainty is the relevant information for this two-group interaction question.
From Data to a Forest-Plot Report
The following sequence helps keep the display tied to the analysis rather than letting the picture dictate the conclusion.
Name the outcome, follow-up, treatment contrast, effect scale, target population, and prespecified subgroups. Check that subgroup rows represent comparable estimands.
For each subgroup, report its estimate, standard error or confidence interval, and the number analyzed in each treatment arm. State the estimator and any model or standardization used.
Mark the no-effect value, label units and direction, and choose limits that show the estimates and intervals without concealing relevant uncertainty.
Report the test statistic, reference distribution, degrees of freedom, and p-value. Ensure the test addresses equality on the same effect scale shown in the plot.
Summarize the pattern, uncertainty, and test result. Do not infer effect modification merely because one subgroup interval excludes the null and another does not.
Worked Example: A Risk-Difference Forest Plot
Consider a hypothetical randomized trial reporting a binary outcome. In participants younger than 65, 18 of 120 treated participants and 24 of 120 controls have the outcome. In participants aged 65 or older, the corresponding counts are 30 of 100 and 40 of 100. Define each effect as treatment risk minus control risk, so a negative risk difference favors treatment. Assume independent treatment arms and subgroups, and use the large-sample binomial standard error.
For those younger than 65, the risks are \(18/120=0.150\) and \(24/120=0.200\), so the estimated risk difference is \(0.150-0.200=-0.050\), or \(-5.0\) percentage points. Its standard error is
The 95% interval is \(-0.050\pm1.96(0.04895)\), or approximately \([-0.1459,0.0459]\). For those aged 65 or older, the risks are \(30/100=0.300\) and \(40/100=0.400\), giving a risk difference of \(-0.100\). Its standard error is
The corresponding 95% interval is \(-0.100\pm1.96(0.06708)\), or approximately \([-0.2315,0.0315]\). In the plot, the point estimates are at \(-0.050\) and \(-0.100\), with the null line at zero; the intervals show substantial uncertainty. The table or labels should show the arm sizes, 120 per arm in the younger subgroup and 100 per arm in the older subgroup.
The estimated interaction contrast, younger minus older, is \(-0.050-(-0.100)=0.050\), or 5.0 percentage points. Under the independence assumption, its standard error is
Thus \(z=0.050/0.08304\approx0.602\), with two-sided p-value approximately \(0.547\). The interaction contrast's 95% interval is \(0.050\pm1.96(0.08304)\), approximately \([-0.1128,0.2128]\), or \(-11.28\) to \(21.28\) percentage points. These data do not provide strong evidence that the subgroup risk differences differ, but the interval is wide. The plot should convey both the estimates and that uncertainty, not imply that the effects are established as equal.
Common Mistakes and Practical Interpretation
- Comparing significance labels instead of effects. “Significant in one subgroup, not significant in another” does not establish that the subgroup effects differ. Test their contrast or the prespecified omnibus interaction.
- Treating interval overlap as an interaction test. The visual amount of overlap is not the test statistic. The uncertainty of the difference depends on the variances and covariance of the estimates.
- Reading subgroup sample size as precision. A larger subgroup often provides more information, but sample size alone does not determine the standard error; outcome variability, event frequency, allocation, and the estimator matter.
- Changing the effect scale without saying so. Heterogeneity can differ across additive and multiplicative scales. State which scale is plotted and tested.
- Presenting a subgroup pattern as causal without design support. The forest plot is a display. Causal interpretation still requires the identification assumptions appropriate to the study.
- Overinterpreting an interaction p-value. A small p-value is evidence against equal effects on the tested scale under the model; it does not measure the size or importance of the differences. A large p-value does not demonstrate homogeneity.
A useful report therefore names the estimand and scale, gives subgroup estimates with confidence intervals and arm-specific sizes, and states the interaction test and its assumptions. The graph helps readers see the pattern; the estimates and formal contrast quantify what the data support.
Check Your Understanding
Use the definitions and examples above to answer these questions.
- Why should all subgroup estimates in one forest plot use a clearly specified, consistent effect scale?
- For a plot of risk differences, where should the no-effect reference line be placed? What changes if the plot displays risk ratios?
- What null hypothesis does an omnibus test of interaction assess for three prespecified subgroups?
- Why does finding a confidence interval that excludes zero in one subgroup but not another fail to establish effect modification?
- What information do treatment-arm subgroup sizes add, and why are they not a substitute for standard errors or confidence intervals?
- In the risk-difference example, what does the interaction confidence interval communicate that the two subgroup intervals alone do not directly quantify?