Forest Plots For Hierarchical Basket Results

What a Hierarchical Forest Plot Shows

A forest plot can make a hierarchical basket analysis easier to inspect: it places each basket’s estimated treatment effect on a common scale, shows its uncertainty, and provides context for the population-level mean and the amount of variation across baskets. Its job is to display the fitted model’s estimates clearly, not to turn a visual pattern into a new statistical test.

The plot is useful only if its different elements are labelled precisely. A subgroup estimate describes an individual basket’s effect; the population-level mean describes the center of the hierarchical distribution; and the between-subgroup standard deviation describes how dispersed basket effects are around that center. These quantities answer different questions and should not be represented as though they were interchangeable.

For each basket, the plotted interval should usually summarize the marginal posterior of its effect: uncertainty in \(\mu\) and \(\tau\) is integrated over, rather than held fixed. The earlier material on reporting subgroup estimates with partial pooling establishes the role of those subgroup summaries; the plot adds a visual arrangement that makes their scales and uncertainty directly comparable. The horizontal axis must identify the effect measure and units. For a continuous outcome, for example, positive values might mean more symptom-score improvement, in score points.

A vertical reference line at zero marks no difference on an additive effect scale. A second line can mark a prespecified clinically important effect, if that threshold is defined on the same scale. For a ratio measure, the no-effect value is typically one rather than zero. The plot’s labels should state which direction favors treatment; a reader should not have to infer that from the ordering of the baskets.

Constructing the Plot from Posterior Summaries

Suppose a fitted model provides posterior draws of \(\theta_1,\ldots,\theta_J,\mu,\tau\). For each basket, summarize its marginal posterior with a measure of center, such as the posterior mean or median, and a credible interval, commonly the central 95% interval. A 95% credible interval is an interval containing 95% of the posterior probability for that parameter, conditional on the model, prior, and observed data. It is not a confidence interval, and its endpoints are not estimates of the most extreme plausible effects.

Use one effect scale and consistent interval definitions for all baskets. If posterior distributions are skewed, plot posterior medians with quantile-based intervals rather than forcing symmetric intervals around posterior means. A summary for \(\mu\) belongs in a clearly distinguished row or panel. Because \(\tau\) is a standard deviation, it is nonnegative and measured in the same units as \(\theta_j\); it is best reported separately, with a posterior interval, rather than placed among the signed subgroup effects as though it were another treatment effect.

Why Shrinkage and Uncertainty Need Careful Display

The hierarchical model links baskets through their shared distribution. In a simple normal example with known \(\mu\) and \(\tau\), let an observed basket estimate satisfy \(y_j\mid\theta_j\sim N(\theta_j,s_j^2)\), and let \(\theta_j\mid\mu,\tau\sim N(\mu,\tau^2)\). Conditional on those fixed hierarchy parameters, the posterior variance and mean are

This follows by multiplying the normal likelihood and normal prior: the terms involving \(\theta_j\) in the exponent have total precision \(1/s_j^2+1/\tau^2\), and the precision-weighted data and prior centers sum to \(y_j/s_j^2+\mu/\tau^2\). Completing the square gives a normal distribution with variance \(v_j\) and mean \(m_j\). The formula shows why a conditional estimate is pulled toward \(\mu\), with the relative precision of the data and prior determining the amount.

In a fitted hierarchical model, however, \(\mu\) and \(\tau\) are generally uncertain. A plotted marginal posterior for \(\theta_j\) averages over that uncertainty. Its interval can therefore differ from the conditional normal interval calculated with fixed \(\mu\) and \(\tau\). Presenting conditional intervals as if they were the full posterior intervals can understate uncertainty.

Changing Scales and Comparing Displays

A forest plot may use an additive scale, such as a mean difference, or a ratio scale, such as an odds ratio. The reference line must match the scale: zero for an additive effect and one for a ratio. If a model is fit on the log-odds-ratio scale, plot either the log effects with zero as the reference or transform posterior draws to the odds-ratio scale and use one as the reference. Do not exponentiate the endpoints and then treat the resulting interval as symmetric.

A table is a useful alternative when readers need exact values, basket labels, or several decimal places, but it makes relative interval widths and the location of a common reference harder to scan. A forest plot is often more effective for visual comparison, while a table can accompany it for precise reporting. Raw, unpooled estimates can also be shown for context, but they answer a different visual question from partially pooled posterior estimates and may be much less stable in sparse baskets. State clearly which marks are raw and which come from the hierarchical model.

Sorting rows by posterior estimate can make the display easier to scan, but the order is uncertain and should not be mistaken for evidence that adjacent baskets differ. Keeping a clinically meaningful, prespecified order may better support interpretation. Either choice should be disclosed, and the basket labels must remain identifiable.

Diagnostics, Failure Modes, and Limitations

A forest plot is a communication tool, not a diagnostic for every aspect of model fit. Inspect the fitted model’s diagnostics separately, including whether posterior computation has converged and whether the posterior draws adequately represent the target distributions. In the plot itself, look for presentation problems: intervals cut off by the axis, inconsistent scales, unclear direction of benefit, or an apparent shared center that is actually created by a truncated axis. If an interval extends beyond the plotted range, indicate that explicitly rather than silently clipping it.

• Do not use interval overlap as a test of difference. Two credible intervals can overlap even when the posterior probability of a contrast being positive is high, or fail to overlap without directly expressing the posterior probability of a particular contrast. A plot of marginal intervals is not a substitute for analyzing the contrast of interest.
• Do not equate a narrow interval for \(\mu\) with small \(\tau\). The population mean can be estimated precisely while basket effects remain quite variable. Read the posterior summary for \(\tau\) separately.
• Do not treat a point estimate as the whole posterior. The center and interval together communicate location and uncertainty. A ranking by centers alone can hide substantial uncertainty.
• Do not use a shared axis without checking its range. An axis that is too narrow can exaggerate differences; one that is unnecessarily wide can make relevant uncertainty unreadable. Include the reference value and all intervals or mark any truncation.

The plot’s conclusions are limited by the model, prior, chosen effect measure, and available data. In particular, the model’s distributional assumptions and the amount of information in sparse baskets affect the posterior summaries being plotted. The visual display does not establish that the hierarchy is appropriate, that every basket is exchangeable, or that a clinical decision follows from any particular interval. It should be read alongside the model description and the substantive clinical thresholds.