Interpreting Posterior Summaries for a Clinical Estimand

What a Posterior Summary Describes

A clinical estimand specifies the quantity a trial is intended to address: for example, a difference in mean symptom change, a risk difference, or a difference in restricted mean survival time. Once a Bayesian analysis has produced a posterior distribution for that estimand, summaries can make the distribution easier to communicate. Their meaning depends on the estimand: a numerical summary without its target, scale, direction, and units is not a complete clinical interpretation.

Let \(\psi\) denote a specified clinical estimand and \(D\) the observed trial data. The posterior distribution \(p(\psi\mid D)\) describes uncertainty about \(\psi\) conditional on the data, the statistical model, and the prior distribution. It is not a distribution of outcomes across individual patients. In particular, a posterior interval for an average treatment effect does not say that most individual patients’ effects lie within that interval.

Under the usual inverse-CDF convention, an equal-tailed quantile interval has endpoints at the \(\alpha/2\) and \(1-\alpha/2\) posterior quantiles. For a discrete posterior, each excluded tail has probability at most \(\alpha/2\), so the interval can have posterior probability greater than \(1-\alpha\); exact tail allocation may require randomization. Other credible intervals, such as highest-posterior-density intervals, use a different construction. When the posterior is skewed or multimodal, interval choice can affect what the endpoints convey. The interval type should therefore be stated when it matters.

Choosing and Interpreting a Summary

The posterior mean uses all values in the distribution, weighted by their posterior probabilities. It is the value minimizing posterior expected squared-error loss. The posterior median is less affected by extreme values and minimizes posterior expected absolute-error loss. Neither is automatically the best summary in every clinical setting: the choice depends on the distribution’s shape and on which loss or decision is relevant. Reporting both can be useful when the posterior is notably asymmetric.

A credible interval communicates the range and uncertainty of plausible values under the posterior. A 95% credible interval has a direct conditional probability interpretation: given the model, prior, and observed data, the posterior probability that \(\psi\) lies in the interval is at least 0.95. This is not a claim that 95% of individual outcomes lie there, nor a statement about the long-run coverage of a procedure over repeated trials.

When clinical interpretation depends on a threshold, a posterior probability can be more informative than an interval alone. If positive values represent benefit and \(\delta\) is a prespecified clinically meaningful benefit, \(P(\psi\geq\delta\mid D)\) is the posterior probability that the estimand reaches that threshold. The threshold must be expressed on the same scale and in the same units as \(\psi\). This probability is not the probability that a hypothesis is true in a model-free sense; it is calculated from the specified posterior distribution.

Scale, Direction, and Posterior Shape

A treatment effect can be expressed on several scales, and the same clinical comparison can yield different summaries on different scales. For a binary outcome, a risk difference is measured in probability units (often reported as percentage points), whereas a risk ratio is dimensionless. A mean difference in symptom score uses score points; a restricted mean survival-time difference uses time units. State which treatment is subtracted from which, because the sign of a contrast depends on that convention.

A symmetric posterior has equal mean and median, but skewness can separate them. A long right tail can pull the mean above the median. In a discrete or multimodal posterior, the median can fall at a value with substantial probability mass, and a single interval may conceal separated regions of posterior support. Inspecting the distribution, not only its summaries, helps detect these cases. For Monte Carlo posterior output, diagnostics should also establish that the computed summaries are stable and that sampling adequately represents the posterior; an unstable quantile is not repaired by precise-looking decimal places.

Why Estimand Uncertainty Is Not Individual Outcome Variation

A posterior distribution for a treatment-effect estimand represents uncertainty about a population quantity. Individual outcomes vary for additional reasons, including variation between patients and measurement error. A predictive distribution for a future patient’s outcome therefore combines uncertainty about model parameters with outcome variability conditional on those parameters. Its spread is generally not the same as the spread of the posterior distribution for a population mean or treatment effect.

The distinction can be shown using the law of total variance. If \(Y_{\text{new}}\) is a future individual outcome and \(\mu\) is the population mean, then:

The first term is average individual-level variability conditional on the mean; the second is posterior uncertainty about that mean. A credible interval for the mean uses the second component, while prediction for an individual must account for both. This decomposition assumes the predictive model for individual outcomes is appropriate; it does not make a misspecified outcome model adequate.

Justification for the Predictive Variance

The variance decomposition follows by adding and subtracting the conditional mean. Write \(m(\mu)=E(Y_{\text{new}}\mid\mu,D)\). Then \(Y_{\text{new}}-E(Y_{\text{new}}\mid D)\) is the sum of \(Y_{\text{new}}-m(\mu)\) and \(m(\mu)-E(Y_{\text{new}}\mid D)\). On squaring and taking the posterior predictive expectation, the cross-product has expectation zero because \(E\{Y_{\text{new}}-m(\mu)\mid\mu,D\}=0\). The remaining terms are the expected conditional variance and the variance of the conditional mean, respectively. In the example, these are \(100\) and \(4\), giving \(104\). This is why parameter uncertainty alone cannot describe individual outcome variation.

Common Mistakes and Practical Checks

• Leaving out the estimand or its units. “The mean is 2.4” is incomplete. Name the population quantity, contrast direction, and units, such as a 2.4-week mean reduction in time to improvement.
• Calling a credible interval a patient range. A posterior interval for a population estimand concerns uncertainty about that estimand. Use a predictive distribution for future outcomes.
• Interpreting an interval that crosses zero as proof of no effect. It indicates posterior probability on both sides of zero. Report relevant posterior probabilities and the interval, with the treatment-effect direction explicit.
• Treating a posterior probability as model-free certainty. \(P(\psi\geq\delta\mid D)\) depends on the likelihood, prior, and analysis model. Its clinical meaning does not remove those assumptions.
• Reporting only a mean for a skewed posterior. The mean can be pulled by a tail. Compare it with the median, inspect the distribution, and choose interval endpoints appropriate to the distribution’s shape.

Before communicating a posterior summary, check that the analysis targets the intended estimand, the sign convention matches the clinical interpretation, the units are correct, and the reported interval is identified as equal-tailed or otherwise defined. Also verify that the posterior calculation is numerically reliable and that conclusions are not stronger than the model and data support. These checks clarify what the summaries say; they do not substitute for evaluating whether the model represents the estimand.