Interpreting Credible Intervals in Clinical Context

What a Credible Interval Says

A clinical trial estimate is easier to interpret when it is accompanied by an interval describing uncertainty. In Bayesian analysis, a credible interval summarizes a specified part of the posterior distribution for an endpoint effect. Its meaning is conditional on the data, the likelihood model, and the prior used to obtain that posterior. The interval is not a statement that the estimate is exact, nor does it by itself determine whether an effect is clinically important.

Before interpreting an interval, identify the effect being summarized. State the endpoint, the patient population, the treatment contrast, the direction of the contrast, the effect scale, and the units. For example, a difference of 2 points has a different clinical meaning from a ratio of 2, even though both are numerical contrasts. The estimand and the analysis assumptions determine what the posterior effect represents.

Definition: A posterior credible interval with probability \(1-\alpha\) is an interval \(C\) for an unknown effect parameter \(\theta\) such that, conditional on the observed data \(y\), the model, and the prior, \(P(\theta\in C\mid y)=1-\alpha\). A 95% credible interval therefore contains 95% of the posterior probability for \(\theta\), under those specified assumptions.

That probability statement is the defining interpretation. It is conditional on the analysis, rather than a repeated-sampling claim about how often a procedure covers a fixed parameter across hypothetical repetitions. It also does not mean that the prior, likelihood, endpoint definition, or model assumptions have a 95% probability of being correct.

Constructing and Reading an Interval

A common choice is the equal-tailed interval. If \(F(\theta\mid y)\) is the posterior cumulative distribution function, its equal-tailed \(100(1-\alpha)\%\) interval runs from the \(\alpha/2\) quantile to the \(1-\alpha/2\) quantile. For a 95% interval, 2.5% of the posterior probability lies below the lower endpoint and 2.5% above the upper endpoint, when the posterior is continuous.

C_{\mathrm{ET}}= \left[ F^{-1}(\alpha/2\mid y),\ F^{-1}(1-\alpha/2\mid y) \right]

A highest posterior density (HPD) interval instead selects a region containing the specified posterior probability whose points have posterior density at least as high as points outside the region. For a unimodal, symmetric posterior, the equal-tailed and HPD intervals often coincide. For a skewed or multimodal posterior, they can differ; an HPD set may even consist of disjoint regions. A report should name the interval convention, particularly when shape makes that choice consequential.

The endpoints are not sufficient without the interval probability and effect scale. A report such as “95% credible interval: 1.2 to 4.7” leaves readers unable to tell whether the numbers are points, days, or a ratio, and whether the interval is 90%, 95%, or another level. In clinical reporting, give the effect scale and units alongside the interval and describe the posterior probability interpretation in context.

Worked Examples

Worked Example: A Difference in Symptom Scores

Hypothetical data. Suppose the endpoint is change in a symptom score at 12 weeks, measured in points, and the treatment effect \(\delta\) is defined as the mean change with the new treatment minus the mean change with control. Positive values therefore favor the new treatment if larger changes represent greater improvement. The posterior distribution is \(N(1.8,1.0)\), where the second parameter is the variance, so the posterior standard deviation is 1.0 point. Assume the posterior is proper and the reported normal approximation is adequate.

For a normal posterior, an equal-tailed 95% interval is approximately the posterior mean plus or minus 1.96 posterior standard deviations:

1.8\ \text{points}\ \pm 1.96(1.0\ \text{point}) =[-0.16,\ 3.76]\ \text{points}.

Conditional on the data, model, and prior, this interval contains 95% of the posterior probability for the mean difference. Its lower end is slightly below zero and its upper end is above zero. Thus, the interval includes effects in both directions as well as positive effects of several points. If clinicians had specified that a difference of 2 points was important, the interval would also include values below and above that benchmark. The interval alone does not state the probability of exceeding 2 points; that is a different posterior summary.

Worked Example: The Same Effect on a Ratio Scale

Hypothetical analysis. Suppose a positive endpoint is summarized by a treatment-to-control ratio \(R\), and the analysis models \(\theta=\log(R)\). Let the posterior for \(\theta\) be \(N(0.1823,0.0100)\), so its standard deviation is \(0.1000\). This is an illustrative scale transformation, not a claim about a particular endpoint.

First calculate the equal-tailed 95% interval on the log scale:

0.1823\pm1.96(0.1000) =[-0.0137,\ 0.3783].

Exponentiating both endpoints gives the interval on the ratio scale:

[\exp(-0.0137),\ \exp(0.3783)] \approx[0.9864,\ 1.460].

Because exponentiation is an increasing transformation, the transformed interval retains the same posterior probability: conditional on the analysis, \(R\) has 95% of its posterior probability between approximately 0.986 and 1.460. The center on the ratio scale is not obtained by simply exponentiating and treating the log-scale mean as though it were a ratio-scale mean. In particular, interval symmetry on the log scale becomes asymmetry around 1 on the ratio scale. Interpretation must use the endpoint scale actually reported.

Worked Example: How Interval Probability Changes Width

Hypothetical posterior. For a treatment contrast measured in days, suppose the posterior is \(N(5,4)\): its mean is 5 days and its standard deviation is 2 days. The posterior distribution is held fixed while we compare a 90% and a 95% equal-tailed interval. This isolates the effect of the chosen interval probability.

Using normal quantiles \(z_{0.95}\approx1.645\) and \(z_{0.975}\approx1.960\), the 90% interval is:

5\pm1.645(2)=[1.710,\ 8.290]\ \text{days}, \qquad \text{width}=8.290-1.710=6.580\ \text{days}.

The 95% interval is:

5\pm1.960(2)=[1.080,\ 8.920]\ \text{days}, \qquad \text{width}=8.920-1.080=7.840\ \text{days}.

The 95% interval is wider because it must include more of the same posterior distribution. The posterior has not become more uncertain; only the requested probability content has changed. The endpoints should therefore always be reported with the interval level.

Why Width and Scale Matter

Interval width is a useful summary of uncertainty, but it is not an interpretation by itself. A wider interval on the same effect scale and at the same probability level generally reflects a more dispersed posterior. Dispersion can depend on the amount and quality of information, the likelihood, the prior, and modeling choices. Width also increases when the requested posterior probability increases, as the previous example showed.

Widths cannot be compared meaningfully without considering scale and units. A width of 4 may be large for a change measured in percentage points and small for a difference in annual costs measured in thousands of dollars. A nonlinear transformation can also alter interval width and symmetry, even when it preserves the probability content. Clinical interpretation requires both the numerical range and a clear account of what each unit represents.

The interval should be judged against clinically relevant values on the same scale. For a difference, zero is the value of no difference; for a ratio, one is the value of no ratio difference. Whether an interval includes such a reference value is a descriptive observation about its endpoints, not a substitute for considering the full posterior distribution or the clinical consequences of effects across the range.

Justification of the Interval Probability

For a continuous posterior with cumulative distribution function \(F(\theta\mid y)\), the probability that \(\theta\) is no larger than its \(u\)-quantile is \(u\), by the definition of the quantile. Applying this fact at the two equal-tailed endpoints gives the interval probability directly:

\begin{aligned} P\!\left( F^{-1}(\alpha/2\mid y)\leq\theta \leq F^{-1}(1-\alpha/2\mid y) \mid y \right) &=(1-\alpha/2)-(\alpha/2)\\ &=1-\alpha. \end{aligned}

The subtraction works because the posterior probability below the lower endpoint is \(\alpha/2\), and the probability below the upper endpoint is \(1-\alpha/2\). For a discrete posterior, quantiles can have jumps, so an interval formed by these quantile rules can contain more than the nominal probability. The reporting method should then make clear how endpoints were selected and what probability the resulting set actually contains.

Assumptions, Diagnostics, and Alternatives

A credible interval is interpretable only if the posterior distribution is well-defined and the target parameter has a clear meaning. In practice, this requires a stated endpoint and contrast, a likelihood and prior that yield a proper posterior, and an analysis that represents the trial design and data appropriately. A precise interval does not remedy a poorly defined estimand or an implausible model.

For intervals computed from simulation, assess whether the posterior sample adequately represents the target posterior and whether the estimated endpoints are stable enough for reporting. Strong skewness, multiple modes, extreme tails, or sensitivity to modeling assumptions can make a single interval an incomplete summary. In those cases, inspect the posterior shape and consider reporting additional summaries or separate regions rather than relying on interval width alone. These checks concern the validity and adequacy of the reported summary; they do not change the interval’s definition.

The main alternatives answer different questions. An equal-tailed interval gives specified posterior tail probabilities at its two ends. An HPD interval emphasizes regions of highest posterior density and may be more compact, but can be harder to summarize when the posterior is multimodal. A frequentist confidence interval has a different repeated-sampling interpretation; it is not interchangeable with a Bayesian credible interval. Choose and label the summary that matches the analysis and makes its assumptions and clinical meaning understandable.

Common Pitfalls

Calling a credible interval a guarantee. A 95% interval does not guarantee that the true effect is inside it. Its 95% posterior probability statement is conditional on the data, model, and prior.
Omitting the probability level or units. An unlabelled pair of endpoints is not enough to communicate either uncertainty or clinical meaning.
Treating width as an absolute measure. Width depends on the effect scale and interval level as well as posterior dispersion; compare intervals only with those features in view.
Reading an interval as a decision by itself. Whether its range is clinically compelling depends on the endpoint, direction, and relevant clinical context. Merely crossing zero or one does not summarize all the information in the posterior.

Key takeaway: Interpret a credible interval as a probability statement about an explicitly defined effect, conditional on the analysis. Report its probability level, endpoints, scale, and units; then explain what the range means clinically without treating width or a reference-value crossing as a complete conclusion.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A 90% equal-tailed credible interval for a difference in recovery time is \([-1, 5]\) days. State its posterior interpretation and identify what information is still needed for clinical interpretation.
Why is a 95% interval generally wider than a 90% interval when both are computed from the same posterior?
A ratio-scale interval is obtained by exponentiating both endpoints of a log-scale interval. Under what condition does this preserve interval probability?
What is wrong with concluding that a 95% credible interval means there is a 95% probability that the analysis model is correct?

Solutions. (1) Conditional on the data, model, and prior, 90% of the posterior probability for the defined recovery-time difference lies between \(-1\) and \(5\) days. Interpretation also requires the treatment contrast direction, population, and relevant clinical values or thresholds. (2) A 95% interval must include more posterior probability than a 90% interval, so its endpoints usually extend farther into the tails. The posterior itself has not changed. (3) The transformation must be one-to-one and monotone over the parameter range, with endpoints transformed in the corresponding order; exponentiation is strictly increasing, so it preserves probability content. (4) The interval’s posterior probability is conditional on the chosen model and prior. It does not assign probability to the model being correct or account for uncertainty about the analysis assumptions unless that uncertainty is explicitly modeled.