From a Posterior Probability to a Trial Decision
A Bayesian analysis can quantify how much posterior probability supports a treatment effect in a specified direction. A trial team can use that probability in a decision rule: proceed when the posterior probability of superiority reaches a threshold chosen in advance. The rule is simple to state, but it is meaningful only when the treatment-effect direction, probability threshold, and decision being made are clear.
Let \(\theta\) denote the treatment effect for the trial's chosen estimand, with larger values defined as more beneficial for the experimental treatment than for the comparator. This direction must be set before calculating a superiority probability. In this tutorial, superiority means \(\theta>0\). The boundary zero is the null boundary for the rule considered here; it is not a claim that every positive effect is clinically important.
The probability is conditional on the observed data and on the statistical model, including its likelihood and prior. The threshold \(\gamma\) is part of the decision rule, not an output of the posterior calculation. It should be specified before the result is examined. A go decision means that the chosen rule has been met; by itself, it does not establish that the treatment is effective in every relevant sense or that the effect is large enough to matter to patients.
Calculating the Probability Under a Normal Posterior
Suppose the posterior distribution of \(\theta\), conditional on the data, is normal with posterior mean \(m\) and posterior standard deviation \(\tau\), where \(\tau>0\). Write this as \(\theta\mid D\sim N(m,\tau^2)\). Standardizing \(\theta\) gives the probability of exceeding zero:
where \(\Phi\) is the cumulative distribution function of a standard normal random variable. This calculation uses the assumed normal posterior; it is not a general formula for every posterior distribution. For a non-normal posterior, calculate the probability from its distribution function or estimate it from posterior draws.
Worked Example: A Normal Posterior Clears the Threshold
In a hypothetical trial, let \(\theta\) be the experimental-minus-comparator effect on a symptom-improvement scale, measured in points, with positive values favoring the experimental treatment. Suppose the posterior is \(\theta\mid D\sim N(0.18,0.10^2)\), and the prespecified go threshold is \(\gamma=0.95\).
The normal-posterior formula gives \(m=0.18\) points and \(\tau=0.10\) points, so \(m/\tau=0.18/0.10=1.80\). Therefore,
Because \(0.9641\geq0.95\), the trial meets this go rule. The interpretation is that, conditional on the observed data and the specified model and prior, the posterior probability that the treatment effect is positive is about 96.4%. The rule does not say that there is a 96.4% probability that the decision itself is correct.
The Threshold Is a Separate Part of the Rule
The posterior distribution determines the probability of superiority; the threshold determines how that probability maps to a decision. Holding the posterior probability fixed while changing \(\gamma\) can change the decision. This is not a recalculation of the evidence. It is a change in the decision rule.
Worked Example: The Same Posterior, Two Thresholds
Consider a hypothetical trial in which positive values of \(\theta\) favor the experimental treatment. Suppose \(\theta\mid D\sim N(0.08,0.12^2)\), with \(\theta\) measured in effect-scale points. Then \(m/\tau=0.08/0.12=0.6667\), giving
At a prespecified threshold of \(\gamma=0.70\), the rule says go because \(0.7475\geq0.70\). At a threshold of \(\gamma=0.80\), it says no-go because \(0.7475<0.80\). The posterior has not changed: only the threshold has. Neither threshold can be called universally correct based on this calculation alone; selecting one is a decision-design question, not a property of the normal distribution.
The comparison also illustrates why a reported posterior probability should not be presented without its threshold when the purpose is to communicate a go decision. A probability such as 0.7475 is a posterior summary. “Go” or “no-go” is the result of applying a specified rule to that summary.
Estimating the Probability From Posterior Draws
When posterior draws are available, the probability \(P(\theta>0\mid D)\) can be estimated by the proportion of draws greater than zero. If \(\theta^{(1)},\ldots,\theta^{(M)}\) are draws from the posterior, an estimate is
where \(I(\cdot)\) equals 1 when its condition is true and 0 otherwise. This estimate is a Monte Carlo approximation to the posterior probability, not a new definition of it. Its accuracy depends on the quality and effective amount of the simulation output. In particular, correlated draws do not provide as much information as the same number of independent draws.
Worked Example: Applying a Threshold to Posterior Draws
In a hypothetical binary-endpoint trial, define \(\theta\) as the experimental-minus-comparator difference in response probabilities, so positive values favor the experimental treatment. After assessing the sampling output, suppose the analysis uses \(M=10{,}000\) posterior draws and finds that 8,740 are greater than zero. The estimated posterior probability is
If the prespecified threshold is \(\gamma=0.87\), the estimated probability meets the rule and the result is go. If the threshold is \(\gamma=0.90\), it does not meet the rule and the result is no-go. The estimated probability is the same in both comparisons.
For a rough Monte Carlo precision check, treat the indicators \(I(\theta^{(m)}>0)\) as Bernoulli observations with estimated probability 0.874. Under an independence approximation, their estimated standard error is
This approximation suggests modest Monte Carlo variation around the estimate, but it can understate uncertainty when draws are autocorrelated. The effective sample size and Monte Carlo standard error should be considered when available. A result close to \(\gamma\) deserves particular care: Monte Carlo error can make it unclear whether the computed probability reliably lies on one side of the threshold. That uncertainty concerns the numerical approximation, not the definition of the go rule.
What the Rule Does—and Does Not—Establish
A posterior-probability go rule is transparent: it identifies an event, computes its posterior probability, and compares that probability with a stated threshold. Its simplicity makes it useful for prespecified trial decisions, but it cannot replace careful specification of the estimand, model, prior, data handling, and decision context. A probability of superiority is conditional on those choices; substantial sensitivity to a plausible alternative model or prior may affect the decision.
The rule also reduces a full posterior distribution to one number. Different posterior shapes can yield similar probabilities of exceeding zero while having different means, spreads, or tail behavior. Conversely, a positive posterior mean does not guarantee a high probability of a positive effect. For example, a diffuse or asymmetric posterior may put considerable probability below zero even when its mean is above zero.
Worked Example: A Positive Mean Does Not Guarantee a Go
For illustration, suppose a hypothetical posterior is a mixture: with probability 0.80, \(\theta\) follows \(N(0.20,0.05^2)\); with probability 0.20, it follows \(N(-0.40,0.05^2)\). Positive effects favor the experimental treatment, and \(\theta\) is measured in effect-scale points. The posterior mean is
The first component has probability essentially 1 of being above zero, since its standardized boundary is \((0-0.20)/0.05=-4\). The second has probability essentially 0, since its boundary is \((0-(-0.40))/0.05=8\). Thus the mixture probability of superiority is approximately \(0.80(1)+0.20(0)=0.80\). At \(\gamma=0.90\), the decision is no-go, despite the positive posterior mean. The probability rule depends on the posterior mass above zero, not on the sign of the mean alone.
A no-go result means only that the specified threshold was not reached. It does not prove that the effect is zero, that the comparator is better, or that the model assigns no probability to benefit. Likewise, a go result means the rule was met; it does not by itself establish clinical importance, guarantee replication, or account for every consequence of the action. These interpretations are especially important when communicating a binary decision derived from a continuous posterior.
Assumptions, Diagnostics, and Common Pitfalls
Before applying the rule, check that the treatment-effect parameter has the intended interpretation and sign convention; that the posterior is computed under the prespecified model and data; and that the threshold is defined for the decision at hand. If posterior draws are used, check the relevant computation diagnostics, including convergence, mixing, effective sample size, and Monte Carlo error. Poorly explored tails or chains that have not converged can make an estimated probability misleading even when the arithmetic is correct.
A useful calculation check is to compare independent ways of summarizing the same posterior when possible. For a normal posterior, compare a numerical posterior tail probability with \(\Phi(m/\tau)\). For simulation, inspect the distribution of draws around zero and confirm that the counted event matches the chosen sign convention. If the treatment effect was coded so that negative values favor treatment, counting draws greater than zero would answer the wrong question.
| Approach | What it provides | Important limitation |
|---|---|---|
| Posterior probability of \(\theta>0\) | Direct probability for the specified superiority event and a simple threshold rule | Depends on model and prior; ignores effect magnitude beyond the boundary |
| Posterior mean of \(\theta\) | Posterior average effect on the estimand's scale | Does not give the probability that the effect is positive |
| Credible interval | A posterior interval describing uncertainty about \(\theta\) | An interval-based decision is a different rule and need not match the probability threshold |
Common mistake: “The posterior mean is positive, so the trial meets the superiority go criterion.” This is incorrect because a mean above zero does not determine how much posterior probability lies above zero. The correction is to calculate \(P(\theta>0\mid D)\) from the posterior and compare that probability with the prespecified \(\gamma\). Another error is to interpret a posterior probability as a frequentist p-value or as the probability that the null hypothesis is true; it is neither.
Check Your Understanding
Answer each question, then compare with the solutions.
- A hypothetical posterior is \(\theta\mid D\sim N(0.10,0.05^2)\), with positive values favorable and \(\gamma=0.95\). Calculate the posterior probability of superiority and state the decision.
- From 2,000 posterior draws, 1,720 are above zero. What is the estimated probability, and what decision follows at \(\gamma=0.90\)?
- If a posterior probability is 0.82, explain how the decision changes between thresholds 0.80 and 0.85. Has the posterior evidence changed?
- A colleague says, “The go decision means there is a 95% probability that the treatment is clinically important.” Identify the error and give a correct interpretation of what the rule says.
Solutions. (1) \(m/\tau=0.10/0.05=2\), so \(P(\theta>0\mid D)=\Phi(2)\approx0.9772\). Since \(0.9772\geq0.95\), the decision is go. (2) The estimate is \(1{,}720/2{,}000=0.86\). Since \(0.86<0.90\), the decision is no-go. (3) At 0.80 the decision is go; at 0.85 it is no-go. The posterior probability remains 0.82, so only the threshold and resulting decision change. (4) The rule concerns posterior probability that the defined effect exceeds zero, conditional on the data and model. It does not establish a 95% probability of clinical importance; the go decision means only that the specified probability threshold was met.