From a Continuous Endpoint to a Go Probability
A continuous endpoint might be a symptom score, blood pressure, or time spent in hospital. A posterior go rule turns uncertainty about the treatment effect on that endpoint into a decision criterion. The calculation is a posterior probability of a specified treatment-benefit event; it is not simply a comparison of observed sample means.
First define the direction of the contrast. In this tutorial, \(\Delta\) is the treatment-group mean minus the control-group mean. If higher endpoint values are better, positive values of \(\Delta\) favor treatment. If lower values are better, define the endpoint or contrast so that positive values still represent benefit, or state explicitly that benefit corresponds to negative values. The direction must be fixed before applying a go rule.
The threshold \(\gamma\) is part of the decision rule, not something determined by the posterior calculation. For instance, a protocol might set \(\gamma=0.95\). The posterior probability is evidence conditional on the model, prior, and observed data; the threshold specifies how much evidence the rule requires before recommending “go.” The zero threshold asks whether there is evidence of benefit in the chosen direction. A positive margin asks whether there is evidence that benefit exceeds a clinically meaningful amount.
Model and Conditions for the Calculation
A basic model for a parallel-group randomized trial assumes that each participant contributes an endpoint value, outcomes are independent between participants conditional on the model, and the endpoint is adequately represented by the chosen distribution. A simple example assumes normally distributed outcomes in each arm, with a common known standard deviation \(\sigma\). It also specifies a prior distribution for the mean difference. More complex analyses may estimate variances, adjust for baseline covariates, or model repeated measurements, but the go probability must still refer to the intended estimand and contrast.
The posterior probability is meaningful only if the model and data analysis are suitable for the trial. Randomization supports a treatment-group comparison, but it does not by itself resolve missing outcomes, protocol deviations, measurement problems, or an unsuitable analysis population. The endpoint definition, analysis population, handling of missing data, and direction of benefit should therefore be prespecified and consistent with the clinical question. A normal model can be a poor description for strongly skewed outcomes or data with influential outliers; model checking and sensitivity analyses matter.
When the posterior distribution of \(\Delta\) is normal, the probability can be calculated by standardizing. If \(\Delta\mid\text{data}\sim N(\mu_{\Delta},\sigma_{\Delta}^{2})\), then
where \(\Phi\) is the standard normal cumulative distribution function. Setting \(m=0\) gives the zero-threshold probability. The posterior standard deviation \(\sigma_{\Delta}\) describes uncertainty in the treatment effect; it is not the endpoint standard deviation among participants.
Worked Example: Zero-Threshold Rule From a Normal Model
Suppose a hypothetical randomized trial measures a continuous outcome in points, with higher scores defined in advance as better. There are 25 participants in each arm. The treatment and control sample means are 108 and 100 points. Assume independent normal outcomes with a common known standard deviation of 10 points, and use the prior \(\Delta\sim N(0,10^2)\). These assumptions provide a simple model for this illustration; in practice, the distributional and prior choices should be justified for the trial.
The observed difference is \(d=108-100=8\) points. Under the stated known-variance model, the sampling variance of this difference is
The likelihood for the difference has variance 8, while the prior variance is 100. Normal-normal updating gives posterior variance \(V=(1/100+1/8)^{-1}=800/108=7.407\) points squared, and posterior mean \(\mu_{\Delta}=V(8/8)=7.407\) points. Thus \(\Delta\mid\text{data}\sim N(7.407,7.407)\), where the second parameter is the variance, and its posterior standard deviation is \(\sqrt{7.407}=2.722\) points.
For the zero-threshold rule,
With a prespecified threshold \(\gamma=0.95\), this rule gives “go,” since \(0.9967\geq0.95\). In this model, the posterior probability that the mean benefit is positive is about 99.67%. It does not say that the treatment has a 99.67% chance of working in every patient, nor does it establish that the benefit is clinically important.
Using a Clinically Meaningful Effect Margin
A zero threshold treats any positive mean difference as benefit, however small. That may not match the clinical objective. A positive margin \(m\) expresses the minimum mean difference regarded as clinically meaningful on the chosen endpoint scale. It must be justified clinically and specified before examining the trial results; changing it after seeing the posterior can make the decision rule misleading.
The margin is not a different estimate of the treatment effect. It changes the event whose posterior probability is assessed. A result can therefore have a high probability of any benefit but a low probability of exceeding the meaningful-effect margin. The difference between these questions is especially important when the posterior distribution is concentrated near zero or when the clinical margin is relatively large.
Worked Example: Applying a Meaningful-Effect Margin
Consider a second hypothetical trial of the same endpoint, in points, with higher values better. Let \(n_T=n_C=25\), the known common standard deviation be 10 points, the prior be \(\Delta\sim N(0,10^2)\), and the observed means be 106 and 100 points. Suppose clinicians have prespecified \(m=3\) points as the minimum meaningful mean benefit, and the decision threshold is \(\gamma=0.80\).
Here \(d=106-100=6\) points and the sampling variance remains \(10^2(1/25+1/25)=8\). The posterior variance is \(V=(1/100+1/8)^{-1}=7.407\), so the posterior mean is \(V(6/8)=5.556\) points and the posterior standard deviation is \(\sqrt{7.407}=2.722\) points.
The probability of exceeding the margin is
Because \(0.826\geq0.80\), the prespecified margin-based rule gives “go.” The interpretation is that, under this model and prior, the posterior probability that the population mean benefit exceeds 3 points is about 82.6%. This is distinct from the probability of any positive benefit, which is higher because the event \(\Delta>3\) is contained within \(\Delta>0\).
More generally, if \(m>0\), then \(\{\Delta>m\}\) is a subset of \(\{\Delta>0\}\). Therefore \(P(\Delta>m\mid\text{data})\leq P(\Delta>0\mid\text{data})\). This simple relationship is a useful check on calculations and interpretation. For a specified \(\gamma\), passing the margin-based rule necessarily passes the zero-threshold rule, but passing the zero-threshold rule does not guarantee passing the margin-based rule.
Estimating the Probability From Posterior Draws
In many analyses, the posterior distribution is represented by MCMC draws rather than a closed-form distribution. For \(M\) retained draws \(\Delta^{(1)},\ldots,\Delta^{(M)}\), estimate the probability of exceeding \(m\) by the fraction of draws that exceed it. This is the Monte Carlo approximation to the posterior probability; the indicator is 1 when a draw exceeds the margin and 0 otherwise.
The calculation assumes that the retained draws adequately represent the target posterior. The earlier material on MCMC diagnostics is relevant here: chain mixing, convergence, effective sample size, and Monte Carlo error affect how accurately the probability is estimated. Treating highly autocorrelated draws as independent can understate simulation error.
Worked Example: A Go Probability Estimated From MCMC Draws
Suppose a hypothetical trial analysis yields 2,000 retained posterior draws for the mean treatment benefit in a symptom score, with higher values better. The prespecified meaningful margin is 2 points. Of the 2,000 draws, 1,850 are greater than 2.
The estimated posterior probability is
Using the independent-draw approximation as a rough Monte Carlo error calculation gives
If \(\gamma=0.90\), the estimated probability exceeds the threshold and the estimated rule gives “go.” The Monte Carlo standard error of about 0.0059 describes simulation uncertainty in estimating the probability, not uncertainty in the clinical effect itself. For correlated MCMC draws, an effective sample size or a suitable chain-based Monte Carlo error estimate is needed rather than the independent-draw approximation. If the estimate is close to the decision threshold, improving simulation accuracy can be important.
Checks, Alternatives, and Limitations
Before acting on a posterior go probability, check that the contrast is oriented correctly, the units of the margin match the endpoint, and the probability corresponds to the prespecified event. Examine whether the fitted model represents the outcome and trial design adequately. For simulation-based results, assess chain behavior and Monte Carlo precision; a small Monte Carlo error cannot repair a misspecified model or biased outcome data.
- Direction errors: If lower values are better but \(\Delta=\mu_T-\mu_C\) is used without reversing the benefit interpretation, the tail probability can point in the wrong direction.
- Scale errors: A margin of 3 points cannot be compared directly with an effect expressed in standardized units unless the conversion is explicitly made.
- Post hoc margins: Selecting \(m\) after inspecting the posterior can make the reported probability appear to answer a question that was not fixed in advance.
- Model and data problems: Outliers, skewness, missing outcomes, or an inappropriate analysis population can alter the posterior and hence the go probability.
- Simulation problems: Too few effectively independent draws or poor chain mixing can make a tail-probability estimate unstable.
A posterior credible interval can summarize uncertainty about \(\Delta\), but a tail probability directly answers the chosen event question. A frequentist confidence interval or hypothesis test is based on a different inferential framework and should not be interpreted as the posterior probability of benefit. These approaches may provide useful complementary summaries, but their numerical statements are not interchangeable. Likewise, a posterior go probability is not by itself a complete clinical decision: consequences of a false go, a false stop, feasibility, safety, and the planned action may also matter.
A common mistake is to conclude, “The mean difference is probably positive, so the treatment is clinically worthwhile.” The correction is to separate the two claims. A zero-threshold probability concerns \(\Delta>0\); a clinically meaningful rule concerns \(\Delta>m\). Report the event, margin, units, posterior probability, threshold, and decision together. The probability is conditional on the specified model and prior, and it does not establish causation beyond the trial's design and analysis assumptions.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A higher-is-better endpoint has posterior \(\Delta\mid\text{data}\sim N(4,2^2)\). What is the posterior probability of any benefit, and would a zero-threshold rule with \(\gamma=0.95\) give “go”?
- For the same posterior, calculate the probability that \(\Delta>3\). Why is it smaller than the probability in Question 1?
- A trial's endpoint is lower-is-better, but the analysis reports \(\Delta=\mu_T-\mu_C\). Which tail represents benefit, and what must be specified to avoid a direction error?
- From 1,000 retained draws, 900 exceed the prespecified margin. Estimate the posterior probability and the rough independent-draw Monte Carlo standard error.
- Why does a posterior probability above a zero threshold not by itself establish a clinically meaningful effect?
Solutions. (1) Standardizing gives \(P(\Delta>0\mid\text{data})=\Phi(4/2)=\Phi(2)\approx0.9772\), so the rule gives “go” at \(\gamma=0.95\). (2) \(P(\Delta>3\mid\text{data})=\Phi((4-3)/2)=\Phi(0.5)\approx0.6915\). It is smaller because \(\Delta>3\) is a narrower event than \(\Delta>0\). (3) For lower-is-better outcomes, benefit corresponds to \(\Delta<0\); state the contrast direction and define the beneficial tail or reorient the endpoint before applying the rule. (4) The estimate is \(900/1000=0.900\), and the rough standard error is \(\sqrt{0.900(0.100)/1000}=\sqrt{0.00009}=0.00949\). (5) The zero-threshold event allows arbitrarily small positive effects. Clinical meaningfulness requires a justified positive margin and evaluation of the posterior probability that the effect exceeds it.