From Posterior Evidence to Three Actions
A trial decision need not force an immediate choice between advancing a treatment and stopping it. When the evidence is neither persuasive enough to justify a go decision nor weak enough to justify a no-go decision, a third action—continue gathering information—can be useful. The rule in this tutorial assigns each possible posterior probability to exactly one of these three actions.
Let \(\theta\) be the treatment effect, defined so that larger values represent greater benefit. For an endpoint on which lower values are better, define the effect with the sign reversed if needed. Let \(\delta>0\) be the prespecified margin for a clinically meaningful benefit. The relevant posterior probability is the probability, given the observed trial data \(D\), that the treatment effect exceeds that margin.
The no-go condition concerns the posterior probability of exceeding the meaningful-benefit margin, not simply the probability of any positive effect. A treatment could have a high probability of a small positive effect but still have a low probability of exceeding \(\delta\). Similarly, a go decision under this rule means that the prespecified posterior criterion is met; it does not, by itself, establish safety, regulatory approval, or clinical usefulness in every setting.
Assumptions and Calculation
The rule is meaningful only when its inputs are defined clearly. The endpoint, treatment-effect direction, meaningful-benefit margin, analysis model, and action thresholds should be specified before using the observed result to make the decision. The posterior distribution must be based on an appropriate likelihood and prior for the trial design and data. These choices matter because \(q\) is conditional on that model and prior.
The rule does not require a particular posterior distribution. If the posterior density of \(\theta\) is \(f(\theta\mid D)\), then \(q\) is the posterior mass above the margin:
For example, if the posterior is normal, \(\theta\mid D\sim N(\mu_D,\sigma_D^2)\), then standardizing the margin gives
where \(\mu_D\) and \(\sigma_D\) are the posterior mean and posterior standard deviation, and \(\Phi\) is the standard normal cumulative distribution function. This calculation assumes the stated normal posterior is an adequate representation of uncertainty under the chosen model. Other posterior distributions require their own tail-probability calculation.
State what \(\theta\) measures, orient it so larger values are better, and specify \(\delta\) in the endpoint’s units.
Find \(q=P(\theta>\delta\mid D)\) from the fitted posterior, using an analytic calculation or posterior draws.
Compare \(q\) with the prespecified \(\gamma_N\) and \(\gamma_G\), and report the resulting action with the criterion and context.
Worked Decisions With a Normal Posterior
The examples use a hypothetical continuous endpoint: improvement in a symptom score, measured in points, with larger improvement being better. Suppose the meaningful-benefit margin is \(\delta=0.20\) points and the prespecified thresholds are \(\gamma_N=0.10\) and \(\gamma_G=0.90\). These values illustrate a decision rule; they are not universal recommendations.
Worked Example: Posterior Evidence Meets the Go Threshold
Suppose a hypothetical trial analysis gives \(\theta\mid D\sim N(0.45,0.15^2)\), where \(\theta\) is the active-treatment improvement minus the control improvement, in symptom-score points. The posterior model is assumed appropriate for this illustration, and benefit is oriented positively. Calculate \(q\) and the action.
The standardized margin is
Therefore,
Because \(0.9522\geq0.90\), the rule gives a go decision. In context, under the specified model and prior, the posterior probability that improvement exceeds 0.20 points is about 95.2%, which meets the prespecified go criterion. It is not a statement that there is a 95.2% probability the treatment will succeed in a future trial.
Worked Example: Posterior Evidence Meets the No-Go Threshold
In a second hypothetical analysis of the same endpoint and margin, suppose \(\theta\mid D\sim N(0.02,0.10^2)\). Use the same thresholds to determine the action.
The standardized difference from the margin is
Thus,
Since \(0.03593\leq0.10\), the rule gives a no-go decision. Under this posterior, there is about a 3.59% probability that the treatment effect exceeds the meaningful-benefit margin. This is evidence against meeting that particular benefit criterion; it is not proof that the treatment has exactly no effect.
Worked Example: Posterior Evidence Falls in the Continue Region
A third hypothetical analysis gives \(\theta\mid D\sim N(0.30,0.10^2)\), again in symptom-score points. Calculate the probability of exceeding 0.20 points and apply the same rule.
Here,
The probability \(0.8413\) is greater than \(0.10\) but less than \(0.90\). The action is therefore continue. The posterior puts substantial probability on meaningful benefit, but it does not reach the go threshold; nor is the probability low enough for no-go. The action describes the prespecified rule’s response to current evidence, not a prediction that additional data will necessarily change the decision.
Why the Rule Forms Three Nonoverlapping Regions
The thresholds divide the probability scale into low, intermediate, and high regions. The ordering condition \(\gamma_N<\gamma_G\) is essential: it ensures that the no-go and go conditions cannot hold simultaneously. To see that the actions cover every possible value of \(q\), note that \(q\) must lie in \([0,1]\). If it is at or below \(\gamma_N\), the rule assigns no-go. If it is at or above \(\gamma_G\), it assigns go. Every remaining value lies strictly between the thresholds and is assigned continue.
The inclusive boundary choices also make the rule operational: \(q=\gamma_N\) is no-go and \(q=\gamma_G\) is go. A protocol could define a different boundary convention, but it must be explicit and must leave no probability value unassigned or assigned to two actions. This partition is the direct justification for the three-action rule; it does not derive the threshold values themselves.
In practice, the thresholds express a decision policy chosen for the trial context. They are not calculated from the posterior probability alone. The introductory material on posterior decision rules for either-or endpoint criteria focuses on a binary action; adding a continue region retains the same posterior evidence measure while allowing an intermediate response. How thresholds can be derived from a formal expected-loss specification is a separate decision-theoretic question.
Diagnostics, Alternatives, and Limitations
Several checks help detect an incorrectly applied rule. First, check the sign and units of \(\theta\) and \(\delta\). If higher values of the recorded endpoint mean worse outcomes but the analysis treats them as better, the posterior tail is in the wrong direction. Second, confirm that the meaningful-benefit margin and thresholds were specified independently of the observed result. Third, verify that \(\gamma_N<\gamma_G\), and check the calculation of the tail probability against the fitted posterior.
If \(q\) is estimated from posterior simulation rather than computed analytically, the estimated probability is the fraction of retained posterior draws above \(\delta\). Its simulation error can matter when the estimate is close to a decision threshold. Increasing the effective number of draws and checking Monte Carlo error can help distinguish a stable classification from one that could change through simulation noise. Poor mixing or other sampling problems can undermine the posterior estimate itself, so computational diagnostics remain relevant.
A two-action rule forces every result into go or no-go, even when evidence is intermediate. A three-action rule makes that middle region explicit, but it does not prescribe what “continue” entails: that action might mean collecting more data, proceeding to a planned next stage, or deferring a decision, depending on the protocol. A credible-interval rule is another possible summary, but it is not automatically equivalent to a tail-probability rule unless its boundaries and probability criterion are constructed to match.
The rule has important limitations. It compresses the posterior distribution into one number, so two posteriors with different shapes can yield the same \(q\). It also relies on a single meaningful-benefit margin and does not by itself incorporate safety, treatment burden, subgroup effects, or other clinical considerations. Its conclusions are conditional on the model, prior, endpoint definition, and chosen thresholds. Report those elements alongside the action rather than presenting “go” or “no-go” as a context-free property of the data.
Check Your Understanding
Use the rule \(q\leq0.10\) for no-go, \(q\geq0.90\) for go, and otherwise continue. Give the decision and a brief interpretation for each problem.
- For a hypothetical trial, \(q=0.90\). Which action applies at this boundary?
- A treatment’s estimated posterior probability of any positive effect is 0.94, but its probability of exceeding the meaningful-benefit margin is 0.87. What action applies?
- Suppose \(\theta\mid D\sim N(0.24,0.10^2)\) and \(\delta=0.20\). Calculate \(q\) and state the action.
- Why must \(\gamma_N<\gamma_G\) for the rule as defined?
Solutions. (1) Since \(q=0.90\geq\gamma_G\), the action is go; the upper boundary is included. (2) Use the probability of exceeding the margin, \(q=0.87\), not the probability of any positive effect. Because \(0.10<0.87<0.90\), the action is continue. (3) Standardize: \(z=(0.24-0.20)/0.10=0.400\). Therefore \(q=\Phi(0.400)\approx0.6554\), which lies between 0.10 and 0.90, so the action is continue. (4) If the thresholds were not ordered, the no-go and go regions could overlap or leave the intended intermediate region ill-defined. With \(\gamma_N<\gamma_G\), the three conditions partition the probability scale without overlap.