From Posterior Evidence to an Action
A posterior probability is an evidence summary, not an action by itself. For example, a statement that the probability of a clinically meaningful treatment benefit is 0.93 describes uncertainty about an effect under a specified model and data set. It does not, on its own, establish that a treatment should be adopted, advanced to a larger trial, or rejected. Those actions also depend on the clinical question, the consequences of being wrong, and the decision rule specified for the setting.
The task here is to make that translation explicit. First define the event that matters clinically; then calculate its posterior probability; finally apply a decision rule to choose among the available actions. These steps keep the scientific evidence separate from the policy used to act on that evidence.
A useful decision rule begins with a precise estimand and an interpretable event. Suppose larger values of \(\delta\) represent greater treatment benefit, and let \(m>0\) be the smallest benefit worth pursuing. Then \(E=\{\delta>m\}\) expresses clinically meaningful superiority. If safety matters as well, the event might be \(E=\{\delta>m,\ h<h_{\max}\}\), where \(h\) is an adverse-event effect coded so larger values mean more harm, and \(h_{\max}\) is the largest acceptable value.
The direction of each effect, the units, the margin, the safety condition, and the posterior model must be stated before interpreting a probability. The relevant posterior is conditional on the observed data and the model, including its likelihood and prior. A probability computed for the wrong direction or an inappropriate margin can be arithmetically correct but clinically irrelevant.
Specify the Action Rule Before Applying It
A simple three-action rule assigns regions of posterior evidence to actions. For example, a development program might go forward when \(q\) reaches a prespecified high threshold, stop when \(q\) falls below a lower threshold, and obtain further evidence in between. The thresholds depend on the decision context; they are not supplied automatically by Bayesian probability.
The middle action is meaningful only if additional evidence is possible and worth its cost. In a completed trial with no realistic opportunity for more data, the operational choice may be between go and no-go; a three-region rule should not disguise that constraint. Conversely, in an interim setting, “further evidence” may mean continuing the trial, collecting additional safety data, or conducting a confirmatory study. It is not a claim that the treatment works or does not work.
This structure extends the posterior go rules and clinical margins introduced earlier in the course. The margin defines what counts as a meaningful effect; the action rule specifies what to do with the posterior evidence about that event. These are separate choices and should be documented separately.
Worked Example: Efficacy and Safety Together
Suppose a hypothetical trial estimates treatment benefit \(\delta\) in points on a clinical scale, with larger values favoring treatment. A benefit above 1 point is considered clinically meaningful. Let \(h\) be the absolute increase in serious adverse-event risk, measured as a proportion, with larger values worse; values below 0.03 meet the prespecified safety condition. Assume the fitted posterior distributions are \(\delta\mid\text{data}\sim N(2.5,1^2)\) points and \(h\mid\text{data}\sim N(0.01,0.02^2)\), and assume their posterior distributions are independent. These are hypothetical posterior summaries, not data from a real study.
The joint event of interest is \(E=\{\delta>1,\ h<0.03\}\). For the efficacy component, standardizing gives \(z=(1-2.5)/1=-1.5\), so \(P(\delta>1\mid\text{data})=\Phi(1.5)=0.9332\), where \(\Phi\) is the standard normal cumulative distribution function. For safety, \(z=(0.03-0.01)/0.02=1\), so \(P(h<0.03\mid\text{data})=\Phi(1)=0.8413\).
By the stated posterior independence assumption, the joint probability is the product: \(q=P(E\mid\text{data})=0.9332\times0.8413=0.7851\) (to four significant figures). Independence matters: without it, multiplying the two marginal probabilities would not generally give the joint probability.
Suppose the prespecified policy is go for \(q\ge0.80\), no-go for \(q\le0.20\), and further evidence otherwise. Since \(0.20<0.7851<0.80\), this rule selects further evidence. The evidence statement remains that the posterior probability of meeting both stated conditions is about 0.785; the action follows only after applying the policy. It would be incorrect to report that the treatment has a 78.5% probability of being “the right decision,” because the calculated event is the explicitly defined clinical event, not the correctness of an action.
Worked Translations in Different Settings
The same logic applies to different endpoints, but the event and its units must match the clinical question. A posterior probability for a binary response difference, a continuous mean difference, or a hazard-ratio criterion is not interchangeable with a probability for some other estimand. The earlier course material on those endpoint-specific rules establishes how to define the relevant margins; here the emphasis is on translating the resulting evidence into an action.
Worked Example: A Binary-Endpoint Go Rule
Consider a hypothetical trial where the estimand is the treatment-minus-control difference in response probabilities, \(\delta=p_T-p_C\), in percentage-point units. Positive values favor treatment. The prespecified meaningful-benefit event is \(\delta>5\) percentage points. A posterior analysis gives \(\delta\mid\text{data}\) approximately \(N(8,4^2)\) percentage points. Assume this approximation adequately represents the posterior in the decision-relevant range. The policy is go if \(P(\delta>5\mid\text{data})\ge0.90\), and no-go if it is at most 0.10; otherwise obtain more evidence if feasible.
Standardize the margin under the stated posterior: \(z=(5-8)/4=-0.75\). Therefore \(P(\delta>5\mid\text{data})=1-\Phi(-0.75)=\Phi(0.75)=0.7734\).
Because \(0.10<0.7734<0.90\), the rule selects further evidence, not go. The posterior mean difference of 8 percentage points exceeds the 5-point margin, but a mean alone does not determine whether the posterior probability clears the prespecified threshold. In this hypothetical case, the posterior probability of exceeding the margin is about 0.773, which falls short of 0.90.
The normal approximation is an assumption to check against the fitted posterior, especially near the margin. If the posterior is skewed or has substantial mass in a region poorly represented by the approximation, the probability should instead be calculated from the appropriate posterior distribution, for example by posterior draws. The units and direction must also be checked: reversing the subtraction to \(p_C-p_T\) would reverse the interpretation.
Some settings use explicit expected consequences rather than fixed probability regions. This is another way to translate evidence into action, not a different interpretation of the posterior probability. For a simple two-state example, suppose the clinically relevant event is \(E\), with posterior probability \(q\), and the available actions are go or stop. If going when \(E\) is false incurs loss \(L_G\), and stopping when \(E\) is true incurs loss \(L_S\), while correct actions have zero loss, then the posterior expected losses are \(L_G(1-q)\) for go and \(L_Sq\) for stop. Choose go when \(L_G(1-q)<L_Sq\), equivalently \(q>L_G/(L_G+L_S)\). The loss values must reflect the decision context; the resulting cutoff is not universal.
Worked Example: Choosing Between Go and No-Go by Expected Loss
Return to a hypothetical decision with \(q=0.7851\), now assuming no further evidence can be collected. Suppose a false-go decision has loss 8 units and a false-stop decision has loss 2 units; correct decisions have zero loss. These are illustrative decision weights, not clinical measurements.
The posterior expected loss of go is \(8(1-0.7851)=8(0.2149)=1.7192\) loss units. The posterior expected loss of no-go is \(2(0.7851)=1.5702\) loss units. Since \(1.5702<1.7192\), no-go has the smaller expected loss under these assumptions.
The equivalent threshold is \(q>8/(8+2)=0.80\) for go, so \(q=0.7851\) falls below it. The result is consistent with the direct loss calculation. If the losses changed, the threshold and possibly the chosen action would change even though the posterior evidence \(q\) stayed the same. This illustrates why evidence and policy must be reported distinctly.
Justification and Practical Checks
The translation has a short but important logical justification. A posterior probability summarizes uncertainty about an event under the model. A decision rule then partitions possible evidence summaries into action regions. In the two-action loss example, the preferred action follows by comparing conditional expected losses, not by relabeling \(q\) as a decision.
Write the clinically meaningful efficacy condition, safety condition, and direction of benefit in terms of the estimand.
Calculate \(q=P(E\mid\text{data})\) under the stated model, checking that any approximation or dependence assumption is appropriate.
Use prespecified probability thresholds or compare posterior expected losses for the available actions.
State the posterior evidence in context and separately name the action selected by the stated policy.
Several diagnostics help reveal when a translation is going wrong:
- Check the event against the clinical question. A superiority probability does not answer a question about noninferiority, equivalence, or acceptable safety unless the event explicitly encodes that question.
- Check the scale and direction. A risk difference is not a risk ratio, and treatment-minus-control has the opposite sign from control-minus-treatment.
- Check joint calculations. Multiplication of marginal probabilities requires posterior independence; otherwise use the joint posterior distribution or posterior draws.
- Check the policy before interpreting its result. A threshold selected after inspecting the posterior can make the action appear more objective than it is.
- Check whether the middle action is feasible. “Further evidence” requires a realistic opportunity and a reason that additional information could improve the decision.
A posterior probability threshold is not the only possible approach. A threshold rule is transparent and easy to communicate when a committee has agreed on an event and action regions. Expected-loss analysis makes the consequences of actions explicit and can accommodate more than two states or actions, but its conclusions depend on credible and sufficiently complete loss assignments. Clinical judgment may also consider treatment burden, feasibility, equity, and evidence outside the trial; those considerations should be made explicit rather than hidden inside an unexplained cutoff.
These approaches share an important limitation: the action is only as defensible as the estimand, model, and policy assumptions behind it. Posterior probabilities can be sensitive to prior choices or model misspecification, particularly with limited data. A rule that ignores uncertainty about the model, relevant patient subgroups, or long-term harms may create a false sense of precision. Sensitivity analyses and clear documentation can show whether the action changes under reasonable alternatives. They do not convert a decision rule into a guarantee of benefit.
Common Mistake: Treating Evidence as the Decision
A common error is to say, “The posterior probability is 0.93, so the treatment should go forward.” That conclusion is incomplete. The probability must refer to a clearly defined clinical event, and the action must follow from an agreed policy or comparison of consequences. A probability of 0.93 might support go under one policy, further evidence under another, or no-go under an unusual but explicitly justified set of losses. The correction is to report both statements: what the posterior says about the defined event, and what the specified decision rule recommends.
Check Your Understanding
For each question, identify the evidence statement separately from the action or explain the calculation.
- A policy says go when \(q\ge0.85\), no-go when \(q\le0.15\), and further evidence otherwise. If \(q=0.82\), what action follows, and what does \(q\) mean?
- A treatment has posterior probability 0.91 of exceeding the clinical margin, but the policy requires 0.95 for go. Explain why “go” does not follow from the evidence alone.
- Suppose efficacy and safety marginal posterior probabilities are 0.90 and 0.80. What additional condition is needed to multiply them to obtain the joint probability?
- With \(q=0.70\), false-go loss 6 units, and false-stop loss 3 units, calculate the expected losses of go and no-go. Which action minimizes expected loss?
Solutions. (1) Further evidence, because \(0.15<0.82<0.85\). The value 0.82 is the posterior probability of the prespecified clinical event conditional on the data and model. (2) The policy's go threshold is not met: \(0.91<0.95\). The evidence is a 0.91 posterior probability for the defined event; under this policy it does not trigger go. (3) The two events must be independent under the joint posterior, or their dependence must otherwise be incorporated in a direct joint calculation. Under independence only, the product would be \(0.90\times0.80=0.72\). (4) Go loss is \(6(1-0.70)=1.8\) units; no-go loss is \(3(0.70)=2.1\) units. Go has the smaller expected loss under the stated loss assumptions.