Why a 0.95 Posterior Threshold Is Not Universal

Why a Familiar Cutoff Can Mislead

A posterior probability of 0.95 can look like a natural boundary between “enough evidence” and “not enough.” But the probability alone does not determine what action is appropriate. A trial decision also depends on what event the probability describes and on the consequences of acting when that event is false or not acting when it is true. In one setting, a 0.95 cutoff may be justified; in another, the same cutoff may be too demanding or too permissive.

The event must be defined in clinical terms. For a treatment effect oriented so that larger values are more beneficial, the relevant question might be whether the effect exceeds a prespecified clinically meaningful margin. A high posterior probability of any benefit is not necessarily a high probability of a benefit large enough to matter. The introductory material on posterior go rules establishes how such probabilities can support decisions; here the emphasis is on why the cutoff itself depends on the decision context.

The margin \(M\) and cutoff \(c\) answer different questions. The margin defines what counts as success; the cutoff states how much posterior support is required before taking an action based on that success event. Neither number can substitute for the other. Both should be chosen for the clinical decision rather than selected merely because they are conventional.

Deriving a Threshold From Two Decision Losses

Begin with a deliberately simple decision model. There are two actions: GO, such as advancing the treatment, and STOP, such as not advancing it. There are also two states: the defined success event \(E\) is true or it is false. Suppose the only losses are \(C_{\mathrm{FG}}>0\) for a false-go decision (GO when \(E\) is false) and \(C_{\mathrm{FS}}>0\) for a false-stop decision (STOP when \(E\) is true). Correct decisions have zero loss in this simplified accounting. These losses are on a common scale and reflect the consequences relevant to the decision.

Given the data, the posterior probability of a false-go state is \(1-q\). The posterior expected loss of GO is therefore \(C_{\mathrm{FG}}(1-q)\). The probability of a false-stop state is \(q\), so the expected loss of STOP is \(C_{\mathrm{FS}}q\). Choose the action with the lower posterior expected loss. The derivation is:

Thus, under these assumptions, the loss-based cutoff is \(c=C_{\mathrm{FG}}/(C_{\mathrm{FG}}+C_{\mathrm{FS}})\). If the two losses are equal, \(c=0.5\); if false-go is much more costly than false-stop, the cutoff is high. At exact equality of expected losses the decision-maker is indifferent between the two actions under this model, so a prespecified tie rule may be needed.

This calculation does not establish that a particular set of losses is correct. It makes the implication of a stated loss ratio transparent. For example, \(C_{\mathrm{FG}}=19\) and \(C_{\mathrm{FS}}=1\) imply \(c=19/(19+1)=0.95\). A 0.95 threshold is justified by that particular event definition and loss ratio—not by a universal Bayesian rule.

Different Consequences, Different Cutoffs

The following examples use the same expected-loss rule but different clinical consequences. All numbers are hypothetical. A posterior probability is evidence about a defined event conditional on the data and model; the loss ratio expresses how the decision-maker values the consequences of the actions. Neither determines the other.

Assumptions, Checks, and Failure Modes

The threshold derivation is valid for the two-action, two-state loss model just stated. In particular, it assumes that the event \(E\) has been defined clearly, that \(q\) is the relevant posterior probability under an appropriate analysis, and that the two loss values represent the decision consequences on a common scale. It also assumes that losses are constant within each state-action combination: for example, every false-go has loss \(C_{\mathrm{FG}}\), regardless of how far below \(M\) the true effect lies.

That last assumption can be unrealistic. An effect just below the margin may have different consequences from no benefit or a harmful effect. If the loss depends on the size of \(\theta\), or if there are more than two actions, comparing only \(q\) with a single cutoff may discard important information. The general decision principle remains to compare posterior expected utilities or losses for the available actions, but the two-loss cutoff may no longer summarize that comparison.

Before applying a cutoff, check the following:

• Event alignment: Does \(q\) refer to the effect that matters for the decision, including the correct direction and clinical margin?
• Loss alignment: Do the stated losses capture the actual consequences of false-go and false-stop, including safety, patient burden, opportunity cost, and available alternatives?
• Model adequacy: Is the posterior probability based on a defensible likelihood and prior for this trial? A decision rule cannot repair a poorly specified model.
• Robustness: Would plausible changes to the prior, margin, or loss ratio change the action? A decision close to its boundary deserves particular scrutiny.
• Decision scope: Is GO versus STOP truly the choice at hand? If continuing evidence collection or choosing a different treatment is possible, the two-action calculation does not cover the full decision.

A common mistake is to say, “The posterior probability is above 0.95, so the treatment should go forward.” This skips both the definition of the event and the consequences of the action. The correction is to state the event explicitly, compute its posterior probability, and explain why the cutoff follows from the relevant decision losses or another justified policy. A 0.95 rule can be retained as a prespecified convention, but it should not be presented as universally optimal.

A further distinction is important: a posterior threshold is not a frequentist error-rate guarantee. The loss calculation compares posterior expected losses conditional on the observed data under the specified model. It does not, by itself, guarantee a particular long-run false-go rate or make the posterior probability a probability that the model or null hypothesis is true. Nor does a high probability of exceeding a margin establish that the treatment caused the observed outcome outside the assumptions of the trial design and analysis.

How This Rule Relates to Other Decision Summaries

A posterior tail probability and a credible interval are different summaries of the same posterior distribution. A tail probability directly reports support for the decision event, such as \(\theta>M\). A credible interval summarizes a range of parameter values and may be used to assess whether values below or above a margin remain plausible. A rule based on an interval is not automatically equivalent to a threshold on \(q\); the interval’s level, sidedness, and construction matter. The decision should be stated in terms of the prespecified rule actually being used.

Likewise, “use 0.95” is not a substitute for specifying clinical policy. A fixed threshold can be useful for consistency across decisions, transparent governance, or a prespecified trial rule. Its value may reflect a convention or policy choice rather than a derivation from patient-level losses. In contrast, the two-loss calculation makes the trade-off explicit. These approaches need not conflict: a policy may adopt a fixed cutoff after considering consequences, but its rationale and limitations should be clear.

The cutoff also does not replace clinical judgment about the endpoint or margin. If the margin is too small, a high \(q\) could support an action for an effect too modest to matter. If the margin is too demanding, a low \(q\) could obscure a worthwhile benefit. Sensitivity analyses over plausible margins and loss ratios can show whether a decision is stable or rests on a narrow choice. They should inform interpretation rather than be used to choose whichever assumptions produce a preferred action.