How Asymmetric False-Go and False-Stop Costs Set Thresholds

Why the Two Errors Need Not Cost the Same

A posterior probability can summarize how strongly trial data support a clinically meaningful treatment effect, but it does not by itself determine what action to take. Acting on a treatment that is ineffective may expose patients to harm, expense, or delay in using better care. Stopping development of an effective treatment may also harm patients by delaying access to a useful therapy. Those consequences need not be equally serious. A decision rule that accounts for them can use different costs for the two errors, rather than treating false-go and false-stop decisions as interchangeable.

The aim here is to derive the posterior threshold implied by a simple loss model. The threshold is not a property of the posterior alone: it depends on how the decision-maker values the possible consequences. For the rule to be meaningful, the clinical criterion defining an effective treatment and the losses attached to each error must be specified clearly.

The event \(E\) must refer to a clinically interpretable criterion, such as exceeding a clinically meaningful superiority margin. It is not enough to say that a treatment has some probability of benefit without specifying what counts as benefit. The earlier material on clinically meaningful margins gives examples of defining such criteria.

The losses can represent monetary costs, health consequences, or a common utility scale. They need not literally be dollars. What matters for the threshold is their relative size, and both losses must be expressed on the same scale. The simple model also assumes that the decision is between just two actions and that the consequences of each action can be summarized by these two error losses.

Deriving the Loss-Based Threshold

Conditional on the observed data, choosing Go incurs a false-go loss only when \(E\) is false. That event has posterior probability \(1-\pi\). Choosing Stop incurs a false-stop loss only when \(E\) is true, with posterior probability \(\pi\). Therefore, the posterior expected losses are

The decision rule chooses the action with the smaller expected loss. Go is preferred when its expected loss is less than the expected loss of Stop:

Adding \(C_{\mathrm{FG}}\pi\) to both sides gives \(C_{\mathrm{FG}}<(C_{\mathrm{FG}}+C_{\mathrm{FS}})\pi\). Dividing by the positive sum of the two losses yields the posterior threshold:

At equality, the two actions have equal posterior expected loss, so this model alone does not prefer one. A prespecified tie-breaking rule or additional considerations would be needed. In odds form, the same comparison is \(\pi/(1-\pi)>C_{\mathrm{FG}}/C_{\mathrm{FS}}\): the posterior odds of effectiveness must exceed the ratio of false-go loss to false-stop loss.

The direction of these effects can also be checked directly. Writing \(A=C_{\mathrm{FG}}\) and \(B=C_{\mathrm{FS}}\), the threshold is \(t=A/(A+B)\). Its rate of change with the false-go loss is \(B/(A+B)^2>0\), while its rate of change with the false-stop loss is \(-A/(A+B)^2<0\). Thus the threshold rises with \(A\) and falls with \(B\), as the decision logic suggests.

Applying the Rule in Hypothetical Trials

Assumptions, Diagnostics, and Alternatives

The threshold is appropriate only to the decision model that produced it. Its basic assumptions are that the decision-maker has a posterior probability for a clearly defined event \(E\), must choose between Go and Stop, and can represent the relevant consequences by the stated losses. It also assumes that expected loss is the decision criterion: the preferred action minimizes the posterior average of its loss. This is a risk-neutral rule on the chosen loss scale.

The loss specification deserves the same scrutiny as the statistical model. In a clinical setting, consequences may differ across patient groups, time horizons, or treatment settings. A single false-go cost may hide important variation, and a single false-stop cost may omit costs of delay or future opportunities. If these consequences are not adequately captured, a threshold can be mathematically correct yet clinically unpersuasive.

Useful diagnostics include checking that the event defining effectiveness matches the trial’s clinical question, that both losses use a common scale, and that the assigned ratio reflects the decision context. Recalculate the threshold over a plausible range of cost ratios. If small changes in the ratio reverse the preferred action, report that sensitivity rather than presenting the result as robust. Also verify the arithmetic by comparing the two expected losses directly; this catches errors such as using \(\pi\) for the probability of a false-go.

A fixed posterior cutoff is a simpler alternative: it may be chosen by convention or protocol rather than derived from losses. The loss-based rule explains what relative losses would justify a particular cutoff, but it does not establish that the loss inputs are correct. A symmetric-error rule is a special case in which the two losses are equal and the threshold is \(0.5\). Neither a fixed cutoff nor equal losses should be assumed to represent clinical priorities without justification. A frequentist significance level is also a different quantity: the posterior threshold here is not a Type I error rate and does not by itself control a long-run false-positive probability.

The model has further limitations. If there are more than two actions, such as several doses or a choice among treatments, each action needs its own loss under each possible state, and one threshold may not describe the decision. If correct actions also have meaningful costs, those costs must be included in the loss table rather than silently set to zero. When losses are uncertain, the threshold is uncertain too; a single calculation using point-valued costs can conceal that uncertainty.