Asymmetric Costs of False-Go and False-Stop Decisions

Why Unequal Decision Errors Change the Go Threshold

A go/no-go decision can be wrong in two different ways. A false-go advances a program that is ineffective; a false-stop stops a program that is effective. Those errors need not have equal consequences. Advancing an ineffective program may consume substantial development resources, while stopping an effective one may forfeit a valuable treatment opportunity. Alternatively, if the unmet medical need is large, the cost of a false-stop may dominate. The appropriate probability threshold therefore depends not just on the chance the program is effective, but also on the relative consequences of the two mistakes.

This tutorial considers a single decision between advancing and stopping, with two possible states: the program is effective or ineffective. Let $q$ be the current probability, given the available evidence, that the program is effective. The decision rule uses that probability and the incremental losses associated with the two errors. It does not assume that one error is inherently more serious than the other.

Definition: A false-go loss, $L_{FG}$, is the incremental loss from advancing rather than stopping when the program is ineffective. A false-stop loss, $L_{FS}$, is the incremental loss from stopping rather than advancing when the program is effective. Both are measured on the same loss scale and are positive. They may include relevant financial, clinical, and opportunity consequences, provided those consequences are represented consistently and are not counted twice.

Assumptions and the Decision Rule

The simple threshold derived here applies under specific conditions. There are two actions, advance and stop, and two states, effective and ineffective. In the effective state, advancing is preferred; in the ineffective state, stopping is preferred. The two losses represent the incremental penalty for choosing the less-preferred action in each state. The probability $q$ is the decision-maker’s probability of effectiveness based on the evidence and is between 0 and 1, inclusive. The losses are treated as known for the calculation, even though in practice they may be uncertain or elicited from experts.

Set the loss of the preferred action in each state to zero as a reference. Then advancing incurs expected loss only when the program is ineffective, whereas stopping incurs expected loss only when it is effective. The expected losses are:

\mathbb{E}[L\mid \text{advance}]=(1-q)L_{FG}, \qquad \mathbb{E}[L\mid \text{stop}]=qL_{FS}.

Choose the action with the smaller expected loss. Advancing is preferred when its expected loss is no greater than the expected loss of stopping:

(1-q)L_{FG}\le qL_{FS}.

Because both losses are positive, rearranging gives:

L_{FG}\le q(L_{FG}+L_{FS}) \quad\Longleftrightarrow\quad q\ge \frac{L_{FG}}{L_{FG}+L_{FS}}.

Decision threshold: With positive false-go and false-stop losses, the go threshold is $$ q^*=\frac{L_{FG}}{L_{FG}+L_{FS}}. $$ Advance if $q>q^*$, stop if $q<q^*$, and regard the actions as tied on expected loss if $q=q^*$. Equivalently, advance when $q\ge q^*$ if ties are assigned to advance.

The threshold is below 0.5 when the false-stop loss is larger than the false-go loss, and above 0.5 when the false-go loss is larger. A useful odds expression is that, for $0\le q<1$, advancing has lower expected loss when

\frac{q}{1-q}>\frac{L_{FG}}{L_{FS}}.

At equality, expected losses are equal. The odds expression is not defined at $q=1$, so the probability-scale inequality is the general rule: at $q=1$, the expected loss of advancing is zero and the expected loss of stopping is $L_{FS}>0$; advance is preferred.

Worked Decisions With Unequal Losses

Worked Example: A Larger False-Stop Loss Lowers the Threshold

Suppose a hypothetical program has an estimated $q=0.40$. The decision team assigns a false-go loss of $L_{FG}=\$30$ million and a false-stop loss of $L_{FS}=\$90$ million. Assume these are incremental losses on the same scale and that the two-state, two-action model is appropriate.

Calculate the threshold.

q^*=\frac{\$30\text{ million}}{\$30\text{ million}+\$90\text{ million}} =\frac{30}{120}=0.25.

Since $q=0.40>0.25$, the rule favors advance. Checking directly, the expected loss of advancing is

(1-0.40)(\$30\text{ million})=0.60(\$30\text{ million})=\$18\text{ million},

and the expected loss of stopping is

0.40(\$90\text{ million})=\$36\text{ million}.

Advance has the lower expected loss by $\$18$ million on these assumptions. This does not mean the program is more likely than not to be effective; its assigned probability is 0.40. The action is favored because the modeled cost of stopping an effective program is relatively large.

Worked Example: A Larger False-Go Loss Raises the Threshold

Consider a different hypothetical program with $q=0.60$, $L_{FG}=\$80$ million, and $L_{FS}=\$20$ million. Assume the losses are incremental, comparable, and positive.

The threshold is

q^*=\frac{\$80\text{ million}}{\$80\text{ million}+\$20\text{ million}} =\frac{80}{100}=0.80.

Because $0.60<0.80$, the rule favors stopping, even though the probability of effectiveness exceeds 0.5. The expected losses confirm the result:

\mathbb{E}[L\mid \text{advance}]=(1-0.60)(\$80\text{ million})=\$32\text{ million},

\mathbb{E}[L\mid \text{stop}]=0.60(\$20\text{ million})=\$12\text{ million}.

Stopping has $\$20$ million less expected loss under this model. A high probability of effectiveness does not by itself determine the action: the relative consequences matter too.

Worked Example: How a Change in Losses Moves the Threshold

Suppose a hypothetical program has $q=0.70$ and a false-go loss of $\$45$ million. First, let the false-stop loss be $\$15$ million.

The threshold is $45/(45+15)=0.75$. Since $0.70<0.75$, stop is preferred. Directly, advancing has expected loss $0.30(\$45\text{ million})=\$13.5$ million, while stopping has expected loss $0.70(\$15\text{ million})=\$10.5$ million.

Now suppose the assessed false-stop loss rises to $\$30$ million, with the other inputs unchanged. The revised threshold is

q^*=\frac{45}{45+30}=0.60.

Now $0.70>0.60$, so advance is preferred. Its expected loss remains $\$13.5$ million, while stopping has expected loss $0.70(\$30\text{ million})=\$21$ million. The decision changes because the false-stop consequence increased, not because the probability of effectiveness changed.

Why the Threshold Has This Form

The derivation compares expected losses rather than comparing $q$ with an arbitrary default such as 0.5. The chance of incurring a false-go loss is $1-q$, so that error’s expected contribution decreases as evidence raises $q$. The chance of incurring a false-stop loss is $q$, so that error’s expected contribution increases as $q$ rises. At the threshold, the contributions balance:

(1-q^*)L_{FG}=q^*L_{FS}.

Solving this equality yields $q^*=L_{FG}/(L_{FG}+L_{FS})$. This also shows how the losses affect the rule. Increasing $L_{FG}$ while holding $L_{FS}$ fixed raises $q^*$, requiring stronger evidence before advancing. Increasing $L_{FS}$ while holding $L_{FG}$ fixed lowers $q^*$, making advance acceptable at a lower probability of effectiveness.

If the two losses are equal and positive, the threshold is 0.5. That familiar special case is not a neutral or universally correct rule; it follows from treating the two error consequences as equal. When the losses differ, using 0.5 without justification can favor the wrong action under the stated loss model.

Checking Inputs, Alternatives, and Limitations

The arithmetic is usually straightforward; specifying the inputs is harder. Decision-makers should make clear what “effective” means for the decision, how $q$ was obtained, and what outcomes are included in each loss. The definition of effectiveness should match the relevant development decision and endpoint. The probability should be conditional on the evidence currently available, not confused with a probability for a different stage or population. Earlier material on probability-of-success calibration is relevant when assessing whether such probabilities are reliable.

The losses also need careful elicitation. A false-go loss might include avoidable development expenditure and the opportunity cost of tying up resources. A false-stop loss might include the value forgone if an effective treatment is not developed. If the same consequence is included in multiple components, it may be counted twice. If costs and benefits are expressed in different units or valued under different perspectives, the resulting threshold may be difficult to interpret. Report the perspective, time horizon, and assumptions behind the loss estimates.

A practical diagnostic is to calculate expected losses directly after calculating the threshold. The chosen action should have the smaller expected loss; disagreement usually signals an arithmetic error, a reversed loss label, or an inconsistent definition of $q$. Also inspect the threshold and the decision under plausible alternative loss inputs. A decision that changes under modest, defensible changes in elicited losses is sensitive to those assumptions and should not be presented as a robust conclusion.

This threshold is one approach among several. With equal error losses it reduces to the 0.5 rule. A fixed probability cutoff can be used operationally, but its rationale should be connected to consequences rather than treated as automatically appropriate. If the decision can be represented by a more detailed payoff or loss table, compare the expected loss of each action using all state-specific entries; the simple formula applies only when the two error penalties adequately describe the relevant differences. In particular, if advancing has a meaningful cost even when the program is effective, or stopping has a meaningful cost even when it is ineffective, those consequences must be incorporated into the state-action losses before using this simplified threshold.

The calculation also treats $q$ and the losses as fixed inputs and assumes the two states are exhaustive for the decision. It does not resolve uncertainty about those inputs, represent multiple future stages, or determine whether collecting additional information is worthwhile. Those questions require a broader decision model. Here the purpose is narrower: given a probability and a defensible pair of incremental error losses, identify which action minimizes expected loss.

Common Pitfall

Wrong reasoning: “The program has a 60% chance of effectiveness, so it should advance because 60% is greater than 50%.” This silently assumes equal false-go and false-stop losses. If false-go losses are much larger, the threshold can be well above 0.5, as in the worked example with a threshold of 0.80.

Correction: State the loss assumptions, compute $q^*=L_{FG}/(L_{FG}+L_{FS})$, and compare the evidence-based $q$ with that threshold. Then check the result by calculating both expected losses. The threshold is a consequence of the specified decision model, not a universal clinical or regulatory cutoff.

Key takeaway: Unequal error consequences imply an unequal go threshold. A larger false-go loss raises the probability required to advance; a larger false-stop loss lowers it. The rule is meaningful only when $q$, the two incremental losses, and the decision context are defined consistently.

Check Your Understanding

Work each problem using the expected-loss rule. The solutions follow the questions.

A hypothetical program has $q=0.30$, $L_{FG}=\$20$ million, and $L_{FS}=\$50$ million. Find the threshold and preferred action.
For $q=0.80$, $L_{FG}=\$60$ million, and $L_{FS}=\$15$ million, calculate both expected losses. Which action is preferred?
With positive losses, what happens to the threshold when $L_{FS}$ increases while $L_{FG}$ stays fixed? Explain using the formula.
At $q=1$, explain why advance is preferred and why the odds form cannot be used directly at this endpoint.

Solutions. (1) $q^*=20/(20+50)=0.2857$. Since $0.30>0.2857$, advance is preferred. The expected losses are $0.70(\$20\text{ million})=\$14$ million for advance and $0.30(\$50\text{ million})=\$15$ million for stop, confirming the choice. (2) Advance has expected loss $(1-0.80)(\$60\text{ million})=\$12$ million; stop has expected loss $0.80(\$15\text{ million})=\$12$ million. The actions tie. The threshold is $60/(60+15)=0.80$. (3) The threshold is $L_{FG}/(L_{FG}+L_{FS})$; raising the positive denominator by increasing $L_{FS}$ while holding the numerator fixed lowers the threshold. (4) At $q=1$, advance has expected loss $(1-1)L_{FG}=0$, while stop has expected loss $1\cdot L_{FS}>0$, so advance is preferred. The odds expression contains $q/(1-q)$, whose denominator is zero at $q=1$; use the expected-loss inequality or the probability-scale threshold at that endpoint.