Deriving Posterior Decision Thresholds From Expected Loss

From Posterior Evidence to a Decision

A posterior probability describes uncertainty about a treatment effect, but a decision also depends on what follows from acting on that uncertainty. Choosing treatment when it does not provide a clinically meaningful benefit may waste resources or expose patients to avoidable harm. Choosing control when treatment does provide that benefit may forgo an opportunity to improve outcomes. Expected loss makes these consequences explicit and gives a direct way to derive a decision threshold.

The decision considered here has two actions: choose treatment or choose control. The relevant state is whether treatment meets a prespecified definition of clinically meaningful benefit. The threshold derived below assumes that the two kinds of incorrect decision have the same loss. That restriction matters: the resulting threshold is not a universal property of posterior probability, but a consequence of the particular decision table.

Definition: Let \(B\) denote the event that treatment meets the prespecified clinical-benefit criterion, and let \(B^c\) denote its complement. Write \(q=P(B\mid D)\), where \(D\) is the trial evidence available at the decision time. A posterior expected loss for an action is the posterior average of its loss over the possible states. The action with the smaller posterior expected loss is preferred under the stated loss table.

For example, \(B\) might mean that the treatment’s response-rate improvement exceeds a clinically meaningful margin, or that a mean symptom reduction exceeds a prespecified margin. The definition must be made before applying the rule: “some benefit” and “benefit large enough to matter” are different events and can have different posterior probabilities.

A Two-Action Loss Table

Suppose the decision maker assigns zero loss to a correct choice and the same loss \(C>0\) to either kind of incorrect choice. The loss is measured in a common unit, such as utility points or a monetary amount. The table describes the loss conditional on the state, not the probability that the state occurs.

Action	Benefit criterion met, \(B\)	Benefit criterion not met, \(B^c\)
Choose treatment	0	\(C\)
Choose control	\(C\)	0

The posterior expected loss of choosing treatment is \(C\) multiplied by the posterior probability that the benefit criterion is not met. The posterior expected loss of choosing control is \(C\) multiplied by the probability that it is met. Thus, the two actions are compared using the same posterior \(q\), but each is penalized in the state where that action is incorrect.

\begin{aligned} \mathbb{E}[L(\text{treatment})\mid D] &=0\cdot P(B\mid D)+C\cdot P(B^c\mid D)\\ &=C(1-q),\\[4pt] \mathbb{E}[L(\text{control})\mid D] &=C\cdot P(B\mid D)+0\cdot P(B^c\mid D)\\ &=Cq. \end{aligned}

Choose treatment when its expected loss is smaller:

C(1-q)<Cq \quad\Longleftrightarrow\quad 1-q<q \quad\Longleftrightarrow\quad q>\frac{1}{2},

where division by \(C\) is valid because \(C>0\). At \(q=1/2\), the expected losses are equal, so this table alone does not prefer either action. If \(q<1/2\), choosing control has the smaller expected loss. This is the full threshold derivation for the equal-error-loss table.

Decision rule: Under the stated two-action table, choose treatment if \(P(B\mid D)>0.5\), choose control if \(P(B\mid D)<0.5\), and regard the actions as tied at \(P(B\mid D)=0.5\). The rule follows from the specified losses; it is not a general rule that treatment is appropriate whenever any posterior probability exceeds one-half.

Worked Decisions

Worked Example: Comparing Expected Losses From Posterior Draws

Suppose a hypothetical binary-endpoint trial defines \(B\) as an improvement in response rate of at least 5 percentage points. Posterior computation gives \(P(B\mid D)=0.72\). For this example, the loss for either incorrect choice is 10 utility units, and a correct choice has loss 0. Which action has the smaller posterior expected loss?

The assumptions match the table: the two states are whether the 5-percentage-point criterion is met or not, there are only two available actions, and both incorrect choices have the same loss. The treatment expected loss is

\mathbb{E}[L(\text{treatment})\mid D] =10(1-0.72)=10(0.28)=2.8\text{ utility units}.

The control expected loss is

\mathbb{E}[L(\text{control})\mid D] =10(0.72)=7.2\text{ utility units}.

Because \(2.8<7.2\), the table favors choosing treatment. The interpretation is conditional on the specified model, data, benefit criterion, actions, and losses. It does not say that treatment is certain to meet the criterion; the posterior probability of meeting it is 0.72.

Worked Example: A Posterior Probability Below the Threshold

In a second hypothetical trial, let \(B\) mean that the treatment improves a continuous symptom score by at least 3 points on the study’s scale. Suppose the posterior probability is \(q=0.48\), and the same loss of 10 units applies to either incorrect action. Check the assumptions and select the action.

Assume that the 3-point margin was specified as clinically meaningful, the posterior probability is calculated for that exact event, and the two actions and equal losses in the table represent the decision. Then

\begin{aligned} \mathbb{E}[L(\text{treatment})\mid D] &=10(1-0.48)=10(0.52)=5.2,\\ \mathbb{E}[L(\text{control})\mid D] &=10(0.48)=4.8. \end{aligned}

Choosing control has the smaller posterior expected loss, \(4.8\) rather than \(5.2\) units. The posterior probability is close to the threshold, but the rule still favors control under this table. The small difference between the expected losses is a reason to examine how stable the calculation and the decision are, not a reason to describe the evidence as decisive.

Worked Example: Correct Choices Have a Common Baseline Loss

Consider a hypothetical trial with \(P(B\mid D)=0.70\), where \(B\) is that treatment meets a prespecified meaningful benefit margin. Suppose either correct choice has loss 2 units and either incorrect choice has loss 14 units. The assumptions are that these losses are comparable on one scale and that the two states and actions in the table are sufficient for the decision.

The expected loss of treatment is the loss 2 when \(B\) holds plus the loss 14 when \(B^c\) holds, each weighted by its posterior probability:

\mathbb{E}[L(\text{treatment})\mid D] =2(0.70)+14(0.30) =1.4+4.2 =5.6\text{ units}.

For control, the incorrect-choice state is \(B\):

\mathbb{E}[L(\text{control})\mid D] =14(0.70)+2(0.30) =9.8+0.6 =10.4\text{ units}.

Treatment is preferred because \(5.6<10.4\). The nonzero loss for correct choices does not change the threshold here: it is the same for both actions and states, while the additional loss for being incorrect is 12 units in either direction. More generally, adding the same constant to every cell adds that constant to each action’s expected loss and therefore does not change which action has the smaller value.

Using Posterior Draws and Checking the Decision

When \(q\) is estimated from posterior draws, define \(B\) for each draw and estimate \(q\) by the fraction of draws for which the criterion is met. For instance, if 840 of 1,000 retained draws satisfy \(B\), then \(\hat q=840/1000=0.84\). Under the equal-error-loss table with \(C=20\) units, the estimated expected losses are \(20(1-0.84)=3.2\) units for treatment and \(20(0.84)=16.8\) units for control. This calculation assumes the draws adequately represent the posterior; poor mixing, too few effective draws, or coding the event incorrectly can make the estimated decision unreliable.

A useful check near the threshold is Monte Carlo precision. For \(M\) approximately independent draws, the rough Monte Carlo standard error of the estimated probability is \(\sqrt{\hat q(1-\hat q)/M}\). MCMC draws are typically autocorrelated, so \(M\) should not be treated as the effective number of independent draws without checking sampling diagnostics. If uncertainty in \(\hat q\) could plausibly move it across 0.5, improve or validate posterior computation before presenting a sharply stated action.

Other checks concern the decision model rather than the sampler. Confirm that the benefit event has the right direction and margin, that its complement really represents the alternative state, and that both losses are expressed on the same scale. Consider whether relevant consequences are omitted: adverse effects, treatment burden, cost, or additional actions may make a two-state, two-action table an inadequate representation. The introductory material on posterior go, no-go, and continue decisions frames choices through posterior evidence; expected-loss comparison supplies a justification for a threshold only when the decision table is also specified.

Interpretation, Alternatives, and Limits

A posterior-probability cutoff can be applied directly, but without a loss table its rationale may be unclear. Expected-loss comparison states what is being traded off: the chance of each state multiplied by the loss attached to the chosen action in that state. Under equal penalties for the two incorrect choices, this comparison reduces to the 0.5 cutoff. A different table can yield a different cutoff; the derivation above does not establish that equal penalties are appropriate in every clinical setting.

This is a Bayesian decision under the chosen posterior model and loss specification. It is not a frequentist significance test, and the expected loss is not the probability that an action will be wrong. For example, when treatment is selected under this table, the probability of the unfavorable state is \(1-q\); the expected loss is that probability multiplied by \(C\). Neither quantity proves that treatment causes benefit: causal interpretation depends on the trial design, estimand, and analysis assumptions.

The rule is limited to the actions and states included in the table. If continuing the trial is available, if benefit has multiple clinically relevant levels, or if losses depend on more detailed outcomes, this two-action comparison may leave out important choices or consequences. The numerical threshold is also conditional on the assigned losses. It should therefore be reported together with the definition of \(B\), the posterior probability, the loss table, and the action selected—not as an unexplained universal cutoff.

Common Mistakes

Using the wrong event. A probability of any improvement is not interchangeable with the probability of meeting a clinically meaningful margin. Define \(B\) before calculating \(q\).
Confusing probability and loss. The posterior probability of an incorrect choice is not itself the expected loss unless the loss for that error is one unit.
Treating the threshold as universal. The value 0.5 follows from equal losses for the two kinds of error. It is not implied by Bayesian analysis alone.
Ignoring a tie or numerical uncertainty. At \(q=0.5\) the actions have equal expected loss under this table. Near that value, check posterior computation and avoid overstating the decision.
Assuming the table is complete. If important harms or available actions are omitted, the calculated minimum applies only to the simplified decision being analyzed.

Key takeaway: Define the benefit event, specify the losses for each action in each state, and calculate posterior expected loss by averaging across states. With equal loss for either incorrect choice and zero loss for correct choices, choose treatment exactly when the posterior probability of the defined benefit event exceeds one-half.

Check Your Understanding

Answer each question using the equal-error-loss table unless the question states otherwise. Solutions follow the questions.

A hypothetical trial gives \(P(B\mid D)=0.65\). Each incorrect action has loss 8 units and each correct action has loss 0. Calculate both expected losses and select an action.
Under the equal-error-loss table, what happens when \(P(B\mid D)=0.50\)? Why does the table not select one action?
If 720 of 900 posterior draws satisfy \(B\), estimate \(q\). With an incorrect-choice loss of 5 units and zero loss for correct choices, calculate both expected losses.
Explain why defining \(B\) as “any improvement” rather than “improvement of at least the prespecified clinical margin” can change the decision.

Solutions. (1) Treatment loss is \(8(1-0.65)=2.8\) units; control loss is \(8(0.65)=5.2\) units, so choose treatment. (2) Both expected losses are \(0.5C\), so they are equal; a tie-breaking preference would require information outside this table. (3) \(\hat q=720/900=0.80\). Treatment loss is \(5(0.20)=1.0\) unit and control loss is \(5(0.80)=4.0\) units, so treatment is preferred. (4) The events include different treatment effects, so they can have different posterior probabilities. The threshold applies to the probability of the event actually specified in the decision table, not to an interchangeable notion of benefit.