Why Average Performance May Not Be Enough
Two candidate doses can have similar average performance but very different consequences in an unfavourable scenario. One dose may cause a modest shortfall from the desired effect for many patients; another may usually perform well but carry a small chance of severe toxicity. When underdosing and toxicity do not have equal consequences, a dose rule based only on an unweighted average can conceal an important clinical trade-off. And even after the harms have been weighted, an average alone does not say whether decision makers wish to avoid unusually poor outcomes.
This tutorial compares three elements of a decision: the clinical weights assigned to different harms, the expected loss across uncertainty, and an additional penalty for losses in the adverse tail. The tail penalty represents a preference for avoiding bad outcomes; it does not replace the clinical judgment needed to define which harms matter and how they should be measured.
The severity measures and their weights need a meaningful interpretation. For example, \(U\) and \(T\) might be points on a prespecified clinical harm scale, and \(c_T>c_U\) would make a unit of toxicity severity count more than a unit of underdosing severity. The numerical scale must be chosen so that this comparison is defensible; the formula does not discover the weights from the dose-response model. Other clinically relevant outcomes can be added if their definitions and weights are justified.
The states and their probabilities may summarize uncertainty relevant to the decision, such as uncertainty in model parameters and predicted outcomes. They must represent a coherent probability distribution for the decision being considered. The calculations below treat that distribution and the loss scale as specified. They do not account automatically for model misspecification, unrepresented outcomes, or uncertainty about the clinical weights.
Expected Loss and Downside-Risk Criteria
A risk-neutral rule selects the candidate dose with the smallest expected loss:
This criterion already accounts for asymmetric harm if the loss function uses unequal weights. “Risk-neutral” here means that, for a fixed loss scale, the choice depends on the average loss rather than an additional preference for or against variability. It does not mean that the clinical consequences are treated as symmetric.
A risk-sensitive rule adds a penalty for a particular part of the loss distribution. One option is a downside penalty based on excess loss above a prespecified reference \(L_0\), such as \(\mathbb{E}[\max\{L(d)-L_0,0\}]\). Another option is a tail criterion: conditional value at risk (CVaR), which averages losses in the worst part of the distribution. For an upper-tail fraction \(1-q\), define the loss quantile and CVaR as follows.
With this convention, a larger \(q\) focuses attention on a narrower, more extreme upper tail. In a discrete distribution, the tail may cut through a probability mass at the quantile. The CVaR calculation must then include only the fraction of that mass needed to make the tail probability exactly \(1-q\); simply averaging every outcome at or above the quantile can give the wrong answer.
The expected loss and CVaR are in the same loss units, so \(\lambda\) is dimensionless. At \(\lambda=0\), the score reduces to expected loss. At larger \(\lambda\), the decision maker gives more weight to severe tail losses. This is a transparent decision rule only if \(q\), \(\lambda\), and the loss scale have been justified and reported.
Worked Dose Comparisons
Worked Example: A Tail Penalty Changes the Preferred Dose
Hypothetical data. Suppose three doses are evaluated in three possible states with probabilities \(0.50\), \(0.30\), and \(0.20\). Underdosing severity is \(U\), toxicity severity is \(T\), and loss is \(L=2U+5T\) points. Thus a unit of toxicity severity receives 2.5 times the weight of a unit of underdosing severity. The severity pairs \((U,T)\) across the three states are: dose A, \((2,0),(2,0),(2,0)\); dose B, \((0,0),(0,0),(0,2.4)\); dose C, \((0,0),(1,1),(2,3)\).
Calculate the losses. For dose A, \(L=2(2)+5(0)=4\) in every state. For dose B, the losses are \(0,0,2(0)+5(2.4)=12\). For dose C, they are \(0,2(1)+5(1)=7,2(2)+5(3)=19\). The probability-weighted expected losses are:
The expected-loss rule selects dose B, with 2.4 points, rather than A or C. Now consider \(\operatorname{CVaR}_{0.80}\), the average loss in the worst 20% of each distribution. For A, every loss is 4, so the CVaR is 4. For B, the loss of 12 occurs with probability 0.20 and is exactly the worst 20%, so the CVaR is 12. For C, the loss of 19 occurs with probability 0.20, so the CVaR is 19.
With \(\lambda=0.30\), the risk-adjusted scores are \(J(A)=4+0.30(4)=5.2\), \(J(B)=2.4+0.30(12)=6.0\), and \(J(C)=5.9+0.30(19)=11.6\). This rule selects dose A. It accepts a larger expected loss than B in exchange for a smaller loss in the worst 20% of states. The decision is not a statistical finding that A is universally safer; it follows from the stated severity weights, scenario probabilities, tail definition, and risk penalty.
Worked Example: CVaR When the Tail Cuts Through a Discrete Outcome
Hypothetical data. Dose D has losses of 0, 6, and 12 points with probabilities 0.60, 0.30, and 0.10. Dose E has a constant loss of 5 points. First, the expected losses are \(\mathbb{E}[L(D)]=0.60(0)+0.30(6)+0.10(12)=3.0\), and \(\mathbb{E}[L(E)]=5\). A risk-neutral rule therefore selects D.
Calculate \(\operatorname{CVaR}_{0.80}\) for D. Its 0.80 quantile is 6, because the cumulative probability at loss 0 is 0.60 and at loss 6 is \(0.60+0.30=0.90\). The worst 20% consists of all 10% of probability at loss 12, plus 10% of probability at loss 6. Therefore:
For E, all outcomes are 5, so \(\operatorname{CVaR}_{0.80}(L(E))=5\). With \(\lambda=0.50\), \(J(D)=3+0.50(9)=7.5\) and \(J(E)=5+0.50(5)=7.5\), a tie. With \(\lambda=0.60\), \(J(D)=3+0.60(9)=8.4\) and \(J(E)=5+0.60(5)=8.0\), so E is selected. The result depends on the stated tail fraction and penalty, not just the two means.
Worked Example: How the Toxicity Weight Affects Expected Loss
Hypothetical data. Across a specified distribution of states, suppose dose F has expected underdosing severity 2.0 and expected toxicity severity 0.20. Dose G has expected underdosing severity 0.50 and expected toxicity severity 0.80. Use \(L=2U+c_TT\), holding the underdosing weight at 2 and varying the toxicity weight \(c_T\).
At \(c_T=4\), the expected losses are \(\mathbb{E}[L(F)]=2(2.0)+4(0.20)=4.8\) and \(\mathbb{E}[L(G)]=2(0.50)+4(0.80)=4.2\). The expected-loss rule selects G. At \(c_T=6\), the results are \(2(2.0)+6(0.20)=5.2\) for F and \(2(0.50)+6(0.80)=5.8\) for G; the rule selects F.
The crossover follows by setting the expected losses equal: \(4+0.20c_T=1+0.80c_T\), so \(3=0.60c_T\) and \(c_T=5\). At that weight, both expected losses are 5.0 points. This calculation shows why harm weights are substantive decision inputs: changing them can change the preferred dose even before any additional tail-risk penalty is applied.
Justifying the Tail Criterion
CVaR is useful because it makes “bad outcomes matter more” operational without discarding the rest of the distribution. At \(\operatorname{CVaR}_{0.80}\), for example, the rule summarizes the average loss in the worst 20%. A short justification follows from the quantile definition: if losses are ordered from low to high, integrating \(\operatorname{VaR}_u\) from \(q\) to 1 accumulates exactly the losses occupying the upper \(1-q\) probability of the distribution. Dividing by \(1-q\) converts that total tail contribution into an average. If the tail starts partway through an atom, the quantile integral includes only the needed fraction of that atom, as in the preceding example.
The added penalty changes the decision only when doses trade off average and tail loss. If dose A has no greater expected loss and no greater CVaR than dose B, then \(J(A)\leq J(B)\) for every \(\lambda\geq0\); B is not preferred by this score. If A has lower expected loss but higher CVaR, the selection can depend on \(\lambda\). The equality point, when one exists, can be found by solving:
This is a comparison under the specified score, not a universal ranking of doses. A reference-based downside penalty and CVaR answer different questions: the former penalizes loss beyond a chosen threshold, whereas CVaR averages a chosen upper tail. A variance penalty is another possible summary, but it can be sensitive to unusually high as well as low outcomes and does not isolate severe harms in the upper tail of loss. The choice of criterion should follow the decision concern, not convenience alone.
Assumptions, Diagnostics, and Limitations
Before using a risk-sensitive score, check that its ingredients are suitable for the decision:
- Loss definition: severity measures and weights must reflect the clinical consequences being compared. If outcomes overlap, adding their penalties may count the same harm twice.
- Scenario distribution: probabilities must be nonnegative, sum to one, and describe the uncertainty relevant to the decision. Check that important adverse states have not been omitted.
- Tail estimation: a very narrow tail can be estimated poorly when only a small number of simulated states are available. Inspect tail counts and quantile stability rather than trusting a single estimated CVaR.
- Decision sensitivity: report how the selected dose changes over plausible values of \(\lambda\), \(q\), and clinical weights. A recommendation that changes with a small input change should be presented as conditional on those inputs.
- Scale and interpretation: keep the loss units consistent across doses. State the dose set, the chosen tail fraction, and the risk penalty so another analyst can reproduce the comparison.
Risk adjustment does not repair biased predictions or incorrect probabilities. If the model understates toxicity in an adverse state, a precise CVaR calculation will still understate the tail loss. Nor does a CVaR score alone specify an acceptable probability of toxicity; a chance constraint directly limits the probability of an event, while CVaR summarizes the severity of losses in a tail. The earlier material on chance-constrained dose selection addresses that different form of decision rule. Expected loss is simpler and appropriate when averages are the intended criterion; minimax regret, covered in the earlier material on minimax regret for dose choice, instead compares a decision with the best dose in each state. These approaches answer distinct questions and need not select the same dose.
Common Pitfalls
- “The expected-loss winner is safest.” A lower average does not imply a smaller worst-tail loss. State whether the choice is based on an average, a tail measure, or both.
- “Toxicity is already weighted, so risk adjustment is redundant.” Unequal weights encode the relative severity of underdosing and toxicity. A tail penalty separately encodes a preference about the distribution of the resulting loss.
- “CVaR is the average of every outcome at or above the quantile.” In a discrete distribution, that can include too much probability at the quantile. Average exactly the worst \(1-q\) probability, using only a fraction of a tied or boundary mass if needed.
- “A larger \(\lambda\) is automatically more clinically appropriate.” It increases the influence of CVaR by construction; its value requires a defensible decision rationale and should be shown transparently.
Check Your Understanding
Use the definitions and decision rules above. Solutions follow the questions.
- A dose has losses 2 and 10 points with probabilities 0.90 and 0.10. What is its expected loss and its \(\operatorname{CVaR}_{0.90}\)?
- Two doses have equal expected loss, but dose X has lower \(\operatorname{CVaR}_{0.80}\) than dose Y. Which dose has the smaller score \(J\) when \(\lambda>0\), and why?
- For a hypothetical dose, underdosing severity is 1.5 and toxicity severity is 0.4 in a state. If \(L=2U+5T\), calculate its loss in that state.
- Why is it incorrect to average all outcomes at or above the quantile when a discrete outcome has probability mass at the CVaR boundary?
Solutions. (1) The expected loss is \(0.90(2)+0.10(10)=1.8+1.0=2.8\) points. The worst 10% is exactly the outcome with loss 10, so \(\operatorname{CVaR}_{0.90}=10\) points. (2) The expected-loss terms are equal, and X has the smaller CVaR; multiplying by \(\lambda>0\) preserves that ordering, so \(J(X)<J(Y)\). (3) \(L=2(1.5)+5(0.4)=3.0+2.0=5.0\) points. (4) The tail contains exactly the worst \(1-q\) fraction of probability. Including an entire boundary mass when only part is needed overweights that outcome and calculates a different tail average.