Why Withhold a Recommendation?
A dosing model can produce a numerical recommendation even when the evidence for that choice is weak. For example, two doses may have nearly equal expected utility, while uncertainty about the patient’s response makes it unclear which dose is preferable. Reporting one dose without qualification can conceal that uncertainty. An abstention rule makes the system’s response conditional: it recommends a dose when a prespecified uncertainty criterion is met, and otherwise withholds its own recommendation.
Abstention is a decision rule, not a claim that no dose is appropriate. Nor does it establish that a patient is unsafe to treat. It means that, under the model and rule, the system does not provide a dose recommendation in that case. Decisions about what happens after abstention require a separate clinical process.
The posterior can represent uncertainty about an individual’s dose-response relationship, about the best dose under that relationship, or about the utility of possible doses. The uncertainty measure should match the clinical decision. A narrow interval for a model parameter is not automatically evidence that a particular dose is appropriate; the parameter’s uncertainty must be translated into consequences for the decision.
Define What Counts as a Reliable Recommendation
Suppose \(U(d,\theta)\) is the utility of dose \(d\) when the relevant state is \(\theta\), with larger values indicating better outcomes under the specified utility scale. A common candidate recommendation is the dose with the greatest posterior expected utility:
where \(\mathcal D\) is the feasible dose set. This point decision does not, by itself, say how uncertain the decision is. One way to quantify decision-relevant uncertainty is posterior regret. For a state \(\theta\), regret from choosing \(d\) is the utility gap between the best feasible dose in that state and \(d\):
Regret is nonnegative. A clinical team can define a tolerance \(\delta\), measured in the same utility units, for what counts as an acceptable loss. The posterior probability that a candidate dose is within this tolerance is
The system may recommend \(\hat d(y)\) only if \(q_\delta(y)\) reaches a prespecified threshold \(\gamma\). This rule is interpretable: conditional on the model and current information, the probability that the recommendation’s regret is no greater than \(\delta\) must be at least \(\gamma\). It is not the probability that the model is true, and it does not guarantee the patient’s outcome.
An alternative is to define a clinical event directly, such as a dose causing a specified toxicity or failing to attain a target exposure, and require its posterior probability to be below a limit. That is a posterior risk rule, whereas a regret rule asks whether the selected dose is close enough in utility to the best dose for the uncertain state. These criteria answer different questions and should not be interchanged without justification.
Choosing the Threshold from the Consequences
A threshold should reflect the clinical costs of both errors: issuing a recommendation that is not acceptably good, and withholding a recommendation when one would have been acceptable. Consider a simplified rule where \(q\) is the posterior probability that a candidate recommendation is acceptable. Let \(L\) be the loss from issuing an unacceptable recommendation and \(c\) the loss assigned to abstaining. Assume these losses are fixed for the decision under consideration, and treat an acceptable recommendation as having zero loss for this comparison.
Under these assumptions, recommending has lower expected loss when \(L(1-q)\leq c\). Rearranging gives \(q\geq 1-c/L\). This derivation explains how the threshold depends on consequences: when the loss of an unacceptable recommendation is much larger than the cost of abstaining, the required confidence is high. If the losses vary substantially across patients or actions, a single fixed threshold may not represent the relevant decision; the expected losses should instead be compared for the particular case.
This is a decision-theoretic justification, not a universal formula for a clinical cutoff. The loss values, acceptable-regret definition, posterior model, and threshold all require clinical and operational justification before use.
Worked Example: A Posterior Interval for the Best Dose
Suppose, hypothetically, the posterior distribution of the dose that maximizes utility is approximately normal with mean 50 mg and standard deviation 8 mg. The system’s candidate dose is 50 mg. For this illustration, define an acceptable decision as being within 10 mg of the best dose, and require posterior probability \(\gamma=0.80\).
Under the stated normal approximation, the posterior probability of acceptable distance is
Here \(d^*\) denotes the dose that maximizes utility under the uncertain state, and \(\Phi\) is the standard normal cumulative distribution function. The calculated probability, about 0.789, is below 0.80. The rule therefore abstains, even though 50 mg is the posterior mean best dose. The calculation assumes the normal approximation is adequate and that being within 10 mg is a clinically justified proxy for acceptable regret. If utility changes sharply with dose, that proxy may be inappropriate; regret should then be calculated from the utility model directly.
Worked Decision-Utility Example
A posterior decision rule can be evaluated directly using the utilities over plausible states, rather than using a dose interval. The following hypothetical example makes the decision consequences explicit.
Worked Example: Regret Probability Across Posterior States
Suppose three feasible doses are 40, 60, and 80 mg. The posterior probabilities of three possible response states are 0.08, 0.87, and 0.05. Utilities are as follows.
| Dose | State 1 utility | State 2 utility | State 3 utility |
|---|---|---|---|
| 40 mg | 10 | 6 | 2 |
| 60 mg | 6 | 10 | 7 |
| 80 mg | 1 | 6 | 10 |
The posterior expected utility for 40 mg is \(0.08(10)+0.87(6)+0.05(2)=6.12\) utility units. For 60 mg it is \(0.08(6)+0.87(10)+0.05(7)=9.53\) utility units. For 80 mg it is \(0.08(1)+0.87(6)+0.05(10)=5.80\) utility units. Thus the candidate recommendation is 60 mg.
The best dose by state is 40 mg in State 1, 60 mg in State 2, and 80 mg in State 3. Regret from recommending 60 mg is therefore 4, 0, and 3 utility units, respectively. With acceptable regret defined as \(\delta=1\), only State 2 is acceptable, so \(q_\delta=0.87\). If the required threshold is \(\gamma=0.90\), the system abstains. If the threshold is 0.85, it recommends 60 mg. The decision changes because the prespecified confidence requirement changes, not because the posterior probabilities or utilities changed.
Evaluating Coverage and Safety
A useful evaluation reports both how often the rule recommends and how safe those recommendations are. If \(N\) cases are evaluated and \(N_R\) receive recommendations, empirical coverage is \(N_R/N\). Among recommended cases, the selective unsafe rate is the proportion judged unacceptable under a prespecified safety or decision criterion. Always state the denominator: an unsafe rate among issued recommendations is not the same as the proportion of all cases with an unsafe issued recommendation.
Abstention is not itself a safe outcome. To assess the safety effect of the rule, evaluation should distinguish recommendations that were issued from cases withheld, and should specify how acceptability is assessed. A retrospective reference assessment may compare the recommendation with relevant outcomes or a defined decision target; its uncertainty and limitations should be reported. Where possible, evaluate multiple thresholds, since stricter thresholds will generally lower coverage but do not necessarily improve safety if the model is poorly calibrated or the criterion is misaligned with clinical harm.
Worked Example: Coverage and Unsafe Recommendations
In a hypothetical evaluation of 100 cases, the system recommends a dose for 72 and abstains for 28. Retrospective assessment judges 4 of the 72 issued recommendations unacceptable. Among the 28 withheld cases, a separately specified retrospective assessment indicates that 5 of the candidate recommendations the system would otherwise have issued would have been unacceptable.
Coverage is \(72/100=0.72\), or 72%. The selective unsafe rate is \(4/72=0.0556\), or 5.56% among recommendations issued. The proportion of all evaluated cases receiving an unacceptable recommendation is \(4/100=0.04\), or 4%. If the rule had issued all 100 candidate recommendations, the hypothetical unacceptable count would have been \(4+5=9\), or 9%; thus, in this evaluation, abstention withheld five candidate recommendations judged unacceptable.
These figures do not show that the 28 abstentions were harmless or that abstention caused better patient outcomes. The comparison depends on how the counterfactual candidate doses were assessed, and the 23 other abstentions may represent lost opportunity or unnecessary withholding. Report the assessment method and, where relevant, clinical outcomes separately from this decision classification.
Diagnostics, Comparisons, and Limitations
Before deployment, check whether the posterior probabilities used by the rule are credible for the intended population and decision. For example, among cases assigned a probability near 0.8 of acceptable regret, roughly 80% should meet that criterion in a sufficiently large, representative evaluation, allowing for sampling uncertainty. Poor calibration can make a nominal confidence threshold misleading. Also examine coverage and selective safety across clinically relevant subgroups: an acceptable overall rate can conceal a group that is frequently given poor recommendations or frequently denied them.
Sensitivity analysis across plausible thresholds is essential. A sudden change in coverage from a small threshold adjustment, or a selective unsafe rate that remains high even as coverage falls, may indicate that the uncertainty measure is not separating reliable from unreliable recommendations. Investigate the posterior model, the utility or regret definition, and the evaluation reference. Posterior uncertainty is conditional on the model and data; it does not automatically capture model misspecification or all clinically relevant uncertainty.
| Approach | Strength | Important limitation |
|---|---|---|
| Posterior abstention rule | Connects uncertainty in the model to a defined decision criterion. | Depends on posterior validity, the chosen criterion, and a justified threshold. |
| Always issue the posterior expected-utility dose | Provides a dose for every case and uses the stated expected-utility objective. | Can hide substantial uncertainty about whether the selected dose is acceptable. |
| Use a fixed conservative dose rule | Can be simple to implement and explain. | Does not necessarily adapt to patient-specific evidence or quantify decision uncertainty. |
A common mistake is to interpret a high posterior probability of being within a dose interval as proof that the dose is safe. The correction is to define what the probability concerns: proximity to the best dose, acceptable regret, or a particular harm event. These are different endpoints. Another mistake is to report only the percentage of cases in which the system abstains. Coverage alone says nothing about the quality or safety of the recommendations that remain.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A rule requires a posterior probability of acceptable regret of at least 0.95. A patient’s value is 0.93. What does the rule do, and what does that result mean?
- If the probability that regret is at most \(\delta\) is \(q=0.92\), and the threshold is \(\gamma=0.90\), should the system recommend or abstain?
- In 80 evaluated cases, 60 receive recommendations and 6 of those are unacceptable. Calculate coverage and the selective unsafe rate.
- Why is a high probability that the selected dose is within 5 mg of the best dose not necessarily evidence of low toxicity risk?
Solutions. (1) The rule abstains because \(0.93<0.95\). This means its prespecified posterior-confidence requirement is unmet, not that the patient has no suitable dose. (2) Recommend, because \(0.92\geq0.90\), subject to the rule’s other stated conditions. (3) Coverage is \(60/80=0.75\), or 75%; the selective unsafe rate is \(6/60=0.10\), or 10% among issued recommendations. (4) Dose proximity concerns distance from the dose that maximizes the specified utility; toxicity risk is a separate event probability and must be assessed with its own definition and threshold.