Quantifying Uncertainty In Dose Recommendations

Why a Dose Recommendation Needs an Uncertainty Summary

A dose-finding procedure can return a single recommended dose, but that output alone does not show how strongly the data support the recommendation. Two studies could name the same dose even though one has a clear posterior preference and the other has several plausible alternatives. In a phase 1/2 setting, uncertainty matters because efficacy and toxicity rates are estimated from limited patient data, and those estimates inform the eventual benefit–risk decision.

This tutorial focuses on three complementary summaries: the posterior probability that a dose is selected by a specified decision rule, credible intervals for dose-specific outcome rates, and the posterior probability that each dose is optimal under a specified criterion. These quantities answer different questions. None is meaningful without stating the model, prior, outcomes, and decision rule that produced it.

Specify the Posterior and the Decision Before Summarizing

Suppose patients are assigned to a finite set of doses. For dose \(d\), let \(p_{E,d}\) be the probability of the prespecified efficacy outcome and \(p_{T,d}\) the probability of the prespecified toxicity outcome. A fitted joint dose model gives a posterior distribution for all dose-specific parameters given the observed outcomes. The model may represent dependence between efficacy and toxicity; this tutorial does not require a particular joint model. The essential requirement is that posterior draws represent uncertainty in the parameters used by the decision rule.

The probability summaries are conditional on that model, its assumptions and priors, and the data included in the analysis. For example, a simple illustrative model might use binomial outcomes at each dose and independent Beta priors for the two marginal rates. Such a model is convenient for hand calculations, but it assumes independent posterior distributions for those rates and does not represent within-patient dependence between efficacy and toxicity. In a real joint analysis, use the fitted model’s joint posterior draws rather than treating marginal models as though they captured that dependence.

A selection rule is an algorithm mapping a parameter draw to a dose recommendation or to no recommendation. It can include safety and efficacy constraints, tie-breaking rules, and operational restrictions. The rule must be fixed before calculating its posterior selection probabilities. Changing the rule changes the probability being summarized.

Optimality is a different concept. It requires a stated objective, such as a utility that trades off efficacy and toxicity, and a set of allowed actions. For example, a dose may be considered only if its toxicity probability is below a prespecified limit. The probability of optimality then describes uncertainty about which action maximizes that objective, not uncertainty about which action a particular algorithm will recommend.

Credible Intervals for Dose-Specific Outcome Rates

A posterior credible interval summarizes uncertainty in a model parameter. An equal-tail \(100(1-\alpha)\%\) credible interval for a rate \(p\) has lower and upper endpoints at posterior quantiles \(\alpha/2\) and \(1-\alpha/2\). This definition does not imply that the parameter has a frequentist probability of \(1-\alpha\) of lying in the particular interval after it has been calculated. Rather, under the stated model and prior, the posterior probability that \(p\) lies in this interval is \(1-\alpha\).

For a simple binomial example, if \(x\) of \(n\) patients experience an outcome and the prior is \(p\sim\mathrm{Beta}(a,b)\), then the posterior is \(p\mid D\sim\mathrm{Beta}(a+x,b+n-x)\). The posterior mean is \((a+x)/(a+b+n)\). A credible interval is obtained from the quantiles of this posterior distribution. This conjugate calculation is a compact illustration; a joint dose model may instead produce intervals from posterior draws.

Posterior Probability That a Dose Is Selected

Let \(A(\theta)\) denote the decision rule applied to a posterior parameter draw. The posterior selection probability for dose \(d\) is \(P\{A(\theta)=d\mid D\}\). If the rule can return no recommendation, the probabilities for the doses need not sum to one; the remaining probability belongs to that no-recommendation outcome. If the rule always chooses exactly one dose, the probabilities over doses sum to one, apart from numerical error.

When the posterior distribution is available only through simulation, draw \(\theta^{(1)},\ldots,\theta^{(M)}\) from it and apply the rule to every draw. If dose \(d\) is selected on \(m_d\) draws, estimate its selection probability by \(\hat{\pi}_d=m_d/M\). For approximately independent posterior draws, the Monte Carlo standard error is estimated by

This standard error describes simulation error in estimating the posterior probability; it is not the posterior uncertainty about the dose response itself. With correlated simulation draws, the independent-draw formula can understate simulation error. Use an effective sample size or an appropriate Monte Carlo error estimate for the sampling method.

Probability That Each Dose Is Optimal

To calculate optimality probabilities, first define a utility and the action set. For illustration, let \(U_d=p_{E,d}-1.5p_{T,d}\), so one unit of toxicity probability receives 1.5 times the weight of one unit of efficacy probability. This numerical trade-off is a decision preference, not a statistical fact. Suppose doses are admissible only when \(p_{T,d}\leq 0.30\), and include a no-treatment option with utility \(U_0=0\). On each posterior draw, compare the utilities of all admissible doses with zero. The probability that dose \(d\) is optimal is the posterior probability that it has the largest utility among those options.

If posterior draws are available, count how often each action maximizes utility and divide by the number of draws. If ties have positive probability, the analysis must specify how ties are handled—for example, splitting the tied draw equally among maximizers or applying a prespecified tie-break rule. With continuous posterior distributions, exact ties will usually have probability zero, but the convention should still be clear.

Interpretation, Diagnostics, and Limitations

Report these summaries together rather than treating any one as a complete description. Credible intervals show uncertainty in rates at individual doses. Selection probabilities show how often a decision algorithm recommends each dose across posterior uncertainty. Optimality probabilities show how often each option maximizes the chosen benefit–risk objective. The choice of objective can matter: changing the toxicity weight, safety constraint, or no-treatment utility can change which dose appears optimal even when the posterior model is unchanged.

For diagnostics, check that posterior draws adequately represent the fitted posterior and that Monte Carlo error is small enough for the intended interpretation. If estimates change substantially when the number of draws is increased, the simulation is not yet precise enough. Check whether dose-specific intervals are unexpectedly broad, whether posterior probabilities cluster near decision thresholds, and whether one dose’s apparent advantage depends on a narrow modeling or prior choice. A probability reported to several decimal places is not informative if its Monte Carlo error or model sensitivity is much larger.

Intervals for separate dose-specific rates are marginal intervals, not simultaneous statements that all rates lie in their intervals together. Also, posterior probabilities do not remove model uncertainty: if the outcome model, prior, or decision criterion is poorly chosen, highly precise simulation can still yield a misleading summary. State the outcome definition, model and prior, decision rule or utility, safety constraints, and simulation method alongside the reported values. In a joint model, preserve the joint posterior structure when evaluating criteria involving both efficacy and toxicity.