Dose Selection Under Model Uncertainty

Why Model Uncertainty Matters in Dose Selection

A dose-response analysis can estimate parameters precisely within a chosen model and still give a fragile recommendation. For example, a saturating model may imply that benefit is nearly at its plateau by a moderate dose, while a second plausible model predicts that benefit continues to increase at higher doses. If the predicted safety risk also differs, choosing a dose from only one model can misstate the benefit-risk trade-off. Model uncertainty is uncertainty about which candidate model adequately represents the dose-response relationship; it is distinct from uncertainty about the parameters of a particular model.

The goal is not to declare that every model is equally credible. It is to make the consequences of plausible alternatives explicit. A decision-maker can average across models when defensible model probabilities are available, or use a robust criterion when protecting against poor performance under any retained model is more important than optimizing a weighted average. Either approach depends on defining a common decision target and a common utility scale.

Set Up Comparable Predictions Before Comparing Models

Every retained model must predict the same estimand for the same population, dosing regimen, follow-up period, and candidate doses. If one model predicts mean change in a clinical outcome and another predicts the probability of crossing a response threshold, those predictions cannot simply be averaged. First express them on a common scale, or calculate each model's expected utility using the same decision criterion. The earlier material on dose-selection estimands and utility-based dose optimization provides the relevant decision framework; here the additional issue is how to carry uncertainty about the model into that framework.

For example, suppose benefit \(B_m(d)\) is expressed in clinical benefit points and harm \(H_m(d)\) is the predicted risk of a specified adverse event in percentage points. If the decision-makers have agreed that one percentage point of additional risk is worth \(\lambda\) benefit points, a simple utility is \(U_m(d)=B_m(d)-\lambda H_m(d)\). The value of \(\lambda\) encodes a preference; it is not estimated merely by fitting a dose-response model. A utility that combines unlike outcomes without an explicit preference rule is not interpretable.

Within each model, the predictions may already average over parameter uncertainty. For instance, \(B_m(d)\) and \(H_m(d)\) can be expectations under the parameter distribution conditional on the data and \(M_m\). This tutorial focuses on the additional step of comparing models; averaging model predictions does not remove uncertainty or bias caused by all candidate models being inadequate.

Model Averaging and Robust Criteria

If \(w_m\) is the probability or decision weight assigned to model \(M_m\), with \(w_m\geq 0\) and \(\sum_{m=1}^K w_m=1\), model-averaged utility at dose \(d\) is

When weights are posterior model probabilities, they represent the relative support for models given the data and the specified candidate set and prior model probabilities. They are not probabilities that a model is literally true. Other justified weights may be used, but their interpretation must be stated. The model-averaged decision selects a dose that maximizes \(\bar U(d)\) among the candidate doses.

A robust decision rule gives less influence to model weights and emphasizes unfavorable scenarios. A common rule is maximin utility: select the dose with the largest worst-case utility across the retained models. A different rule is minimax regret. For each model, regret is the utility lost relative to the best dose that could have been chosen if that model were known. The minimax-regret dose minimizes the largest such loss.

Maximin can be cautious when a model assigns very low utility to a dose, even if that model has little support. Minimax regret instead asks how costly a decision could be relative to the model-specific best choice. Neither criterion is automatically superior: the relevant choice depends on how the decision-makers value downside protection, and on whether the candidate model set meaningfully represents plausible alternatives.

Why Averaging Utilities Is the Relevant Calculation

For a fixed dose, expected utility can be averaged over model uncertainty because expectation is linear. If model \(M_m\) has weight \(w_m\), then the expectation of utility across models is the sum of each model's expected utility multiplied by its weight. This gives the model-averaged criterion above. But choosing a dose is an optimization step, and maximization is not generally interchangeable with averaging.

To justify the inequality, for every dose \(d\), \(U_m(d)\leq \max_{d'}U_m(d')\) in every model. Multiplying by nonnegative \(w_m\) and summing gives \(\sum_m w_mU_m(d)\leq\sum_m w_m\max_{d'}U_m(d')\). Taking the maximum over \(d\) on the left preserves the inequality. The right side describes the expected model-specific optimum if the model were known before choosing the dose; it is not the utility available when one dose must be selected under current uncertainty.

In the worked example, the best dose under \(M_1\) is 50 mg/day, with utility 6.5; under \(M_2\), it is 75 mg/day, with utility 6.0. The weighted average of those model-specific best utilities is \(0.8(6.5)+0.2(6.0)=6.40\), greater than the best achievable model-averaged utility, 5.90. Selecting a dose separately for each model would require knowing which model applies; it does not justify reporting 6.40 as the utility of one currently selectable dose.

How Much Do the Model Weights Matter?

Weights should be treated as inputs to a decision calculation, not as an invisible technical detail. A useful check is to find the weight at which the preferred dose changes. For the two leading doses in the example, let \(w\) be the weight on \(M_1\); the weight on \(M_2\) is \(1-w\). Then:

Setting these equal gives \(3.5+3w=6.0-0.5w\), so \(w=2.5/3.5=0.7143\). If \(w>0.7143\), 50 mg/day has greater model-averaged utility than 75 mg/day; if \(w<0.7143\), 75 mg/day has greater utility. At exactly that weight, the two candidate doses tie on this criterion. This calculation makes clear that the 50 mg/day recommendation depends on assigning more than about 71.4% weight to \(M_1\).

Assumptions, Diagnostics, and Alternatives

Model averaging and robust criteria are useful only when their inputs are defensible. At minimum, check that the candidate models address the same estimand, use compatible outcome and safety definitions, and make predictions over the doses being compared. Confirm that model weights are nonnegative and sum to one, and record how they were obtained. If weights come from a Bayesian analysis, they depend on the candidate model set, prior model probabilities, likelihood assumptions, and data; they should not be presented as objective probabilities that a model is true.

Inspect model-specific predicted benefit, harm, and utility across the dose range. Large disagreement is itself decision-relevant: it may indicate that the selected dose is driven by a model-dependent extrapolation or by poorly informed parts of the response curve. Check whether the rankings change under reasonable utility weights and model weights, whether an apparently favorable utility hides an unacceptable safety prediction, and whether predictions at untested doses rely on extrapolation beyond the data. If model-averaged predictions look smooth, do not mistake that smoothness for agreement: averaging can conceal sharply different model-specific curves.

The main alternatives answer different questions. Selecting the single best-fitting model is simpler but can ignore uncertainty among models with similar support. Model averaging accounts for that uncertainty through weights, but its recommendation depends on the candidate set and weights. Maximin and minimax regret protect against specific worst-case losses but can be conservative or depend strongly on which models are included. No rule repairs a shared structural flaw: if all retained models omit an important feature, agreement among them does not establish that the predictions are reliable.

A dose recommendation should therefore be accompanied by its decision rule, model set, weights or robustness criterion, utility definition, and model-specific predictions. That information allows the reader to distinguish a stable recommendation from one that turns on a particular structural assumption. The numerical choice is conditional on these ingredients; it is not a model-free property of the data.