Why Average Across Candidate Models?
A dose recommendation can depend on the form of the dose-response model. One plausible model may predict a rapid increase in response at low doses, while another predicts a more gradual increase. If the data do not decisively distinguish these models, selecting one model and acting as if it were certain can make the recommendation sensitive to that choice. Model averaging provides a direct way to carry uncertainty about model identity into response predictions and decisions.
The aim here is specific: combine predictions across a finite set of candidate models using model weights, then use those combined predictions to recommend a dose. The models must address the same endpoint, target population, and dose decision. Model averaging does not make an unsuitable candidate model suitable, and it does not replace checking the assumptions behind the models and their predictions.
A weight expresses how much the model contributes to the mixture prediction. In Bayesian model averaging, \(w_k\) is often the posterior probability \(P(M_k\mid\text{data})\), calculated from model prior probabilities and the models' marginal likelihoods. Other justified schemes can assign weights from measures of predictive performance. Whatever the source, the weights should be stated, nonnegative, and normalized. They are not the same thing as a model's probability of making a particular response prediction.
The conditional prediction \(p_k(d)\) should represent the intended prediction under model \(M_k\). For example, it may already average over parameter uncertainty within that model. If it is only a prediction at a point estimate, then the resulting model average reflects model-weight uncertainty but omits that within-model parameter uncertainty. The analysis should make clear which prediction is being averaged.
Why the Weighted Probability Is the Right Mixture
The defining calculation follows from conditioning on which model generated the prediction. Let \(A\) be the event that a patient meets the response target at dose \(d\). The candidate models form mutually exclusive possibilities for this calculation: exactly one model index \(M_k\) applies. The law of total probability gives
This justification requires the model-specific probabilities to refer to the same event and conditioning information, and the weights to describe the same set of candidate models. In a non-Bayesian weighting approach, the same weighted sum is a defined mixture prediction under the chosen weights; it should not automatically be interpreted as a posterior probability.
A corresponding result holds for decision-making. Let \(U_k(d)\) be the expected utility of dose \(d\) under model \(M_k\), conditional on the information used for the decision. The expected utility under the mixture is
This follows from the same conditioning argument: average the utility conditional on each model, then weight by that model's probability or assigned mixture weight. A dose chosen by maximizing \(\bar{U}(d)\) is the dose with the greatest expected utility under this particular mixture and utility definition. It is not necessarily the dose preferred by every model.
Worked Example: Averaging a Response Probability at One Dose
Suppose three candidate models predict the probability of meeting a specified efficacy target at a hypothetical dose of 50 mg. The predictions are 0.72, 0.60, and 0.45 for \(M_1,M_2,M_3\), respectively. Suppose their normalized weights are 0.50, 0.30, and 0.20. Treat these values as hypothetical and assume each prediction describes the same target event and population.
The model-averaged probability is
Thus the mixture prediction is a 0.630 probability, or 63.0%, of meeting the target at 50 mg. This result is not the unweighted average of the predictions and is not a claim that the response probability is known without uncertainty. It is conditional on the candidate set, the stated predictions, and the weights.
From Averaged Probabilities to a Dose Rule
A probability calculation becomes a recommendation only after specifying a decision rule. One transparent rule is to choose the lowest dose on an allowed dose grid for which the model-averaged target-attainment probability reaches a prespecified threshold \(\tau\). The grid, target event, and threshold should be established for the decision at hand; the calculation alone does not determine them.
Worked Example: Choosing the Lowest Dose That Meets an Averaged Target
Consider three hypothetical models with weights 0.50, 0.30, and 0.20. Their predictions are probabilities of meeting the same efficacy target, and parameter uncertainty is assumed to have already been incorporated within each model. The allowed doses are 25, 50, and 75 mg. The decision threshold is \(\tau=0.80\).
| Dose (mg) | \(p_1(d)\) | \(p_2(d)\) | \(p_3(d)\) |
|---|---|---|---|
| 25 | 0.42 | 0.55 | 0.30 |
| 50 | 0.72 | 0.68 | 0.58 |
| 75 | 0.88 | 0.78 | 0.80 |
At 25 mg, the average is \( (0.50)(0.42)+(0.30)(0.55)+(0.20)(0.30)=0.435 \). At 50 mg, it is \( (0.50)(0.72)+(0.30)(0.68)+(0.20)(0.58)=0.680 \). At 75 mg, it is \( (0.50)(0.88)+(0.30)(0.78)+(0.20)(0.80)=0.834 \).
Only the 75 mg dose reaches the required probability of 0.80, so the rule selects 75 mg. This conclusion is specific to the given grid and threshold. It does not establish that 75 mg is safe, optimal under another objective, or adequate for a different target event.
Recommendations Based on Expected Utility
A threshold rule focuses on the probability of crossing one target. It may not capture the costs of higher doses, outcomes below the target, or other consequences that matter to the decision. When an expected-utility scale has been specified, calculate each model's expected utility at each dose, average those utilities using the model weights, and select the allowed dose with the largest average. The utility definition must encode the relevant benefits and harms; a numerical maximum has no decision meaning apart from that definition.
Worked Example: Maximizing Model-Averaged Utility
Suppose two hypothetical models have weights 0.60 and 0.40. Their expected utilities, measured in prespecified utility points, have been calculated for three allowed doses. Assume the utility values use the same scale and account for the decision consequences of interest.
| Dose (mg) | \(U_1(d)\) | \(U_2(d)\) |
|---|---|---|
| 25 | 4 | 8 |
| 50 | 9 | 4 |
| 75 | 10 | 1 |
The averaged utilities are
The largest average is 7.0, so the expected-utility rule selects 50 mg from this grid. Model \(M_1\) alone prefers 75 mg, while \(M_2\) alone prefers 25 mg; the mixture recommendation differs from both because it trades off their utilities according to the stated weights.
A tempting alternative is to average the models' separately optimal doses. That gives \( (0.60)(75)+(0.40)(25)=55 \) mg. But 55 mg is not on the allowed grid, and averaging dose values does not maximize the weighted utility values shown. Even if 55 mg were allowed, the best dose would have to be evaluated using utility predictions at 55 mg; it cannot be inferred from the average of the two optima.
Assumptions, Diagnostics, and Alternatives
Before using a model average, verify that its components support a coherent comparison. In particular, all models should predict the same endpoint at the same time point and for the same target population. Doses must have consistent units and meanings. Weights must be normalized and refer to the models whose predictions are being combined. If predictions are restricted to a dose range, avoid interpreting averages outside the range where the constituent models are supported.
Useful checks include recomputing the averaged probabilities from the model-specific values, confirming that they remain between 0 and 1, and checking that the weights sum to 1. Inspect each model's predictions at the recommended dose and nearby allowed doses. A mixture result can conceal substantial disagreement: for instance, an average near 0.80 could combine several predictions near 0.80 or predictions far above and below it. Those cases can have different implications for confidence in the recommendation.
Also examine sensitivity to the weights. Recalculate the recommendation under reasonable alternative weights, or remove a very low-weight model as a diagnostic and note whether the decision changes. Such checks do not create a new justified weighting scheme, but they reveal whether a recommendation depends heavily on a particular weighting choice. Predictions that are non-monotone in dose when monotonicity is scientifically expected, unexpectedly extreme predictions, or a sharp recommendation change after a small weight adjustment warrant review of model fit, implementation, and extrapolation.
The main alternatives answer different questions. Selecting a single best-supported model is simpler but treats model choice as settled. Model averaging represents uncertainty across the specified candidates, but depends on their coverage and weights. A worst-case or minimax rule focuses on performance under the least favorable candidate rather than the weighted average. These approaches can therefore recommend different doses without any arithmetic error: they encode different attitudes to model uncertainty and different decision objectives.
Model averaging has important limits. It cannot account for a plausible model that was omitted, repair biased predictions shared by all candidates, or make poorly justified weights reliable. It also does not by itself resolve uncertainty in clinical targets or establish that model-specific probabilities are calibrated. Report the candidate models, weights, prediction definition, dose rule, and resulting recommendation together so the numerical answer can be interpreted in context.
Check Your Understanding
Try each question, then compare with the solutions below.
- Two models have weights 0.7 and 0.3, and predict target-attainment probabilities 0.60 and 0.90 at a dose. What is the averaged probability?
- Why must weights for the full candidate set sum to 1 before they are used as mixture weights?
- At doses 20 and 40 mg, the averaged target probabilities are 0.74 and 0.82. Under a rule requiring at least 0.80 and selecting the lowest qualifying dose, what is recommended?
- Is the weighted average of model-specific optimal doses generally the dose that maximizes model-averaged expected utility? Explain.
Solutions. (1) \( (0.7)(0.60)+(0.3)(0.90)=0.42+0.27=0.69 \), or 69.0%. (2) Normalization makes the weights a probability distribution over the candidate models, so the weighted sum is a mixture prediction; weights summing to another value would not have that interpretation without normalization. (3) The 20 mg dose is below 0.80 and the 40 mg dose reaches 0.80, so recommend 40 mg. (4) No. The utility of each dose must be averaged across models and the resulting average compared across allowed doses; averaging separately chosen doses does not generally maximize that average utility.