Why Average Across Dose-Response Models?
Dose selection often depends on predictions at doses that were not directly studied, or on how predicted benefit and risk compare across candidate doses. A dose-response model translates the available data into those predictions. But even when several models are scientifically plausible, choosing one model alone can make the decision depend heavily on that choice. Model averaging makes this dependence explicit: it combines predictions from the candidate models, giving each model a specified weight.
The object to average is usually the prediction relevant to the decision, such as expected efficacy, probability of meeting a target, or expected utility. Averaging predictions does not make the candidate models interchangeable, prove that the set of models is adequate, or remove uncertainty. It provides a decision summary that accounts for uncertainty about which model best represents the dose-response relationship.
The predictions must refer to the same endpoint, population, treatment duration, estimand, and dose. If one model predicts change from baseline and another predicts change relative to placebo, their outputs cannot be directly averaged without first making them comparable. Similarly, weights must be attached to clearly specified models and should reflect the evidence and purpose of the analysis, not a desired dose ranking.
Where the Weights Come From
One principled choice is Bayesian model averaging. If \(D\) denotes the observed data and \(\pi_k\) is the prior probability assigned to model \(M_k\), the posterior model probability is
Here \(P(D\mid M_k)\) is the marginal likelihood: the probability of the observed data under model \(M_k\), averaged over that model's parameter uncertainty under its parameter prior. These weights sum to one and represent relative support for the models in the specified candidate set, conditional on the data and prior choices. They are not the probability that a model is literally true.
Other weighting schemes answer somewhat different questions. For example, normalized Akaike weights are often calculated as \(\exp(-\Delta_k/2)/\sum_j\exp(-\Delta_j/2)\), where \(\Delta_k\) is model \(k\)'s AIC minus the smallest AIC in the set. These are useful relative predictive-support weights, but they are not generally Bayesian posterior model probabilities. Stacking weights instead choose a mixture to perform well for out-of-sample prediction. The weight method, model set, and rationale should be reported because different choices can change the averaged prediction.
A model-specific prediction may itself account for parameter uncertainty. In a Bayesian analysis, for instance, the prediction under \(M_k\) integrates over the posterior distribution of that model's parameters. The overall mixture then accounts for both parameter uncertainty within models and uncertainty about which model to use. Avoid treating a model prediction calculated at a single point estimate as if it automatically included either source of uncertainty.
Worked Example: Averaging Expected Efficacy Across Two Models
Suppose two plausible models predict placebo-adjusted improvement at three candidate doses. The endpoint is measured in points, with higher improvement better. The assigned weights are \(w_1=0.65\) and \(w_2=0.35\). All values are hypothetical.
| Dose (mg) | Model 1 prediction (points) | Model 2 prediction (points) |
|---|---|---|
| 25 | 4 | 3 |
| 50 | 7 | 6 |
| 100 | 9 | 10 |
At 25 mg, the model-averaged prediction is \(0.65(4)+0.35(3)=2.60+1.05=3.65\) points. At 50 mg it is \(0.65(7)+0.35(6)=4.55+2.10=6.65\) points. At 100 mg it is \(0.65(9)+0.35(10)=5.85+3.50=9.35\) points. Thus the averaged expected improvement increases across these candidate doses, and 100 mg has the largest averaged efficacy prediction. This ranking alone does not establish that 100 mg should be selected: safety, uncertainty, and the prespecified decision criterion also matter.
Average the Decision-Relevant Quantity
Dose selection is a decision, not merely a prediction exercise. A decision rule may require a minimum efficacy target, a maximum acceptable risk, or a balance between benefit and harm. The model-averaged prediction should match that rule. For a quantity \(U(d,M_k)\) representing the utility of dose \(d\) under model \(M_k\), the model-averaged utility is
When utility is linear in the predicted outcomes, averaging the model-specific outcomes and then calculating utility gives the same result as averaging the model-specific utilities. With nonlinear utility, threshold rules, or joint safety-efficacy criteria, that equivalence need not hold. In such cases, calculate the relevant probability or utility within each model and then average those quantities using the model weights.
Worked Example: Model Uncertainty Changes the Dose Ranking
Consider two candidate doses, 50 mg and 100 mg, and two plausible models with weights \(0.60\) and \(0.40\). For illustration, define utility in efficacy points as efficacy improvement minus 20 times the probability of a specified adverse event. The multiplier is a hypothetical, prespecified penalty: a 0.10 increase in event probability subtracts 2 utility points. The table entries below are hypothetical.
| Model | Weight | 50 mg efficacy | 50 mg event probability | 100 mg efficacy | 100 mg event probability |
|---|---|---|---|---|---|
| 1 | 0.60 | 7.0 | 0.04 | 9.0 | 0.12 |
| 2 | 0.40 | 7.2 | 0.05 | 8.5 | 0.25 |
Under Model 1, utility at 50 mg is \(7.0-20(0.04)=6.2\) points; at 100 mg it is \(9.0-20(0.12)=6.6\) points. Model 1 therefore ranks 100 mg higher. Under Model 2, utility at 50 mg is \(7.2-20(0.05)=6.2\) points; at 100 mg it is \(8.5-20(0.25)=3.5\) points. Model 2 ranks 50 mg higher.
The model-averaged utilities are \(\bar U(50)=0.60(6.2)+0.40(6.2)=6.20\) points and \(\bar U(100)=0.60(6.6)+0.40(3.5)=3.96+1.40=5.36\) points. With this utility definition and these weights, 50 mg has the higher averaged utility, even though the higher dose wins under Model 1. The result is conditional on the specified penalty, candidate models, and weights; it is not a universal dose recommendation.
Predictive Uncertainty in a Model Mixture
A model-averaged mean does not describe the full uncertainty in the prediction. Suppose the predictive distribution under model \(M_k\) at dose \(d\) has mean \(\mu_k(d)\) and variance \(\sigma_k^2(d)\). The model-averaged predictive distribution is a mixture: first a model is selected according to the weights, then an outcome is drawn from that model's predictive distribution. Its mean is the weighted mean of the model means. Its variance contains both within-model predictive variance and between-model disagreement.
The first term is the average predictive variance within models. The second is the weighted variance of the model-specific means. To justify the result, write the total variance as the expected conditional variance plus the variance of the conditional mean. Conditional on \(M_k\), the outcome has variance \(\sigma_k^2(d)\) and mean \(\mu_k(d)\). Averaging the conditional variance gives the first term; taking the variance of the conditional mean across models gives the second. Omitting the second term understates uncertainty whenever the models disagree in their predictions.
Worked Example: Calculating a Mixture Probability for a Target
At 100 mg, suppose two models give hypothetical predictive distributions for a continuous efficacy endpoint \(Y\), measured in points: under Model 1, \(Y\sim N(9,2^2)\); under Model 2, \(Y\sim N(7,3^2)\). Let their weights be 0.60 and 0.40. Calculate the model-averaged probability that \(Y\) is at least 8 points.
For Model 1, standardizing the target gives \(z=(8-9)/2=-0.5\), so \(P(Y\geq8\mid M_1)=1-\Phi(-0.5)=0.6915\). For Model 2, \(z=(8-7)/3=0.3333\), so \(P(Y\geq8\mid M_2)=1-\Phi(0.3333)\approx0.3694\). The model-averaged target probability is therefore \(0.60(0.6915)+0.40(0.3694)=0.4149+0.1478=0.5627\), or about 56.3%.
The predictive mean is \(\bar{\mu}=0.60(9)+0.40(7)=8.20\) points. The within-model variance contribution is \(0.60(4)+0.40(9)=6.00\) points squared. The between-model contribution is \(0.60(9-8.2)^2+0.40(7-8.2)^2=0.384+0.576=0.960\) points squared. Thus the mixture variance is \(6.00+0.960=6.96\) points squared, and its standard deviation is \(\sqrt{6.96}=2.638\) points.
The mixture is not generally normal. Replacing it with a normal distribution having mean 8.20 and variance 6.96 would give a target probability of approximately \(1-\Phi((8-8.2)/2.638)=0.5302\), not the mixture probability of 0.5627. For a threshold decision, calculate and average the model-specific probabilities rather than assuming that a moment-matched normal distribution preserves the relevant tail probability.
Practical Checks, Alternatives, and Limitations
Before using a model-averaged result, check that each model produces predictions for the same dose, endpoint, time point, population, and estimand, and that all weights are nonnegative and sum to one. Recalculate the ranking under reasonable alternative weights or model sets. A ranking that changes substantially is evidence that the decision is sensitive to model uncertainty and should be presented as such, not hidden by reporting only the average.
Model averaging is not a substitute for checking whether the candidate models are scientifically credible. A mixture of inadequate models can still make poor predictions. Examine the fit and predictive performance of each model using appropriate checks, and investigate implausible predictions, boundary behavior, or unsupported extrapolation. Detailed diagnosis of dose-response model misspecification is a separate task; averaging should not be presented as a remedy for it.
The main alternatives have distinct limitations. Selecting the single best-supported model is simpler, but discards uncertainty about model choice and can make the result depend on a small difference in fit. Equal weighting is transparent but does not, by itself, reflect relative support or predictive performance. Averaging model parameters and then predicting is generally not equivalent to averaging predictions: for a nonlinear dose-response function \(f\), \(\sum_k w_k f(\theta_k,d)\) need not equal \(f(\sum_k w_k\theta_k,d)\). Model averaging also remains conditional on the candidate model set, the weight method, and any assumptions used to obtain each model's predictions.
For a dose decision, report the model-specific predictions alongside the averaged result, the weight method, and the decision criterion. Include uncertainty relevant to that criterion—for example, a model-averaged target probability or a predictive interval—not only an averaged point estimate. This makes clear whether the recommendation reflects consistent evidence across models or a weighted compromise between materially different conclusions.
Check Your Understanding
Answer each question, then compare your work with the solutions below.
- Two models have weights 0.70 and 0.30 and predict 5 and 9 points of improvement at a candidate dose. What is the model-averaged prediction?
- At a dose, two models have predictive means 6 and 10 points, weights 0.50 each, and within-model variances 4 and 4 points squared. Calculate the model-averaged mean and variance.
- Why can the probability of exceeding a target not always be calculated accurately by assuming a normal distribution with the model-averaged mean and variance?
- At one dose, model-specific utilities are 8 and 2 points, with weights 0.25 and 0.75. Calculate averaged utility and state which model has the greater weight.
Solutions. (1) \(0.70(5)+0.30(9)=3.5+2.7=6.2\) points. (2) The mean is \(0.50(6)+0.50(10)=8\) points. The within-model contribution is \(0.50(4)+0.50(4)=4\); the between-model contribution is \(0.50(6-8)^2+0.50(10-8)^2=4\). Total variance is \(4+4=8\) points squared. (3) A mixture can have a shape different from a normal distribution, so its tail probability need not match the tail probability of a normal distribution with the same first two moments. Average model-specific target probabilities using the weights. (4) Averaged utility is \(0.25(8)+0.75(2)=2+1.5=3.5\) points. Model 2 has the greater weight, 0.75.