Posterior Model Weights for Dose Selection

Why Posterior Model Weights Matter

A continual reassessment method (CRM) can use more than one plausible dose-toxicity skeleton. The observed outcomes may support some candidate skeletons more than others, so assigning equal importance to every skeleton after observing data can discard relevant evidence. Posterior model weights quantify how the prior probabilities on the candidate skeletons change in light of the trial data.

For dose selection, these weights are not merely a model-comparison summary. They combine model-specific posterior toxicity predictions into a single prediction for each candidate dose. A dose rule can then be applied to those averaged predictions. This is different from selecting the most probable skeleton first and acting as if it were certainly correct.

Definition: Let $M_j$ denote candidate CRM skeleton model $j$, with prior model probability $\pi_j=P(M_j)$, where $\sum_{j=1}^J \pi_j=1$. Let $\theta_j$ be its CRM parameter, with prior density $g_j(\theta_j)$. For observed trial data $D$, the marginal likelihood under model $M_j$ is $$ m_j(D)=P(D\mid M_j)=\int L_j(\theta_j;D)g_j(\theta_j)\,d\theta_j, $$ where $L_j(\theta_j;D)$ is the likelihood. The posterior model weight is $$ w_j=P(M_j\mid D)=\frac{\pi_jm_j(D)}{\sum_{k=1}^J \pi_km_k(D)}. $$

Likelihood Integration and the Weight Update

Suppose patient $r$ receives dose $d_r$, and $y_r=1$ indicates a dose-limiting toxicity (DLT), while $y_r=0$ indicates no DLT. Under a candidate CRM model, the probability of DLT at dose $d$, conditional on $\theta_j$, is $p_{j,d}(\theta_j)$. Conditional on the assigned doses and $\theta_j$, the usual Bernoulli likelihood is

L_j(\theta_j;D)=\prod_{r=1}^n p_{j,d_r}(\theta_j)^{y_r} \{1-p_{j,d_r}(\theta_j)\}^{1-y_r}.

This likelihood conditions on the dose assignments. In a trial with adaptive assignments, the doses were chosen using the trial’s allocation rule; the toxicity likelihood above is the standard conditional likelihood when inference is based on the observed dose-outcome pairs and the allocation mechanism does not add an unmodelled outcome-dependent selection process. The model also assumes that, conditional on the model parameter and assigned doses, patient DLT outcomes follow the specified Bernoulli probabilities and are independent. Delayed or dependent outcomes require an appropriate extension rather than an automatic use of this likelihood.

The marginal likelihood averages the likelihood over the model’s prior parameter distribution. It is not the likelihood evaluated at a best-fitting parameter, and it is not the posterior probability of the data. Integrating rather than maximizing matters: a model that fits well only in a narrow region of its prior parameter space can receive less marginal support than one that fits reasonably well across a broader region.

For a finite prior distribution or numerical quadrature with nodes $\theta_{j\ell}$ and prior weights $a_{j\ell}$, where $\sum_\ell a_{j\ell}=1$, the integral is approximated by

m_j(D)\approx \sum_{\ell}a_{j\ell}L_j(\theta_{j\ell};D).

After calculating each $m_j(D)$, multiply it by the prior model probability and normalize across models. This update follows directly from Bayes’ rule: $P(M_j,D)=P(D\mid M_j)P(M_j)$, and summing that joint probability over all candidate models gives $P(D)=\sum_k P(D\mid M_k)P(M_k)$.

Worked Example: Updating Two Skeleton Weights

Consider a hypothetical trial with one patient treated at dose 1 and no observed DLT. There are two candidate skeleton models, each with prior probability 0.50. For a transparent calculation, each model has two equally likely parameter values. The dose-1 DLT probabilities at those values are 0.10 and 0.40 under $M_1$, and 0.20 and 0.50 under $M_2$. The observed outcome is $y_1=0$, so the likelihood at each parameter value is one minus its dose-1 DLT probability.

Under $M_1$, the two likelihoods are $1-0.10=0.90$ and $1-0.40=0.60$. Thus

$$ m_1(D)=0.50(0.90)+0.50(0.60)=0.75. $$

Under $M_2$, the likelihoods are $1-0.20=0.80$ and $1-0.50=0.50$, giving

$$ m_2(D)=0.50(0.80)+0.50(0.50)=0.65. $$

The unnormalized posterior weights are $0.50(0.75)=0.375$ and $0.50(0.65)=0.325$. Their sum is $0.700$, so

w_1=\frac{0.375}{0.700}=0.5357,\qquad w_2=\frac{0.325}{0.700}=0.4643.

The observed no-DLT outcome makes $M_1$ somewhat more probable than it was a priori, because its marginal likelihood is larger. It does not make $M_1$ certain; $M_2$ retains posterior weight 0.4643.

From Model Weights to a Dose Recommendation

Within each model, the parameter posterior is updated using the likelihood:

g_j(\theta_j\mid D)= \frac{L_j(\theta_j;D)g_j(\theta_j)}{m_j(D)}.

For candidate dose $d$, the model-specific posterior mean DLT probability is $\tilde p_{j,d}=E[p_{j,d}(\theta_j)\mid D,M_j]$. Averaging over uncertainty about which skeleton is appropriate gives

\bar p_d=E[p_d\mid D] =\sum_{j=1}^J w_j\tilde p_{j,d}.

This is the posterior expectation obtained by first conditioning on each candidate model and then averaging over the posterior model probabilities. Under a target-toxicity rule, a common recommendation is the eligible dose whose $\bar p_d$ is closest to the target $\phi$. Eligibility restrictions, safety exclusions, and stopping rules remain in force; model averaging does not replace them.

Worked Example: Averaging Predictions Before Selecting a Dose

Suppose a hypothetical CRM analysis of accumulated trial data gives two skeletons posterior weights $w_1=0.48498$ and $w_2=0.51502$. These weights result, for example, from equal prior model probabilities and marginal likelihoods $m_1(D)=0.09460$ and $m_2(D)=0.10046$: normalizing gives $0.09460/(0.09460+0.10046)=0.48498$ and $0.51502$, to five decimal places. Suppose the model-specific posterior mean DLT probabilities are as follows. These are illustrative posterior summaries from the hypothetical analysis.

Candidate dose	$\tilde p_{1,d}$	$\tilde p_{2,d}$
Dose 1	0.08000	0.10167
Dose 2	0.20000	0.25258
Dose 3	0.41593	0.49333

For Dose 1, the model-averaged probability is

\bar p_1=(0.48498)(0.08000)+(0.51502)(0.10167) =0.0387984+0.0523621=0.09116.

For Dose 2,

\bar p_2=(0.48498)(0.20000)+(0.51502)(0.25258) =0.0969960+0.1300838=0.22708.

For Dose 3,

\bar p_3=(0.48498)(0.41593)+(0.51502)(0.49333) =0.2017177314+0.2540748166=0.455792548\approx0.45579.

With target $\phi=0.25$ and all three doses eligible, compare absolute differences from the target. They are $0.15884$ for Dose 1, $0.02292$ for Dose 2, and $0.20579$ for Dose 3. Dose 2 is closest to the target and is therefore selected by this rule. These probabilities describe posterior toxicity predictions, not guarantees about the outcome of a future patient.

A Short Derivation and a Prior-Sensitivity Check

The model-averaged prediction is a direct consequence of conditioning on the model label. For any dose $d$, partition the posterior expectation according to the mutually exclusive models:

E[p_d\mid D] =\sum_{j=1}^J E[p_d\mid D,M_j]P(M_j\mid D) =\sum_{j=1}^J \tilde p_{j,d}w_j.

The result requires the candidate models to form the set over which the analysis is averaging, with posterior weights that sum to one. It does not require the skeletons to be equally likely, nor does it mean the averaged dose-response curve is itself one of the candidate CRM skeletons.

Worked Example: Changing the Prior Model Probabilities

Use the marginal likelihoods $m_1(D)=0.75$ and $m_2(D)=0.65$ from the first example, but suppose prior model probabilities are now $P(M_1)=0.25$ and $P(M_2)=0.75$. The unnormalized weights are $0.25(0.75)=0.1875$ and $0.75(0.65)=0.4875$. Therefore

w_1=\frac{0.1875}{0.1875+0.4875}=0.2778,\qquad w_2=0.7222.

For illustration, suppose the posterior mean DLT probabilities at a candidate dose are $0.39$ under $M_1$ and $0.4846$ under $M_2$. The model-averaged probability is

\bar p_d=(0.2778)(0.39)+(0.7222)(0.4846)=0.4583.

The likelihood evidence is unchanged, but the posterior average changes because the prior model probabilities changed. This is why sensitivity analysis should examine both the skeletons and their prior weights when a recommendation depends materially on model averaging.

Diagnostics, Alternatives, and Limitations

Numerical integration should be checked rather than treated as an invisible computational detail. For each model, confirm that prior quadrature weights sum to one and that $m_j(D)$ is positive. Repeat the calculation with a finer grid or a more accurate integration method; substantial changes in the marginal likelihoods can change the posterior model weights. For many patients, products of small likelihood terms can underflow numerically, so computations on the log scale are preferable: calculate log-likelihoods, use a stable log-sum-exp calculation for the integral, and normalize the resulting log weights.

Other warning signs include a candidate model receiving nearly all posterior weight after a small amount of data, large recommendation changes under modest changes to prior skeleton weights, and a model-averaged prediction that conflicts sharply with observed outcomes. These do not automatically show that the calculation is wrong. They may indicate that the candidate skeletons, parameter priors, or assumed likelihood deserve closer examination. Posterior model weights can only distribute support among the models included; they cannot reveal that all candidate skeletons are inadequate.

Three approaches should be distinguished. Model averaging uses every model’s prediction, weighted by its posterior probability. Model selection chooses one model, often the one with the largest posterior weight, and bases the recommendation on that model alone; this discards uncertainty about the model label. Prior-weighted averaging without updating uses $\pi_j$ in place of $w_j$, so it does not incorporate the relative marginal likelihoods from the observed data. These are different decision procedures and can recommend different doses.

The target-closeness rule also has limits. The dose with average toxicity closest to $\phi$ need not be the dose with the largest posterior probability of being within a specified interval around $\phi$. That is a different decision criterion. Moreover, an average can conceal disagreement: for example, two models can place predictions on opposite sides of the target even when their weighted average is close to it. Inspect model-specific predictions alongside $\bar p_d$, apply clinical eligibility constraints, and report sensitivity when the recommendation is fragile. Posterior model weights quantify uncertainty only across the candidate skeletons and priors used in the analysis.

Common mistake: “The most probable skeleton has weight 0.52, so I should use only its dose recommendation.” That replaces model averaging with model selection. If the intended rule is model averaging, calculate $\bar p_d=\sum_jw_j\tilde p_{j,d}$ at each dose and then apply the stated dose rule. Selecting one skeleton may be reasonable under a separately specified decision procedure, but it is not equivalent to averaging.

Check Your Understanding

Use the definitions and calculations above to answer each question. Solutions follow each prompt.

If two models have equal prior weights and marginal likelihoods 0.30 and 0.20, what are their posterior weights?
Solution: The unnormalized weights are $0.50(0.30)=0.15$ and $0.50(0.20)=0.10$. Dividing by their sum, 0.25, gives posterior weights 0.60 and 0.40.
Why is a marginal likelihood not generally the likelihood at the posterior mean parameter?
Solution: The marginal likelihood integrates the likelihood over the prior parameter distribution, $m_j(D)=\int L_j(\theta_j;D)g_j(\theta_j)\,d\theta_j$. Evaluating the likelihood at one parameter value does not perform this prior-weighted integration.
At a dose, two models have posterior weights 0.70 and 0.30 and model-specific posterior mean DLT probabilities 0.20 and 0.50. Find the model-averaged probability.
Solution: $\bar p_d=(0.70)(0.20)+(0.30)(0.50)=0.14+0.15=0.29$.
Does a posterior model weight of 0.80 mean there is an 80% probability that the selected model is scientifically true?
Solution: No. It is the posterior probability assigned to that model conditional on the candidate model set, prior probabilities, likelihood, and observed data. It does not establish that the model is literally true or that omitted models are impossible.
Two models predict DLT probabilities 0.10 and 0.40 at a dose, with posterior weights 0.50 each. The target is 0.25. What important diagnostic should accompany the average?
Solution: The average is $0.50(0.10)+0.50(0.40)=0.25$, exactly the target. Report the model-specific predictions as well: they disagree substantially and lie on opposite sides of the target, a disagreement hidden by the average.