Prior and Skeleton Sensitivity Analysis

Why Examine CRM Sensitivity?

A CRM recommendation is conditional on choices made before the trial data are observed. In the one-parameter power CRM, these include the toxicity skeleton, the prior distribution for the model parameter, and the target toxicity probability. Two specifications can both be clinically plausible yet produce different posterior toxicity estimates from the same cohort outcomes. Sensitivity analysis makes this dependence visible: it asks whether the dose recommendation and any prespecified stopping decision persist under reasonable alternatives.

The purpose is not to select whichever specification gives a preferred dose. Nor does a recommendation that stays unchanged prove that the model is correct. Rather, the analysis shows which assumptions have practical influence and whether the trial decision is robust to plausible changes in them.

Definition: A prior and skeleton sensitivity analysis compares CRM posterior dose estimates, recommendations, or stopping decisions across a prespecified set of plausible prior distributions and toxicity skeletons, using the same target and observed trial data unless a comparison explicitly changes them.

Model, Data, and Comparison Rules

For a fixed skeleton \(s_1,\ldots,s_J\), the one-parameter power CRM specifies the toxicity probability at dose \(j\) as \(p_j(\theta)=s_j^\theta\), where \(0<s_1<\cdots<s_J<1\) and \(\theta>0\). The skeleton represents the model's probabilities at \(\theta=1\). For any positive \(\theta\), the probabilities increase with dose and remain between zero and one.

Let \(y_i\) be the number of dose-limiting toxicities among \(n_i\) patients treated at dose \(i\), after their outcomes have been observed. Under the model, outcomes are conditionally independent Bernoulli observations with probability \(p_i(\theta)\), given \(\theta\) and the assigned doses. The likelihood, omitting binomial coefficients that do not depend on \(\theta\), is

L(\theta)=\prod_{i=1}^{J}p_i(\theta)^{y_i}\{1-p_i(\theta)\}^{n_i-y_i}.

A prior \(\pi(\theta)\) gives posterior density proportional to \(L(\theta)\pi(\theta)\). For a discrete prior with support \(\theta_1,\ldots,\theta_K\) and prior probabilities \(w_1,\ldots,w_K\), the posterior probability of \(\theta_k\) is

P(\theta=\theta_k\mid\text{data}) =\frac{w_kL(\theta_k)}{\sum_{\ell=1}^{K}w_\ell L(\theta_\ell)}.

At each dose \(j\), the posterior mean toxicity probability is the posterior average of \(p_j(\theta)\). The usual target-dose rule selects the dose whose posterior mean toxicity probability is closest to the target \(\tau\), subject to any protocol safety restrictions. If the protocol has a stopping rule, apply it separately and exactly as prespecified; changing a prior or skeleton does not itself define a new stopping rule.

Sensitivity comparisons are most interpretable when the same data, candidate doses, target, and decision rules are held fixed while one modeling choice is changed. A second comparison can then examine a different choice or a jointly changed specification. Skeletons should be increasing and lie strictly between zero and one; prior support must be in the parameter space. The Bernoulli likelihood also presumes the outcomes included in the calculation are observed and classified as specified by the trial protocol.

How Posterior Sensitivity Arises

For two parameter values, the posterior odds equal the prior odds multiplied by the likelihood ratio. This gives a compact way to see how the observed outcomes and prior interact. Write \(\theta_L\) and \(\theta_H\) for two possible parameter values. At a dose with \(y\) toxicities among \(n\) patients, the likelihood contribution at \(\theta\) is \(p(\theta)^y\{1-p(\theta)\}^{n-y}\). Therefore,

\frac{P(\theta_L\mid\text{data})}{P(\theta_H\mid\text{data})} = \frac{P(\theta_L)}{P(\theta_H)} \prod_{i=1}^{J} \left\{\frac{p_i(\theta_L)}{p_i(\theta_H)}\right\}^{y_i} \left\{\frac{1-p_i(\theta_L)}{1-p_i(\theta_H)}\right\}^{n_i-y_i}.

Taking logarithms converts the product into a sum of dose-specific contributions. A toxicity contributes \( \log\{p_i(\theta_L)/p_i(\theta_H)\}\); a nontoxicity contributes \( \log\{[1-p_i(\theta_L)]/[1-p_i(\theta_H)]\}\). The prior odds remain as an additive term on the log scale. Thus a small amount of data can leave posterior weights sensitive to the prior, while a different skeleton can change the likelihood contributions themselves. Posterior dose estimates inherit both effects through the probabilities \(p_j(\theta)\).

Worked Example: Changing the Prior Changes the Recommended Dose

Suppose three doses have skeleton \(s=(0.09,0.25,0.49)\), the target is \(\tau=0.30\), and the prior places equal probability on \(\theta=0.5\) and \(\theta=1.5\). One patient has been treated at dose 2 with no toxicity. Compare the equal prior with a prior assigning probabilities 0.80 and 0.20 to those parameter values, respectively.

The toxicity probabilities at the two parameter values are \((0.30,0.50,0.70)\) for \(\theta=0.5\), and \((0.027,0.125,0.343)\) for \(\theta=1.5\). Because the observation is one nontoxicity at dose 2, the likelihoods are \(1-0.50=0.50\) and \(1-0.125=0.875\). With equal prior probabilities, the posterior weights are

P(\theta=0.5\mid\text{data})=\frac{0.5(0.50)}{0.5(0.50)+0.5(0.875)} =\frac{4}{11},\qquad P(\theta=1.5\mid\text{data})=\frac{7}{11}.

The posterior mean probabilities at doses 1, 2, and 3 are

\begin{aligned} \bar p_1&=\frac{4(0.30)+7(0.027)}{11}=\frac{1.389}{11}=0.1263,\\ \bar p_2&=\frac{4(0.50)+7(0.125)}{11}=\frac{2.875}{11}=0.2614,\\ \bar p_3&=\frac{4(0.70)+7(0.343)}{11}=\frac{5.201}{11}=0.4728. \end{aligned}

Their absolute distances from 0.30 are 0.1737, 0.03864, and 0.1728, so the target-distance rule recommends dose 2.

Now assign prior probabilities 0.80 and 0.20. The unnormalized posterior weights are \(0.80(0.50)=0.40\) and \(0.20(0.875)=0.175\), with total 0.575. The normalized weights are \(16/23=0.6957\) and \(7/23=0.3043\). The dose-specific posterior means become

\begin{aligned} \bar p_1&=\frac{16(0.30)+7(0.027)}{23}=\frac{4.989}{23}=0.2169,\\ \bar p_2&=\frac{16(0.50)+7(0.125)}{23}=\frac{8.875}{23}=0.3859,\\ \bar p_3&=\frac{16(0.70)+7(0.343)}{23}=\frac{13.601}{23}=0.5913. \end{aligned}

The distances from the target are 0.08309, 0.08587, and 0.2913. The recommendation changes to dose 1. This difference is not caused by different patient outcomes: it is a direct consequence of changing the prior weights.

Worked Example: Changing the Skeleton Changes the Recommendation

Keep the equal prior on \(\theta=0.5\) and \(\theta=1.5\), the target 0.30, and the same observation of no toxicity among one patient at dose 2. Replace the skeleton with \(s=(0.04,0.16,0.36)\). The toxicity probabilities are \((0.20,0.40,0.60)\) at \(\theta=0.5\), and \((0.008,0.064,0.216)\) at \(\theta=1.5\). The likelihoods of the observed nontoxicity at dose 2 are 0.60 and 0.936, so the posterior weights are

P(\theta=0.5\mid\text{data})=\frac{0.60}{0.60+0.936}=\frac{25}{64}, \qquad P(\theta=1.5\mid\text{data})=\frac{39}{64}.

The posterior means are

\begin{aligned} \bar p_1&=\frac{25(0.20)+39(0.008)}{64}=\frac{5.312}{64}=0.08300,\\ \bar p_2&=\frac{25(0.40)+39(0.064)}{64}=\frac{12.496}{64}=0.1953,\\ \bar p_3&=\frac{25(0.60)+39(0.216)}{64}=\frac{23.424}{64}=0.3660. \end{aligned}

The distances from 0.30 are 0.2170, 0.1048, and 0.06600, so dose 3 is closest. Under the first skeleton, the same data and equal prior recommended dose 2. The comparison isolates skeleton sensitivity: the model probabilities attached to the doses changed, and that altered both the likelihood update and the posterior dose estimates.

Sensitivity of Stopping Decisions

Sensitivity analysis should report stopping decisions as well as dose rankings when the protocol uses a stopping rule. For illustration, suppose a protocol stops if no dose has a posterior mean toxicity probability within 0.08 of the target. This is an example of a prespecified near-target rule, not a universal CRM stopping criterion. Under the equal-prior analysis with skeleton \((0.09,0.25,0.49)\), dose 2 has posterior mean 0.2614, which is 0.03864 from the target; the rule does not stop.

Worked Example: A Skeleton Can Change a Near-Target Stopping Decision

Use the same one-patient nontoxicity at dose 2, equal prior on \(\theta=0.5\) and \(1.5\), target 0.30, and stopping tolerance 0.08. Consider the more conservative skeleton \(s=(0.01,0.04,0.09)\). Its probability vectors are \((0.10,0.20,0.30)\) and \((0.001,0.008,0.027)\). The likelihoods for the nontoxicity at dose 2 are 0.80 and 0.992, giving posterior weights

\frac{0.80}{0.80+0.992}=\frac{25}{56} \quad\text{and}\quad \frac{31}{56}.

The resulting posterior means are

\begin{aligned} \bar p_1&=\frac{25(0.10)+31(0.001)}{56}=\frac{2.531}{56}=0.04520,\\ \bar p_2&=\frac{25(0.20)+31(0.008)}{56}=\frac{5.248}{56}=0.09371,\\ \bar p_3&=\frac{25(0.30)+31(0.027)}{56}=\frac{8.337}{56}=0.1489. \end{aligned}

The stopping band is \([0.22,0.38]\), since \(0.30\pm0.08\) gives those endpoints. All three posterior means are below 0.22, so this specification triggers the illustrative rule. The first skeleton did not trigger it. This does not establish that either skeleton is correct; it shows that the stopping decision depends on the assumed dose-toxicity scale and should be identified as such.

Conducting and Interpreting the Analysis

A useful report makes the comparison auditable. For each specification, record the prior and skeleton, posterior parameter weights or the method used to compute the posterior, posterior mean toxicity probability at each dose, distance from the target, selected dose, and stopping outcome. State which elements were held fixed. If both prior and skeleton are changed together, the result is a joint sensitivity analysis and cannot by itself identify which change caused a different decision.

Common diagnostics include the following:

Check posterior weights. Verify that likelihoods use the observed toxicity counts and nontoxicity counts correctly, and that normalized posterior weights sum to one.
Compare distances, not just dose labels. A recommendation may be technically unchanged while the winning dose is nearly tied with another dose. Small changes in a near-tie can reverse the ranking.
Track decision changes explicitly. Note whether a specification changes the selected dose, invokes a stopping rule, or alters eligibility under protocol safety restrictions.
Check plausibility before calculation. Candidate skeletons and priors should represent defensible clinical uncertainty, not be chosen after seeing which one yields a preferred decision.

Sensitivity analysis differs from a single-specification analysis, which reports the decision under one chosen model, and from Bayesian model averaging, which combines models using posterior model weights. Here each plausible specification is examined separately. A stable decision across the set supports robustness to those particular alternatives; a change flags dependence on assumptions. Neither conclusion covers alternatives that were not examined.

Common Mistakes and Limitations

Changing several inputs without saying so. If the prior, skeleton, target, and safety rule all change at once, a changed decision cannot be attributed to any one assumption. First vary one component at a time, then consider prespecified joint scenarios.
Using the skeleton probability as the posterior estimate. The skeleton is the dose-probability vector at \(\theta=1\); posterior dose estimates average \(s_j^\theta\) over the posterior distribution of \(\theta\). These quantities need not be equal.
Calling a robust result correct. Agreement across a small set of alternatives only shows stability within that set. It does not verify the CRM model, the skeleton range, or the prior's clinical adequacy.
Ignoring the stopping rule. A dose ranking alone does not capture whether a protocol would stop. Recalculate the rule's stated quantities under every specification.

The analysis also has limits: a finite set of alternatives cannot represent every plausible prior or skeleton, and early in a small dose-finding trial the data may be too sparse to overcome prior assumptions. Results depend on the chosen target, dose grid, likelihood, and decision rules. Report the examined alternatives and their clinical rationale so the scope of the robustness claim is clear.

Key takeaway: Recompute posterior dose estimates and protocol decisions under plausible priors and skeletons while holding other inputs fixed. A changed recommendation or stopping outcome is evidence of sensitivity to those assumptions, not a reason to choose the specification with the preferred result.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In the first worked example, what are the posterior odds for \(\theta=0.5\) versus \(\theta=1.5\) under the equal prior after the observed nontoxicity? Show the likelihood-ratio calculation.
Why is the likelihood contribution of a nontoxicity at a dose \(1-p_j(\theta)\), rather than \(p_j(\theta)\)?
For the second worked example, which dose is selected under the target-distance rule, and what is its posterior mean distance from the target?
With a stopping tolerance of 0.05, does the equal-prior analysis under skeleton \((0.09,0.25,0.49)\) stop under the illustrative near-target rule? What about skeleton \((0.04,0.16,0.36)\)?
Does an unchanged recommendation across several prior and skeleton specifications prove that the CRM model is correct? Explain.

Solutions. (1) The prior odds are \(1\). The likelihood ratio is \(0.50/0.875=4/7\), so the posterior odds are \(4/7\), matching posterior weights \(4/11\) and \(7/11\). (2) A nontoxicity is the Bernoulli outcome with probability \(1-p_j(\theta)\); \(p_j(\theta)\) is the probability of toxicity. (3) Dose 3 is selected: its posterior mean is 0.3660, whose distance from 0.30 is 0.06600; this is smaller than the distances at doses 1 and 2. (4) Under the first skeleton, dose 2 is 0.03864 from target, so the analysis does not stop. Under the second skeleton, the closest dose is dose 3 at distance 0.06600, which exceeds 0.05, so the stated rule stops. (5) No. It supports stability only across the alternatives examined and does not establish model correctness or cover unexamined assumptions.