Posterior Probability of Exceeding the Toxicity Limit

What Is the Posterior Safety Probability?

A dose has an unknown toxicity probability. Even after observing outcomes, uncertainty about that probability remains. A posterior probability of exceeding a toxicity limit quantifies how much posterior support lies on the side of the limit considered unsafe. It is a probability about an unknown dose-specific rate, not the probability that a particular patient will experience toxicity.

Let $p_j(\theta)$ denote the probability of dose-limiting toxicity (DLT) at dose $j$, under model parameter $\theta$. Let $\tau$ be a prespecified toxicity limit. Given observed data $D$, the event of interest is $p_j(\theta)>\tau$, and its posterior probability is computed using the distribution of $\theta$ after updating with $D$.

Definition: The posterior probability that dose $j$ exceeds the toxicity limit $\tau$ is $$ \Pr\{p_j(\theta)>\tau\mid D\} =\int I\{p_j(\theta)>\tau\}\,\pi(\theta\mid D)\,d\theta, $$ where $\pi(\theta\mid D)$ is the posterior distribution of the model parameter and $I\{\cdot\}$ equals 1 when its condition is true and 0 otherwise.

The probability depends on the dose, the chosen limit, the toxicity model, the prior, and the observed data. It is not calculated by comparing only a point estimate of $p_j$ with $\tau$: posterior uncertainty matters.

Model and Calculation

For patients with fully observed binary DLT outcomes, write the data as $D=\{(d_i,y_i):i=1,\ldots,n\}$, where $d_i$ is the dose received and $y_i=1$ indicates a DLT within the defined assessment window; otherwise $y_i=0$. A standard CRM analysis specifies dose-specific probabilities $p_{d_i}(\theta)$, a prior distribution for $\theta$, and a likelihood

L(\theta\mid D) =\prod_{i=1}^{n}p_{d_i}(\theta)^{y_i} \{1-p_{d_i}(\theta)\}^{1-y_i}.

This likelihood assumes that, conditional on $\theta$ and the assigned doses, patient outcomes are independent Bernoulli observations with the modelled DLT probabilities. The posterior is proportional to the likelihood times the prior. A CRM may also impose a monotone dose-toxicity relationship; if so, the probabilities and the event are evaluated under that specified model. The calculation does not make the model correct: it summarizes uncertainty conditional on the model and prior.

If the posterior is represented by weighted parameter values $\theta^{(m)}$ with normalized weights $w_m$, calculate the probability by adding the weights for which the dose-specific probability exceeds the limit. With posterior draws, use the fraction of draws satisfying the event. These are numerical versions of the integral in the definition.

Key distinction: A target rate $\gamma$ expresses the desired toxicity rate for dose finding; a safety limit $\tau$ defines the rate whose exceedance is being assessed. The events $p_j(\theta)>\gamma$ and $p_j(\theta)>\tau$ need not be the same. Their posterior probabilities differ whenever the thresholds differ and there is posterior mass between them.

Worked Example: Summing Posterior Mass Above a Limit

Suppose a hypothetical one-parameter dose-toxicity model is $p_j(\theta)=1-\exp(-c_j\theta)$, with $c_j=2$ at a proposed dose. The parameter has three possible values, $\theta=0.1,0.3,0.6$, with prior probability $1/3$ each. Three patients were treated at this dose, and one had a DLT. Assume the outcomes are conditionally independent Bernoulli observations under the model.

The model gives dose-specific probabilities $1-\exp(-0.2)=0.1813$, $1-\exp(-0.6)=0.4512$, and $1-\exp(-1.2)=0.6988$, respectively. For one DLT among three patients, the likelihood at a probability $p$ is $p(1-p)^2$. Thus the three likelihoods are approximately $0.1215$, $0.1359$, and $0.06339$. Because the prior probabilities are equal, normalizing these likelihoods gives posterior weights:

(0.1215,\;0.1359,\;0.06339)/0.3208 \approx (0.3787,\;0.4236,\;0.1976).

For a prespecified safety limit $\tau=0.60$, only the third parameter value gives $p_j(\theta)>\tau$, since $0.6988>0.60$ while $0.1813$ and $0.4512$ are below it. The posterior probability of exceeding the limit is therefore $0.1976$, or about 19.8%.

Now suppose the target rate is $\gamma=0.30$. The second and third parameter values exceed this target, so the posterior probability of being above target is $0.4236+0.1976=0.6212$, or about 62.1%. The target-rate event is much more probable than the safety-limit event because the thresholds differ. Neither result is the probability that a patient will have a DLT.

Why the Integral Has This Form

The event probability is the posterior expectation of an indicator. This follows directly by partitioning the parameter space into values that do and do not satisfy the event. The indicator is 1 on the first set and 0 on the second, so its posterior expectation is exactly the posterior mass of the first set:

\begin{aligned} E[I\{p_j(\theta)>\tau\}\mid D] &=\int I\{p_j(\theta)>\tau\}\pi(\theta\mid D)\,d\theta\\ &=\int_{\{\theta:p_j(\theta)>\tau\}}\pi(\theta\mid D)\,d\theta\\ &=\Pr\{p_j(\theta)>\tau\mid D\}. \end{aligned}

If posterior draws $\theta^{(1)},\ldots,\theta^{(M)}$ are available, this expectation is estimated by

\widehat{q}_j =\frac{1}{M}\sum_{m=1}^{M}I\{p_j(\theta^{(m)})>\tau\}.

For independent draws, the approximate Monte Carlo standard error is $\sqrt{\widehat{q}_j(1-\widehat{q}_j)/M}$. For correlated simulation draws, this expression does not account for autocorrelation; effective sample size or another suitable simulation diagnostic is needed.

Worked Example: Estimating the Probability From Posterior Draws

Suppose a hypothetical posterior simulation produces $M=1{,}000$ draws for a proposed dose. For each draw, the model gives a value of $p_j(\theta)$. In 273 draws, this value is above the prespecified limit $\tau=0.30$. Assume the draws are effectively independent for this calculation.

The estimated posterior probability is the proportion of draws above the limit:

\widehat{q}_j=\frac{273}{1{,}000}=0.273.

The approximate Monte Carlo standard error is

\sqrt{\frac{0.273(1-0.273)}{1{,}000}} =\sqrt{0.0001985} \approx 0.0141.

Thus the estimated posterior probability is 27.3%, with simulation error on the order of 1.4 percentage points in standard-error units. This quantifies numerical error in estimating the posterior probability; it is not a confidence interval for the true toxicity probability. If the draws are autocorrelated, the independent-draw calculation can understate this error.

Model Averaging and Other Quantities

If the CRM analysis averages over alternative skeletons or models, the posterior event probability must include uncertainty about which model applies. Let $M_k$ denote model $k$, with posterior model weight $w_k=\Pr(M_k\mid D)$. The overall event probability is the weighted average of the model-specific event probabilities:

\Pr\{p_j(\theta)>\tau\mid D\} =\sum_k w_k\Pr\{p_j(\theta)>\tau\mid D,M_k\}.

This averages event probabilities, not toxicity probabilities. Averaging the dose-specific rates first and then checking whether the average exceeds $\tau$ generally answers a different question.

Worked Example: Combining Two CRM Models

Consider a hypothetical model-averaged analysis with two CRM skeletons. Their posterior model weights are $0.65$ and $0.35$. At a proposed dose, the posterior probability of exceeding $\tau=0.30$ is $0.12$ under the first model and $0.40$ under the second.

The model-averaged probability is

$$ 0.65(0.12)+0.35(0.40) =0.078+0.140 =0.218. $$

The posterior probability of exceeding the limit is therefore 21.8%. It would not be correct to average the two model-specific toxicity rates and compare that single rate with 0.30: that calculation does not in general equal the posterior probability of the event.

Interpretation, Diagnostics, and Limitations

A posterior probability of 0.20 means that, under the specified model, prior, and observed data, 20% of the posterior distribution of the dose-specific toxicity probability lies above the chosen limit. It does not mean there is a 20% probability that the limit itself is correct, nor does it mean that 20% of patients at the dose will experience toxicity. The latter quantity concerns a future outcome and is based on the posterior predictive probability $E[p_j(\theta)\mid D]$, which is not the same posterior event probability.

Before interpreting a calculation, verify that the event uses the intended dose and threshold, and that the DLT definition and assessment window match those used in the model. Check that the likelihood includes the outcomes and dose assignments correctly, and that the posterior calculation uses normalized weights or draws from the intended posterior. For a monotone dose model, confirm the direction of the inequality: a higher model parameter may increase or decrease $p_j(\theta)$, depending on the parameterization. The event should be checked using $p_j(\theta)$ itself, not inferred from the parameter’s numerical ordering without confirming that direction.

Numerical diagnostics matter as well. With weighted calculations, confirm that posterior weights sum to 1 and that every value exactly at the limit is handled consistently with the strict event $>\tau$. With simulation, inspect mixing and effective sample size, and report enough draws to make Monte Carlo error acceptably small for the intended interpretation. A small apparent event probability based on few effective draws may be numerically unstable.

The result is conditional on the chosen dose-toxicity model and prior. A misspecified model, poorly supported dose range, or sparse data can leave substantial uncertainty unrepresented by the calculation. Posterior probability also does not by itself specify a dose-selection rule: the clinical safety limit and any decision threshold applied to this probability are separate design choices.

Quantity	What it describes	Why it differs
$\Pr\{p_j(\theta)>\tau\mid D\}$	Posterior mass where the dose-specific rate exceeds the safety limit	The event is a threshold statement about an unknown rate
$\Pr\{p_j(\theta)>\gamma\mid D\}$	Posterior mass where the rate exceeds the target	It uses the target rate, which may differ from the safety limit
$E[p_j(\theta)\mid D]$	Posterior mean toxicity probability at the dose	It averages rates rather than calculating the probability of a threshold event
$E[p_j(\theta)\mid D]$ for a future patient	Posterior predictive DLT probability for one patient, under the model	It concerns a future outcome, not whether the unknown rate exceeds a limit

A common mistake is to say, “The posterior mean rate is below the safety limit, so the dose has a low probability of exceeding it.” A mean alone does not determine the mass above a threshold: different posterior distributions can have the same mean but very different upper-tail probabilities. Calculate the event probability directly from the posterior distribution.

Check Your Understanding

Answer each question, then compare with the brief solutions.

A posterior distribution assigns probability 0.25 to $p_j=0.18$, 0.50 to $p_j=0.32$, and 0.25 to $p_j=0.55$. For $\tau=0.30$, what is the posterior probability of exceeding the limit?
If the target rate is $\gamma=0.20$ in the same distribution, what is the posterior probability of being above target? Why is it not the same event as exceeding $\tau=0.30$?
In 600 effectively independent posterior draws, 90 dose-specific rates exceed the safety limit. Calculate the estimated probability and its approximate Monte Carlo standard error.
Two models have posterior weights 0.8 and 0.2, with event probabilities 0.10 and 0.50. Calculate the model-averaged event probability.

Solutions. (1) The values above 0.30 are 0.32 and 0.55, so the probability is $0.50+0.25=0.75$. (2) All three values exceed 0.20, so the probability is 1.00. The threshold differs, so the event differs. (3) The estimate is $90/600=0.15$; its approximate standard error is $\sqrt{0.15(0.85)/600}=0.0146$. (4) The weighted probability is $0.8(0.10)+0.2(0.50)=0.18$, or 18%.

Quantity	What it describes	Why it differs
\(\Pr\{p_j(\theta)>\tau\mid D\}\)	Posterior mass where the dose-specific rate exceeds the safety limit	The event is a threshold statement about an unknown rate
\(\Pr\{p_j(\theta)>\gamma\mid D\}\)	Posterior mass where the rate exceeds the target	It uses the target rate, which may differ from the safety limit
\(E[p_j(\theta)\mid D]\)	Posterior mean toxicity probability at the dose	It averages rates rather than calculating the probability of a threshold event
\(E[p_j(\theta)\mid D]\) for a future patient	Posterior predictive DLT probability for one patient, under the model	It concerns a future outcome, not whether the unknown rate exceeds a limit