Bayesian Dose Selection With a Logistic Toxicity Model

Why Model Toxicity Across Dose Levels?

In a first-in-human study, the decision problem is not simply to identify which observed dose had the fewest toxicities. Higher doses may be more active, but they may also cause more harm; moreover, each dose is often studied in only a small number of participants. A dose-toxicity model uses the information across dose levels to estimate the probability of toxicity at each dose, including levels with sparse observations.

This tutorial considers dose selection by a specified target toxicity rate. It does not define a complete escalation algorithm. The model estimates toxicity probabilities and supports a rule that selects the dose whose estimated probability is closest to the target. The target and the acceptable range around it must be chosen for the clinical setting before the model is used.

Definition: A dose-limiting toxicity (DLT) is a prespecified toxicity outcome assessed within a defined observation window. At dose level \(j\), let \(n_j\) be the number of evaluable participants and \(y_j\) the number who experience a DLT. The observed DLT proportion is \(y_j/n_j\); the modelled probability of DLT at that dose is \(p_j\).

Logistic Dose-toxicity Model

Let \(d_j\) denote the administered dose at level \(j\). A convenient dose covariate is the base-2 logarithm relative to a reference dose \(d_0\):

x_j=\log_2(d_j/d_0).

With this coding, each one-unit increase in \(x_j\) corresponds to doubling the dose. A logistic model relates the DLT probability to dose:

\operatorname{logit}(p_j)=\alpha+\beta x_j, \qquad p_j=\frac{\exp(\alpha+\beta x_j)}{1+\exp(\alpha+\beta x_j)}.

The intercept \(\alpha\) is the log-odds of DLT at the reference dose, and \(\beta\) is the change in log-odds for a doubling of dose. Requiring \(\beta>0\) makes the model’s DLT probability increase with dose. That monotonicity is a substantive modelling assumption, not a conclusion proven by the observed data.

Model assumptions: Conditional on the dose, each participant has a binary DLT outcome within the same prespecified assessment window; outcomes are independent; participants at a dose share the same DLT probability; and the log-odds of DLT are linear in the chosen dose covariate. The model also assumes the dose-toxicity relationship is increasing when the prior restricts \(\beta>0\). Incomplete DLT follow-up, substantial dependence between participants, or a sharply non-linear dose relationship can undermine these assumptions.

For a Bayesian analysis, specify a prior distribution for \(\alpha\) and \(\beta\) before observing the DLT outcomes. For illustration, use independent priors

\alpha\sim N(-2,1.5^2), \qquad \beta\sim \text{half-}N(0,1.5^2).

The half-normal prior is a normal distribution restricted to positive values, so it encodes increasing toxicity while allowing a range of slopes. The prior on \(\alpha\) places substantial probability on relatively low DLT probabilities at the reference dose, but it is not a universal choice. Prior predictive checks should assess whether the implied probabilities over the planned dose range are clinically plausible.

Likelihood and Posterior

Assume the DLT counts at different dose levels are conditionally independent. Given \(\alpha\) and \(\beta\), the count at dose \(j\) follows a binomial distribution with size \(n_j\) and probability \(p_j\). The likelihood, ignoring no data-dependent terms, is

L(\alpha,\beta) = \prod_{j=1}^{J} {n_j\choose y_j}p_j^{y_j}(1-p_j)^{n_j-y_j}.

Bayes’ rule combines this likelihood with the prior density \(\pi(\alpha,\beta)\). The posterior density is proportional to their product:

\pi(\alpha,\beta\mid \mathbf{y}) \propto L(\alpha,\beta)\pi(\alpha,\beta), \qquad \beta>0.

The model is not conjugate: the posterior does not have a simple named distribution obtained by substituting updated parameters into the prior. Posterior summaries are therefore usually computed by numerical integration or a validated simulation method. Each posterior draw \((\alpha^{(m)},\beta^{(m)})\) gives a probability at each dose:

p_j^{(m)} = \frac{\exp(\alpha^{(m)}+\beta^{(m)}x_j)} {1+\exp(\alpha^{(m)}+\beta^{(m)}x_j)}.

The posterior mean of \(p_j\) is the average of these draw-specific probabilities. Posterior quantiles provide a credible interval for the DLT probability at that dose. Applying the logistic function to the posterior mean of \(\alpha+\beta x_j\) is generally not the same as averaging the draw-specific probabilities, because the logistic function is nonlinear.

Worked Examples

Worked Example: Building the Likelihood From Escalation Data

Consider hypothetical data from four dose levels: 1, 2, 4, and 8 mg/m². Six participants were evaluable at each level, with DLT counts 0, 1, 2, and 4. DLT status was assessed over the same planned window for all 24 participants. Set \(d_0=1\) mg/m², so the corresponding covariates are \(x=0,1,2,3\).

For a concrete likelihood calculation, take \(\alpha=-2.2\) and \(\beta=1.0\). These are parameter values at which to evaluate the likelihood, not fitted estimates. At \(x=1\), for example,

p_2 = \frac{\exp(-2.2+1.0)}{1+\exp(-2.2+1.0)} = \frac{\exp(-1.2)}{1+\exp(-1.2)} \approx 0.2315.

At \(x=0,1,2,3\), the probabilities are approximately \(0.0998, 0.2315, 0.4502,\) and \(0.6900\). Substituting the counts into the binomial likelihood gives

\begin{aligned} L(-2.2,1.0) ={}& {6\choose0}(0.0998)^0(0.9002)^6 {6\choose1}(0.2315)^1(0.7685)^5\\ &\times {6\choose2}(0.4502)^2(0.5498)^4 {6\choose4}(0.6900)^4(0.3100)^2\\ \approx{}& 0.0180. \end{aligned}

The likelihood value measures how compatible these counts are with this particular parameter pair, relative to other pairs under the same data model. It is not itself a posterior probability. The posterior also incorporates the prior density and is normalized across the possible parameter values.

Worked Example: Estimating Probabilities and Selecting a Dose

Use the same hypothetical data and priors. Suppose numerical integration of the posterior gives the following rounded summaries. The intervals are 95% equal-tail credible intervals, meaning that 2.5% of the posterior probability for each \(p_j\) lies below the lower endpoint and 2.5% lies above the upper endpoint.

Dose (mg/m²)	DLTs / participants	Posterior mean of \(p_j\)	95% credible interval
1	0 / 6	0.09	0.01 to 0.25
2	1 / 6	0.20	0.04 to 0.41
4	2 / 6	0.36	0.12 to 0.62
8	4 / 6	0.59	0.32 to 0.82

Suppose the prespecified target DLT rate is \(\tau=0.25\). Under the rule used here, select the dose minimizing the absolute difference between its posterior mean DLT probability and the target:

j^*=\arg\min_j\left|E(p_j\mid\mathbf{y})-\tau\right|.

The absolute differences at doses 1, 2, 4, and 8 mg/m² are \( |0.09-0.25|=0.16\), \( |0.20-0.25|=0.05\), \( |0.36-0.25|=0.11\), and \( |0.59-0.25|=0.34\). Thus, this rule selects 2 mg/m². The estimate at that dose is 0.20, or 20%, which is 5 percentage points below the 25% target. Its interval is wide, so the point-estimate ranking should not be mistaken for certainty about the true DLT probability.

Worked Example: Checking Sensitivity to the Prior

A small escalation data set can leave the dose-response slope uncertain, making the result sensitive to the prior. As a sensitivity analysis for the same hypothetical counts, suppose the analysis instead uses \(\alpha\sim N(-1.5,1.5^2)\) and \(\beta\sim\text{half-}N(0,1.0^2)\). Suppose a second numerical integration gives posterior means of \(0.11, 0.22, 0.37,\) and \(0.57\) at 1, 2, 4, and 8 mg/m².

For the same 25% target, the distances are \(0.14, 0.03, 0.12,\) and \(0.32\), respectively. The rule again selects 2 mg/m². The agreement is reassuring about this particular ranking under these two priors, but it does not establish that the model or priors are correct. A materially different ranking, especially one driven by a large change in the prior rather than the data, would need careful review before relying on the selection.

Why the Posterior Calculation Works

The likelihood contributes evidence about the parameters through the observed DLT counts, while the prior contributes information specified before observing those counts. The posterior mean toxicity at a dose is an average over uncertainty about both the intercept and slope. Its computation can be justified directly from the definition of conditional expectation.

Obtain the posterior density.
Multiply the binomial likelihood by the joint prior and divide by the integral of that product over all allowed \((\alpha,\beta)\), with \(\beta>0\).

Transform each parameter pair to a dose probability.
At dose \(j\), use \(g_j(\alpha,\beta)=\operatorname{logit}^{-1}(\alpha+\beta x_j)\).

Average over posterior uncertainty.
The posterior mean is the integral of \(g_j(\alpha,\beta)\) against the posterior density. With posterior draws, approximate that integral by the arithmetic mean of the draw-specific \(g_j\) values.

In symbols, the quantity being estimated is

E(p_j\mid\mathbf{y}) = \int \frac{\exp(\alpha+\beta x_j)} {1+\exp(\alpha+\beta x_j)} \,\pi(\alpha,\beta\mid\mathbf{y})\,d\alpha\,d\beta.

For \(M\) posterior draws, the corresponding Monte Carlo approximation is

\widehat{E}(p_j\mid\mathbf{y}) = \frac{1}{M}\sum_{m=1}^{M} \frac{\exp(\alpha^{(m)}+\beta^{(m)}x_j)} {1+\exp(\alpha^{(m)}+\beta^{(m)}x_j)}.

This calculation averages probabilities, not odds or log-odds. In implementation, numerical integration should be accurate enough to stabilize the displayed summaries and dose ranking; simulation-based calculations should also be checked for adequate mixing and effective sample size.

Diagnostics, Alternatives, and Limitations

A smooth increasing curve is useful when toxicity is expected to increase with dose, but it can conceal a poor fit. Review the observed counts alongside the fitted posterior means, and compare observed DLT counts with posterior predictive counts generated under the fitted model. If the model repeatedly predicts patterns unlike the observed counts, the linear log-odds assumption may be too restrictive. A single dose with unexpectedly many or few DLTs can be especially influential when cohort sizes are small.

Prior predictive checks are also important: simulate toxicity probabilities at all planned doses from the prior before using the data. If the prior frequently implies near-zero or near-certain toxicity across the entire dose range, or implausibly steep changes between adjacent doses, revise the prior with clinical input and document the rationale. After fitting, compare posterior estimates under reasonable alternative priors, as in the sensitivity example.

The logistic curve borrows information across dose levels and imposes a smooth monotone shape. A separate beta-binomial model at each dose avoids a linear dose-response assumption, but does not naturally borrow information across levels and may be highly uncertain with small cohorts. A non-Bayesian logistic fit can estimate a similar curve, but does not by itself provide posterior probability summaries. The choice among approaches depends on the scientific assumptions and the decision context; no model can compensate for inadequate DLT follow-up or an outcome definition that changes across doses.

The closest-posterior-mean rule is a particular selection criterion, not a guarantee that the chosen dose is safe or optimal for efficacy. It ignores differences in posterior uncertainty when ranking doses: two doses can have similar posterior means but very different credible intervals. A decision rule that explicitly penalizes uncertainty or the consequences of excessive toxicity would need to specify those losses separately. This tutorial reports uncertainty alongside the selected dose rather than treating the point estimate as a safety certificate.

Common pitfall: “The observed DLT proportion at a dose is closest to the target, so that dose must be selected.” This reasoning ignores the information shared across dose levels, posterior uncertainty, and the stated selection rule. Instead, fit the prespecified model, calculate the posterior toxicity estimate at every dose, report uncertainty, and apply the defined criterion consistently. Also, a 95% credible interval is a posterior statement conditional on the model and prior; it is not a guarantee that the true probability lies inside the interval.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

At dose \(d=4\) mg/m² with reference \(d_0=1\) mg/m², what is \(x\)? If \(\alpha=-2\) and \(\beta=0.8\), calculate the modelled DLT probability.
For a target rate of 0.30, posterior mean estimates at three doses are 0.18, 0.34, and 0.55. Which dose is selected by the closest-posterior-mean rule, and what is its distance from target?
Why does a positive-slope restriction encode an assumption rather than demonstrate from the data that toxicity must increase with dose?
Name one diagnostic that can reveal a poor dose-response fit and one limitation that remains even if the posterior computation is accurate.

Solutions. (1) \(x=\log_2(4/1)=2\). The linear predictor is \(-2+0.8(2)=-0.4\), so \(p=1/(1+\exp(0.4))\approx0.4013\). (2) The distances are \(0.12,0.04,\) and \(0.25\), so select the second dose; its posterior mean is 0.04 above target. (3) The restriction is part of the model and prior specification. If the data are sparse, they may not determine the slope’s sign; the restriction enforces monotonicity rather than testing it. (4) One can compare observed counts with posterior predictive counts. Accurate computation does not remove limitations such as an incorrect linear log-odds shape, incomplete DLT ascertainment, or uncertainty about whether the target-based rule reflects the full clinical decision.