Tutorials › Adaptive Clinical Trial Design › Deriving Constrained Dose Selection Rules

Phase 1/2 Designs With Efficacy and Toxicity · Tutorial 626 of 799

Deriving Constrained Dose Selection Rules

Learn how to define the toxicity-feasible doses, select the one with greatest posterior mean efficacy, and handle uncertainty and ties transparently.

Advanced 12 min read

What You'll Learn

  • Define a finite-dose Bayesian efficacy objective and posterior toxicity constraint
  • Derive the constrained selection rule from its feasible set
  • Calculate posterior safety probabilities using beta-binomial models
  • Explain why a posterior mean toxicity threshold does not express the same uncertainty requirement
  • Apply a prespecified tie-breaking rule when efficacy criteria are equal
  • Diagnose empty feasible sets and sensitivity to model assumptions

From a Safety Requirement to a Dose Rule

A dose can have the highest estimated efficacy and still be an unsuitable recommendation if its toxicity is too likely to exceed an acceptable level. A constrained selection rule makes the tradeoff explicit: first require adequate posterior evidence that a dose meets the toxicity standard, then choose the most efficacious dose among those that pass. The rule is a decision procedure based on a specified model, toxicity threshold, and evidential standard; it does not make the safety requirement disappear when the data are uncertain.

Let \(d_1,\ldots,d_K\) be the prespecified candidate doses. For a patient treated at dose \(d_j\), let \(p_{E,j}\) and \(p_{T,j}\) denote the underlying probabilities of the defined efficacy and toxicity outcomes over their specified assessment windows. Write \(\mathcal D\) for the observed data and \(\theta_T\) for the largest acceptable toxicity probability. The posterior safety probability for dose \(d_j\) is \(q_j=P(p_{T,j}\leq\theta_T\mid\mathcal D)\). A design specifies a minimum value \(\gamma\), between zero and one, that this probability must meet.

Definition: The toxicity-feasible set is \(\mathcal A=\{d_j:q_j\geq\gamma\}\). For a nonempty \(\mathcal A\), the constrained Bayesian dose rule selects a dose maximizing posterior mean efficacy: \(\hat d\in\arg\max_{d_j\in\mathcal A} E(p_{E,j}\mid\mathcal D)\). Here \(\arg\max\) denotes the set of doses attaining the largest value. If \(\mathcal A\) is empty, this rule makes no dose recommendation; a separate prespecified action, such as stopping or obtaining more information, is needed.

The parameter \(\theta_T\) states what toxicity rate is considered acceptable, while \(\gamma\) states how much posterior support is required that a candidate’s true rate is at or below that limit. These are distinct design choices. For example, \(\theta_T=0.30\) and \(\gamma=0.80\) require at least an 80% posterior probability that a dose’s toxicity probability is no greater than 30%. They do not assert that the dose has an 80% chance of being safe in a future patient; the probability concerns the unknown model parameter, conditional on the data and model.

Why the Rule Has This Form

Suppose the decision maker’s reward for selecting dose \(d_j\) is the probability of efficacy \(p_{E,j}\), and the toxicity requirement is a hard constraint rather than a penalty traded against benefit. The posterior expected reward for that decision is \(E(p_{E,j}\mid\mathcal D)\). The rule follows by restricting the available decisions to those that satisfy the posterior safety requirement, then maximizing expected reward over the remaining finite set.

$$ \begin{aligned} \mathcal A &=\{d_j:P(p_{T,j}\leq\theta_T\mid\mathcal D)\geq\gamma\},\\ \hat d &\in\arg\max_{d_j\in\mathcal A}E(p_{E,j}\mid\mathcal D). \end{aligned} $$

This is a constrained optimization rule, not a single combined efficacy-toxicity score. It uses posterior uncertainty in toxicity through \(q_j\), and posterior uncertainty in efficacy through its expectation. If two doses have equal posterior mean efficacy, both are maximizers unless a tie-breaking convention is added. The rule uses the marginal posterior distributions of efficacy and toxicity; it does not require those parameters to be posterior-independent. However, the model must produce valid posterior distributions for both, and any dependence represented by the model should be retained when calculating them.

For a simple derivation of the posterior mean in a common model, suppose the efficacy outcomes at one dose are Bernoulli with probability \(p_E\), and use a \(\operatorname{Beta}(a,b)\) prior. With \(x\) responses among \(n\) patients, the likelihood is proportional to \(p_E^x(1-p_E)^{n-x}\). Multiplying by the prior density gives a \(\operatorname{Beta}(a+x,b+n-x)\) posterior. The mean of a \(\operatorname{Beta}(u,v)\) distribution is \(u/(u+v)\), so the posterior mean efficacy is \((a+x)/(a+b+n)\). This posterior mean, rather than the observed proportion alone, is the efficacy criterion in the rule.

Worked Example: Screening Doses and Maximizing Efficacy

Consider three hypothetical doses. Use independent uniform \(\operatorname{Beta}(1,1)\) priors for each dose’s efficacy and toxicity probability, and binomial models for the two marginal endpoints. For this illustration, the endpoint likelihoods are modeled separately. The hypothetical data are: efficacy responses of 4/10, 7/10, and 9/10 patients at doses 1, 2, and 3; toxicities of 0/5, 1/8, and 2/5 patients, respectively. Set \(\theta_T=0.30\) and \(\gamma=0.80\).

After \(x\) events in \(n\) observations, the uniform-prior posterior is \(\operatorname{Beta}(x+1,n-x+1)\). Thus the efficacy posteriors are \(\operatorname{Beta}(5,7)\), \(\operatorname{Beta}(8,4)\), and \(\operatorname{Beta}(10,2)\). Their posterior means are \(5/12=0.4167\), \(8/12=0.6667\), and \(10/12=0.8333\), respectively.

The toxicity posteriors are \(\operatorname{Beta}(1,6)\), \(\operatorname{Beta}(2,8)\), and \(\operatorname{Beta}(3,4)\). For dose 1, \(q_1=P(p_{T,1}\leq0.30\mid\mathcal D)=1-(1-0.30)^6=1-0.117649=0.882351\). For dose 2, the \(\operatorname{Beta}(2,8)\) cumulative distribution function at \(x\) is \(1-(1-x)^8(1+8x)\), giving \(q_2=1-(0.70)^8(1+8(0.30))=1-(0.05764801)(3.4)=0.803996766\).

For dose 3, integrating the \(\operatorname{Beta}(3,4)\) density gives \(P(p_{T,3}\leq x\mid\mathcal D)=20x^3-45x^4+36x^5-10x^6\). At \(x=0.30\), this is \(0.540000-0.364500+0.087480-0.007290=0.255690\). Therefore doses 1 and 2 pass the 0.80 requirement, while dose 3 does not. Among the feasible doses, dose 2 has the greater posterior mean efficacy, \(0.6667\) versus \(0.4167\), so the rule selects dose 2. Dose 3’s larger efficacy mean cannot compensate for failing the stated toxicity constraint.

Posterior Uncertainty Is Part of the Constraint

The posterior safety probability is not interchangeable with comparing the posterior mean toxicity to \(\theta_T\). A posterior mean describes the center of a distribution; \(q_j\) also reflects how much posterior mass lies on either side of the toxicity limit. Two doses with the same posterior mean toxicity can have different safety probabilities if their posterior uncertainty differs. Conversely, a mean at or below the limit does not guarantee that a specified fraction of the posterior distribution is below it.

Worked Example: A Mean at the Limit Can Still Fail

Suppose a hypothetical dose has 2 toxicities among 8 patients and a uniform \(\operatorname{Beta}(1,1)\) prior. Its toxicity posterior is \(\operatorname{Beta}(3,7)\), with posterior mean \(3/(3+7)=0.30\). Let the acceptable limit be \(\theta_T=0.30\) and the required posterior probability be \(\gamma=0.80\).

For integer beta parameters, \(P(p_T\leq x)\) under a \(\operatorname{Beta}(a,b)\) distribution equals \(P(Z\geq a)\) for \(Z\sim\operatorname{Binomial}(a+b-1,x)\). Here, \(q=P(p_T\leq0.30\mid\mathcal D)=1-P(Z\leq2)\), with \(Z\sim\operatorname{Binomial}(9,0.30)\).

The lower-tail probability is \(P(Z\leq2)=(0.70)^9+9(0.30)(0.70)^8+36(0.30)^2(0.70)^7\) \(=0.040353607+0.155649627+0.266827932=0.462831166\). Thus \(q=1-0.462831166=0.537168834\), approximately 0.5372. The dose fails the 0.80 requirement even though its posterior mean toxicity equals the limit. The mean-only comparison would conceal how much posterior probability lies above that limit.

Posterior probabilities can be evaluated from an analytic posterior distribution or estimated from posterior simulation draws. With \(M\) draws \(p_{T,j}^{(1)},\ldots,p_{T,j}^{(M)}\), the Monte Carlo estimate is \(\hat q_j=M^{-1}\sum_{m=1}^M I(p_{T,j}^{(m)}\leq\theta_T)\), where \(I(\cdot)\) equals 1 when its condition is true and 0 otherwise. A value near \(\gamma\) warrants enough simulation draws to determine whether Monte Carlo error could change eligibility. This numerical uncertainty is additional to the posterior uncertainty about toxicity itself.

Ties, Diagnostics, and Alternatives

If multiple feasible doses attain the same posterior mean efficacy, the mathematical rule returns all of them. A single recommendation requires a tie-breaking policy stated before applying the rule. Possible policies include selecting the lower dose or, among efficacy ties, selecting the dose with the lower posterior mean toxicity. Such a policy is an additional decision convention, not a consequence of the efficacy maximization itself. If estimates are rounded for display, apparent ties may be artifacts; comparisons should use unrounded values and a prespecified numerical tolerance.

Worked Example: Applying a Tie-Breaking Convention

Two hypothetical doses each have 5 efficacy responses among 8 patients and uniform beta priors. Both therefore have efficacy posterior \(\operatorname{Beta}(6,4)\), with posterior mean \(6/10=0.60\). Dose A has 0 toxicities among 8 patients, giving a \(\operatorname{Beta}(1,9)\) toxicity posterior. Dose B has 0 among 3, giving \(\operatorname{Beta}(1,4)\). Use \(\theta_T=0.35\), \(\gamma=0.80\), and the prespecified tie-break of choosing the lower posterior mean toxicity.

For a \(\operatorname{Beta}(1,b)\) distribution, \(P(p_T\leq x)=1-(1-x)^b\). Thus dose A has safety probability \(1-(0.65)^9=0.979288\), and dose B has \(1-(0.65)^4=0.821494\). Both pass the 0.80 constraint. Their efficacy means are exactly equal, so the efficacy objective alone cannot distinguish them. Their toxicity posterior means are \(1/10=0.10\) and \(1/5=0.20\); the stated tie-break selects dose A. This selection reflects the declared convention, not a greater estimated efficacy for A.

Before using the rule, verify that the toxicity endpoint, observation window, threshold, and candidate doses match the intended clinical decision. Check posterior predictive or model-fit diagnostics appropriate to the dose-response model, and assess whether conclusions are sensitive to plausible prior and model choices. For simulation-based posteriors, check that estimates of \(q_j\) are stable and that enough effective draws were used. A dose just above or below \(\gamma\) may change eligibility with small changes in assumptions or data, so report the probability itself rather than only a pass/fail label.

The rule differs from two common alternatives. A plug-in rule compares a point estimate, often a posterior mean toxicity, with \(\theta_T\); it does not directly enforce a posterior probability requirement. A rule that maximizes a weighted efficacy-toxicity score allows a specified penalty for toxicity to trade against efficacy, whereas the constrained rule excludes doses that fail the threshold regardless of their efficacy mean. The constrained rule is transparent, but its result can change sharply around the eligibility boundary. If no dose is feasible, choosing the dose with the highest efficacy anyway would violate the rule; the design must instead specify what happens when \(\mathcal A\) is empty.

The output is conditional on the candidate set, outcome definitions, prior, likelihood, \(\theta_T\), and \(\gamma\). It is not a guarantee that the selected dose’s true toxicity rate is acceptable, nor does it establish that efficacy is clinically worthwhile. In particular, maximizing posterior mean efficacy among feasible doses can favor a dose with only a small efficacy advantage if that advantage is estimated imprecisely; the objective is the posterior mean, not a requirement for a particular efficacy probability. These limitations should be visible when interpreting a recommendation.

Key takeaway: Calculate posterior safety probabilities first, retain only doses with \(P(p_{T,j}\leq\theta_T\mid\mathcal D)\geq\gamma\), and then maximize posterior mean efficacy within that feasible set. State a separate action for an empty set and a prespecified convention for ties.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

  1. A dose has posterior toxicity distribution \(\operatorname{Beta}(1,5)\), \(\theta_T=0.20\), and \(\gamma=0.70\). Does it pass the constraint?
  2. Why does a posterior mean toxicity equal to \(\theta_T\) not, by itself, establish eligibility under a posterior probability constraint?
  3. Two feasible doses have posterior mean efficacies 0.58 and 0.58. What does the constrained rule select, and what must be added to select one dose?
  4. All candidate doses have \(q_j<\gamma\). What conclusion follows from the rule?

Solutions. (1) For \(\operatorname{Beta}(1,5)\), \(q=1-(1-0.20)^5=1-0.32768=0.67232\), which is below 0.70, so it fails. (2) The posterior mean is the distribution’s center, whereas eligibility depends on the posterior mass at or below the limit; that mass can be less than \(\gamma\). (3) Both doses maximize the stated objective, so both are in the argmax; a prespecified tie-break, such as choosing the lower dose, is needed for a single selection. (4) The feasible set is empty, so the rule makes no dose recommendation; a separate design action must be specified.

Clinical Biostats