Tutorials › Pharmacokinetic-Pharmacodynamic (PK/PD) Modeling › Selecting Doses When Toxicity Constrains Efficacy

Dose-Response and Dose Selection · Tutorial 550 of 598

Selecting Doses When Toxicity Constrains Efficacy

Apply toxicity constraints to identify feasible doses, select the most efficacious feasible option, and quantify the benefit forgone relative to an unconstrained choice.

Advanced 11 min read

What You'll Learn

  • Define a dose-selection problem using efficacy and explicit toxicity limits.
  • Distinguish a point-estimate constraint from one based on an upper confidence bound.
  • Identify the optimal dose by comparing efficacy only among feasible options.
  • Quantify the efficacy sacrificed relative to the unconstrained efficacy-maximizing dose.
  • Handle multiple toxicity constraints and nonmonotonic efficacy patterns.
  • Recognize assumptions and diagnostics that can make a safety constraint unreliable.

Why Put Toxicity in the Constraint?

When efficacy increases with dose, choosing the dose with the largest estimated effect can be inappropriate if toxicity also rises. A toxicity constraint makes the safety requirement explicit: first rule out doses that exceed the specified limit, then compare efficacy among those that remain. This differs from allowing a large efficacy benefit to compensate mathematically for an unacceptable toxicity risk.

The constraint is only as meaningful as its definition. Before comparing doses, specify the toxicity outcome, the population and time period over which it is measured, and the acceptable limit. For example, “no more than a 10% risk” is incomplete unless it is clear whether the outcome is any treatment-emergent adverse event, a particular event, or a severe event, and over what follow-up period.

Definition: Let \(d\) denote a dose, \(E(d)\) its efficacy measure (with larger values defined to be better), and \(T(d)\) the risk of a prespecified toxicity outcome in the target population and time period. Given an acceptable toxicity limit \(\tau\), a dose is feasible if \(T(d)\leq\tau\). The constrained dose is a feasible dose with the greatest efficacy. For several toxicity requirements \(T_j(d)\leq\tau_j\), a dose is feasible only if it meets every requirement.

For a finite set of candidate doses \(\mathcal{D}\), the decision can be written as

$$ d^*=\underset{d\in\mathcal{D}:\,T(d)\leq\tau}{\operatorname{arg\,max}}\,E(d). $$

This is a constrained optimization rule, not a claim that efficacy or toxicity is known without error. In practice, \(E(d)\) and \(T(d)\) are estimated from data or a dose-response model. The decision rule must state whether feasibility is determined from point estimates or from a more conservative uncertainty bound.

How the Constraint Determines the Choice

Suppose there are finitely many candidate doses and each has an efficacy value and a toxicity risk defined on the same target population. The selection procedure is to evaluate each toxicity requirement, remove any dose that fails at least one requirement, and maximize efficacy over the remaining set. This follows directly from the meaning of the constraint: an infeasible dose is not an admissible choice, however high its efficacy estimate.

A useful special case occurs when toxicity risk is nondecreasing with dose and efficacy is also nondecreasing. If the doses form an ordered set, the feasible doses then run from the lowest dose through the highest dose that meets the toxicity limit. Among those feasible doses, the highest one has at least as much efficacy as every lower feasible dose, so it is optimal. This shortcut depends on both monotonicity assumptions. If efficacy is nonmonotonic, or if estimated toxicity does not increase consistently, compare efficacy across all feasible doses rather than automatically choosing the highest.

Derivation: Suppose \(d_1<d_2<\cdots<d_k\), \(T(d)\) is nondecreasing, and \(E(d)\) is nondecreasing. If \(d_k\) is the highest feasible dose, then every \(d_i\leq d_k\) is feasible because \(T(d_i)\leq T(d_k)\leq\tau\). Also \(E(d_i)\leq E(d_k)\). Therefore \(d_k\) maximizes efficacy over the feasible doses. If either monotonicity condition fails, this argument does not establish the optimum.

Worked Examples

Worked Example: Selecting Among Tested Doses Under a Risk Limit

Consider hypothetical trial results for a treatment. Efficacy is the mean improvement on a symptom scale, in points, with larger improvements better. Toxicity is the proportion of participants with a specified adverse event during 12 weeks. Suppose the acceptable risk is \(\tau=0.10\), and the decision uses the reported estimates as its feasibility criteria.

Dose (mg/day)Mean improvement (points)Estimated toxicity riskFeasible at 10%?
00.00.02Yes
252.10.04Yes
504.00.08Yes
755.30.17No
1005.80.29No

The 75 mg and 100 mg doses exceed the risk limit, so the feasible set is 0, 25, and 50 mg/day. Their efficacy estimates are 0.0, 2.1, and 4.0 points. The greatest efficacy among feasible doses is 4.0 points, so the selected dose is 50 mg/day. The fact that 100 mg/day has the largest efficacy estimate does not make it an admissible choice under this constraint.

The unconstrained efficacy-maximizing dose is 100 mg/day, with a mean improvement of 5.8 points. The estimated improvement forgone by choosing the constrained dose is

$$ 5.8-4.0=1.8\text{ points}. $$

Relative to the unconstrained improvement, this is \(1.8/5.8\times100=31.0\%\). This percentage describes the estimated efficacy difference in this example; it does not describe a toxicity reduction or establish that the selected dose is clinically preferable under every decision framework.

Worked Example: Requiring an Upper Confidence Bound to Meet the Limit

In a second hypothetical study, toxicity is observed in 4 of 100 participants at 20 mg/day, 10 of 100 at 40 mg/day, and 18 of 100 at 60 mg/day. Suppose the safety rule requires a one-sided 95% upper confidence bound for the toxicity risk to be at most 0.15. For illustration, use the approximate Wald bound \(\hat p+1.645\sqrt{\hat p(1-\hat p)/n}\), where \(\hat p\) is the observed proportion and \(n=100\). Assume the efficacy estimates are 2.0, 3.6, and 4.5 points, respectively.

At 20 mg/day, \(\hat p=0.04\), so the approximate upper bound is

$$ 0.04+1.645\sqrt{\frac{0.04(0.96)}{100}} =0.04+1.645(0.01960) =0.0722. $$

At 40 mg/day, \(\hat p=0.10\), giving

$$ 0.10+1.645\sqrt{\frac{0.10(0.90)}{100}} =0.10+1.645(0.03000) =0.1494. $$

At 60 mg/day, \(\hat p=0.18\), giving

$$ 0.18+1.645\sqrt{\frac{0.18(0.82)}{100}} =0.18+1.645(0.03842) =0.2432. $$

The 20 mg and 40 mg doses meet the stated upper-bound rule; 60 mg does not. Of the feasible doses, 40 mg/day has the greater efficacy estimate, 3.6 points, and is selected. Compared with the unconstrained maximum of 4.5 points at 60 mg/day, the estimated efficacy sacrificed is \(4.5-3.6=0.9\) points, or \(0.9/4.5\times100=20.0\%\) of that maximum.

The bound is an approximate frequentist confidence bound, not a 95% probability that the true risk is below 0.15. It relies on a normal approximation; sparse event counts or risks near zero or one can make this approximation unreliable. A prespecified Wilson, exact, or model-based bound may be more appropriate in an actual analysis. The example's choice is conditional on the stated approximation and rule.

Worked Example: Applying Two Toxicity Constraints When Efficacy Is Nonmonotonic

Suppose a hypothetical study reports mean efficacy improvements in points, the risk of any specified toxicity, and the risk of a severe toxicity over the same follow-up period. The decision limits are 10% for any toxicity and 2% for severe toxicity. For this example, feasibility is assessed using the displayed estimates.

Dose (mg/day)Efficacy (points)Any toxicitySevere toxicity
102.03%0.2%
203.47%0.8%
303.19%1.5%
403.814%2.8%

The 40 mg dose fails both toxicity limits. The other three doses meet both, so they form the feasible set. Their efficacy estimates are 2.0, 3.4, and 3.1 points; therefore, 20 mg/day is selected. The 30 mg estimate is lower than the 20 mg estimate, so selecting the highest feasible dose would be incorrect here.

The unconstrained efficacy maximum is 3.8 points at 40 mg/day. The estimated benefit sacrificed is \(3.8-3.4=0.4\) points, which is \(0.4/3.8\times100=10.5\%\) of the unconstrained maximum. Both safety criteria matter: a dose that meets one but fails the other is not feasible under this decision rule.

Choosing and Checking the Safety Rule

A point-estimate constraint and an uncertainty-aware constraint answer different operational questions. Requiring \(\hat T(d)\leq\tau\) screens on the estimated risk. Requiring an upper confidence bound \(U(d)\leq\tau\) screens on whether the data support risk below the limit at a specified confidence level. A Bayesian rule could instead require a prespecified posterior probability that risk is below the limit. These rules can select different doses, so the choice of rule and its uncertainty level should be stated before interpreting the result.

The toxicity endpoint must also match the decision. A limit for any adverse event does not automatically constrain the risk of a severe event; conversely, a severe-event limit alone may not control a high burden of less severe events. For multiple outcomes, define each \(T_j(d)\) and \(\tau_j\) and require all of them to be met. The populations should be clinically relevant: a population-average constraint may conceal a high risk in an important subgroup. The earlier material on dose selection under interindividual variability addresses how variation across people affects dose decisions.

Before trusting the selected dose, check that doses, follow-up, toxicity ascertainment, and denominators are comparable. Review missing safety observations, treatment discontinuation, event timing, and whether the toxicity definition was applied consistently. Sparse data can make a dose appear to meet a point-estimate limit simply because few events were observed. Inspect the dose-response pattern and its uncertainty rather than assuming a smooth increase; the earlier material on diagnosing nonmonotonic dose-response patterns is relevant when the observed pattern is irregular.

What the Constrained Choice Does—and Does Not—Say

The efficacy sacrificed is the difference between the estimated efficacy at the unconstrained maximum and at the selected feasible dose. If efficacy is measured as an improvement from placebo or baseline, calculate the difference on that same scale. A relative percentage is informative only when the denominator is meaningful and nonzero. Report the absolute difference and its units even when a percentage is also shown.

This comparison is not a substitute for uncertainty assessment. If estimated efficacy differences are imprecise, the dose with the largest point estimate among feasible options may not have a reliably greater true effect. Likewise, a dose that meets a risk limit on a point estimate is not thereby proven safe. Communicate the estimates, uncertainty, constraint definition, and decision rule together.

A hard constraint is also different from utility-based dose optimization or joint efficacy-safety benefit-risk optimization. Those approaches combine outcomes or trade-offs according to specified preferences; the constrained rule instead treats doses outside the safety boundary as inadmissible. Neither approach removes the need to justify the toxicity limit. A constraint may be too permissive, too strict, or based on an endpoint that does not represent the relevant clinical concern.

Common Pitfalls

  • Choosing the efficacy maximum before checking safety. A dose with the greatest efficacy estimate is not an option under a hard constraint if it fails the specified safety rule.
  • Calling a dose safe because its point estimate is below the limit. That conclusion ignores uncertainty. If the rule requires an upper bound, compare the upper bound—not just the point estimate—with the limit.
  • Using only one of several required limits. If every toxicity criterion must be met, failing any one criterion makes the dose infeasible.
  • Automatically choosing the highest feasible dose. This shortcut requires monotonic efficacy. When efficacy is nonmonotonic, compare all feasible doses.
  • Reporting only relative benefit sacrificed. A percentage can obscure the size and units of the effect difference; report the absolute difference as well.
  • Treating a chosen threshold as a universal biological boundary. The toxicity limit is a decision criterion for a defined outcome, population, and time period, not proof that risk just above it is unacceptable in every context or risk just below it is harmless.
Key takeaway: Define the toxicity endpoint and limit, state how uncertainty enters the feasibility rule, exclude doses that fail any required constraint, and select the dose with the greatest efficacy among the remaining options. Quantify the efficacy sacrificed against the unconstrained choice without overstating what the estimates establish.

Check Your Understanding

Answer each question using the stated rule. Solutions follow the questions.

  1. Doses of 5, 10, and 15 mg have efficacy estimates of 1.2, 2.4, and 3.0 points and toxicity risks of 4%, 9%, and 13%. If the risk limit is 10% and point estimates determine feasibility, which dose is selected, and how much efficacy is sacrificed compared with the unconstrained maximum?
  2. Why can a dose with a toxicity point estimate below the limit fail a rule based on an upper confidence bound?
  3. Three doses meet all toxicity limits, but their efficacy estimates are 2.8, 3.1, and 2.9 points in increasing dose order. Is the highest feasible dose optimal under the constrained rule?
  4. A dose meets the limit for severe toxicity but exceeds the limit for any toxicity, and both limits are required. Is the dose feasible?

Solutions. (1) The 15 mg dose is infeasible because 13% exceeds 10%; among the 5 mg and 10 mg doses, 10 mg has the greater efficacy estimate. The sacrifice is \(3.0-2.4=0.6\) points. (2) The upper confidence bound incorporates uncertainty and may exceed the limit even when the point estimate does not; the dose fails if the rule requires the bound itself to be no greater than the limit. (3) No. The middle dose has the greatest efficacy estimate, 3.1 points, so it is selected if all three are feasible. (4) No. A dose must meet every required constraint, and exceeding either one makes it infeasible.

Clinical Biostats