Adaptive Randomization Using Joint Outcomes

Why Randomize Adaptively?

In a dose-finding trial, assigning every new patient to the dose with the largest current estimated benefit-risk utility may appear efficient. But that estimate is uncertain, especially early in the trial. A dose that looks best by chance could receive most of the remaining patients, while doses that might ultimately be preferable receive too little evidence for a reliable comparison. Adaptive randomization addresses this tension by letting accumulated joint efficacy-toxicity evidence influence allocation while retaining some random assignment among candidate doses.

The allocation rule is not itself a joint outcome model or a safety rule. It takes the model’s current evidence as input and converts that evidence into probabilities of assigning the next patient. The rule must therefore be specified alongside the utility definition, eligibility restrictions, and timing of outcome updates.

Definition: Let $D_n$ denote the available trial data after $n$ patients, and let $d$ index the candidate doses still eligible for allocation. Let $u(E,T)$ be the prespecified utility for a patient’s joint efficacy outcome $E$ and toxicity outcome $T$. The posterior expected utility at dose $d$ is $$ U_d=E\{u(E,T)\mid D_n,\text{ dose }d\}. $$ An adaptive randomization rule maps the values $U_d$, and possibly other summaries of uncertainty, to probabilities $\pi_d$ for assigning the next patient, with $\pi_d\geq 0$ and $\sum_d\pi_d=1$.

For example, when efficacy and toxicity are binary, the joint outcome has four possible states. The utility must assign a value to each state before the trial uses the allocation rule. A joint model matters because efficacy and toxicity may be associated: calculating a utility from separate marginal estimates while ignoring their joint distribution can misrepresent how often benefit and harm occur together.

The rule also relies on operational assumptions. The set of doses eligible for randomization must be determined by the protocol before probabilities are calculated; a dose that has been ruled unsafe cannot be made eligible merely by a favorable utility estimate. The outcome data used at an allocation update must be defined consistently, and the model must provide meaningful uncertainty summaries under the trial’s prespecified assumptions. If outcomes are incomplete or delayed, the protocol must specify what data are available at each update and how the model handles them. Finally, the assignment probabilities must be recorded and implemented as calculated.

Softmax Allocation From Expected Utility

One common rule assigns larger probabilities to doses with larger posterior expected utility, but does not usually assign probability one to the current leader. For a set $\mathcal{E}$ of $K$ eligible doses, the softmax probabilities are

q_d=\frac{\exp(\lambda U_d)}{\sum_{j\in\mathcal{E}}\exp(\lambda U_j)},\qquad d\in\mathcal{E},

where $\lambda\geq 0$ controls how strongly allocation favors higher expected utility. With $\lambda=0$, the rule is uniform over eligible doses. As $\lambda$ increases, allocation becomes more concentrated on doses with the highest utility. Because the exponential’s argument must be dimensionless, $\lambda$ has units inverse to those of utility. The numerical choice of $\lambda$ therefore depends on the utility scale.

Justification: Every softmax weight is positive for a finite $U_d$, and normalization makes the weights sum to one: $$ \sum_{d\in\mathcal{E}}q_d= \frac{\sum_{d\in\mathcal{E}}\exp(\lambda U_d)} {\sum_{j\in\mathcal{E}}\exp(\lambda U_j)}=1. $$ For two eligible doses $a$ and $b$, their allocation odds satisfy $$ \frac{q_a}{q_b}=\exp\{\lambda(U_a-U_b)\}. $$ Thus the rule depends on utility differences, not the absolute utility origin. Adding the same constant $c$ to every utility multiplies every weight by $\exp(\lambda c)$, which cancels in normalization.

The softmax rule is a tunable allocation policy, not a claim that its probabilities are uniquely optimal. A small utility difference can still yield a large allocation imbalance if $\lambda$ is chosen too large. Conversely, a very small $\lambda$ may allocate almost uniformly even when evidence favors one eligible dose. If the utility scale is rescaled, $\lambda$ must be adjusted to preserve the same allocation behavior.

Worked Example: Softmax Probabilities and an Exploration Floor

Suppose two doses are eligible and their posterior expected utilities are $U_1=3/13\approx0.230769$ and $U_2=0.2000$ utility units per patient. Use $\lambda=6$ per utility unit, then add a uniform exploration mixture with $\epsilon=0.20$.

First calculate the unnormalized weights using the unrounded utility for dose 1:

w_1=\exp(6\times 3/13)=\exp(18/13)\approx3.9933,\qquad w_2=\exp(6\times0.2000)=\exp(1.2)\approx3.3201.

The normalized softmax probabilities are

q_1=\frac{3.9933}{3.9933+3.3201}\approx0.5460,\qquad q_2=\frac{3.3201}{3.9933+3.3201}\approx0.4540.

Now use the exploration mixture $\pi_d=(1-\epsilon)q_d+\epsilon/K$, with $K=2$. Therefore,

\pi_1=0.80(0.5460)+0.10=0.5368,\qquad \pi_2=0.80(0.4540)+0.10=0.4632.

The next patient has a 53.68% assignment probability to dose 1 and a 46.32% probability to dose 2. The mixture prevents the leading dose from receiving all assignments; it is a chosen design safeguard, not a guarantee that the resulting dose estimates will be precise.

As a rounding check, if two softmax weights are $2.226$ and $3.597$, normalization gives $q_1\approx2.226/(2.226+3.597)=0.3823$ and $q_2\approx0.6177$. Rounding both probabilities from the same normalized calculation keeps them complementary to the reported precision.

Computing Utility From Joint Evidence

The allocation rule should use utility summaries consistent with the trial’s joint outcome model. The following hypothetical example shows how probabilities for the four binary efficacy-toxicity combinations produce an expected utility. Suppose efficacy is coded $E=1$ for success and $E=0$ otherwise, toxicity is $T=1$ for toxicity and $T=0$ otherwise, and the prespecified utility is $u(E,T)=2E-3T$. The coefficients are illustrative utility weights, not universal clinical values.

Worked Example: Expected Utility From Four Joint Outcomes

For two eligible doses, suppose the model’s current joint outcome probabilities are as follows. Each row’s four probabilities sum to one.

Dose	$P(E=1,T=0\mid D_n)$	$P(E=1,T=1\mid D_n)$	$P(E=0,T=0\mid D_n)$	$P(E=0,T=1\mid D_n)$
A	0.35	0.15	0.40	0.10
B	0.55	0.20	0.15	0.10

For dose A, the four utilities are $2$, $-1$, $0$, and $-3$, respectively. Thus

$$ U_A=(0.35)(2)+(0.15)(-1)+(0.40)(0)+(0.10)(-3)=0.25. $$

For dose B, the same four utility values give

$$ U_B=(0.55)(2)+(0.20)(-1)+(0.15)(0)+(0.10)(-3)=0.60. $$

If $\lambda=2.5$ per utility unit, the weights are $\exp(2.5\times0.25)=\exp(0.625)\approx1.8682$ and $\exp(2.5\times0.60)=\exp(1.5)\approx4.4817$. Normalizing gives $q_A\approx1.8682/6.3499=0.2942$ and $q_B\approx4.4817/6.3499=0.7058$. The allocation favors B because its current expected utility is higher; this does not establish that B is truly superior or that its toxicity risk is acceptable outside the specified eligibility rule.

Making Room for Dose Learning

Expected-utility softmax allocation trades off concentration on the current utility leader against randomization to other eligible doses. It does not, by itself, reward uncertainty or information gain. An exploration floor is one transparent way to preserve some allocation to each eligible dose:

\pi_d=(1-\epsilon)q_d+\frac{\epsilon}{K},\qquad 0\leq\epsilon\leq1.

The resulting probability is at least $\epsilon/K$ for every eligible dose. The guarantee applies only while a dose remains eligible; it does not override safety restrictions. Increasing $\epsilon$ supports broader data collection but reduces the fraction of assignments driven by current utility estimates. This is an explicit design tradeoff, not a free improvement in both patient benefit and learning.

A different approach is posterior-optimality randomization. For each posterior draw, identify the eligible dose with the highest utility; allocate in proportion to the posterior probability that each dose is optimal. This uses uncertainty in the utilities: a dose with a modest posterior mean may still receive assignments if it has a meaningful chance of being best. But this approach is not the same as a softmax of posterior means, and it may allocate to a dose that has low current expected utility. Safety eligibility and any minimum allocation protections still need to be specified separately.

Worked Example: Allocation by Posterior Probability of Being Optimal

Suppose 100 draws from the fitted posterior are used to compare three eligible doses. Dose 1 has the highest utility in 25 draws, dose 2 in 60, and dose 3 in 15. The estimated posterior probabilities of being optimal are therefore $0.25$, $0.60$, and $0.15$.

Use these probabilities as $q_d$, then apply a uniform exploration mixture with $\epsilon=0.15$ and $K=3$. The uniform contribution to each dose is $0.15/3=0.05$, so

\pi_1=0.85(0.25)+0.05=0.2625,\qquad \pi_2=0.85(0.60)+0.05=0.5600,\qquad \pi_3=0.85(0.15)+0.05=0.1775.

These probabilities sum to $1.0000$. Dose 2 is favored, while dose 3 retains a positive chance of assignment despite being optimal in only 15% of the posterior draws. The posterior probabilities here are estimated from a finite number of draws, so more draws may be needed if Monte Carlo error could materially change allocation.

Assumptions, Diagnostics, and Alternatives

Adaptive allocation is only as credible as the inputs and implementation supporting it. The utility must reflect prespecified clinical priorities, the joint model must represent the relevant efficacy-toxicity outcomes adequately, and the eligibility set must be applied consistently. Repeatedly changing the utility weights after seeing trial results can make the allocation rule difficult to interpret and can undermine its intended benefit-risk meaning. Allocation probabilities should be computed from the data available at the protocol-defined update, not from future outcomes.

Useful operational checks include verifying that all probabilities are nonnegative and sum to one; confirming that ineligible doses receive probability zero; checking that every eligible dose meets any prespecified probability floor; and examining whether rounding or software implementation changes the intended assignments. At each update, review the utilities and allocation probabilities together. A highly concentrated allocation with little separation in estimated utility may indicate an overly aggressive $\lambda$, an unexpected utility scale, or a coding error. Very diffuse allocation despite substantial utility differences can indicate the opposite.

Compare the observed allocation pattern with the intended rule, including the number assigned to each dose and the evidence available when assignments were made. Sparse allocation to a dose can limit the information available to compare its outcomes, especially if the model’s conclusions depend strongly on assumptions or borrowed information. Conversely, extensive allocation to a dose with poor benefit-risk may expose more patients than intended. These are reasons to assess the policy before use and monitor its operation, not reasons to interpret any one allocation as proof that a dose is best.

The main alternatives answer different questions. Fixed randomization allocates according to probabilities set in advance, giving stable planned comparisons but not adapting to accumulating evidence. Softmax allocation emphasizes posterior expected utility, with $\lambda$ controlling concentration. Posterior-optimality allocation emphasizes how often a dose is best across posterior draws. A mixture with an exploration floor can be added to either utility-based approach to preserve minimum assignment probabilities. None automatically guarantees adequate precision, ethical acceptability, or a reliable final dose recommendation.

Common Pitfall

A frequent error is to say, “Dose 2 has the largest posterior expected utility, so every new patient should receive dose 2.” That reasoning treats an uncertain estimate as if it were known and ignores the information lost when other eligible doses receive no assignments. The correction is to define a protocol-based allocation rule that balances the current utility evidence and the desired learning protection, then report the resulting probabilities as assignment chances conditional on the available data. These probabilities are not the probability that a dose is safe, nor the probability that it is truly optimal.

Key takeaway: Adaptive randomization converts joint efficacy-toxicity evidence into assignment probabilities. Utility-based concentration can favor doses with higher estimated patient benefit, while explicit exploration provisions can preserve opportunities to learn. Their balance must be chosen in advance, applied only to eligible doses, and checked at every update.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Two eligible doses have utilities $0.30$ and $0.10$, and $\lambda=5$. Which dose has greater softmax probability, and what are the approximate probabilities?
Why can a uniform exploration mixture not assign patients to a dose that has been ruled ineligible?
In the joint-outcome example, how would dose A’s expected utility change if the utility for the $E=0,T=1$ outcome were $-2$, with all probabilities unchanged?
How does posterior-optimality randomization differ from softmax allocation based on posterior expected utilities?

Solutions. For question 1, the weights are $\exp(5\times0.30)=\exp(1.5)\approx4.4817$ and $\exp(5\times0.10)=\exp(0.5)\approx1.6487$. Their sum is approximately $6.1304$, so the probabilities are $4.4817/6.1304\approx0.7311$ and $1.6487/6.1304\approx0.2689$; dose 1 has the greater probability. For question 2, the eligible set is determined before allocation. An ineligible dose is excluded from the normalization and receives probability zero; the exploration floor is defined only over eligible doses. For question 3, the original dose A contribution from that cell was $(0.10)(-3)=-0.30$; changing its utility to $-2$ gives $(0.10)(-2)=-0.20$, increasing $U_A$ by $0.10$, from $0.25$ to $0.35$. For question 4, softmax uses transformed posterior mean utilities, whereas posterior-optimality allocation uses the fraction of posterior draws in which each eligible dose is best. They can therefore produce different probabilities, especially when uncertainty differs across doses.