Why the Reward Definition Matters
An adaptive randomization rule uses accumulating outcomes to change the probabilities with which future patients receive treatments. But the rule cannot identify a “better” treatment without a definition of better. A high probability of meeting a response threshold, a high average outcome, and a high expected utility are distinct objectives. They may rank treatments differently, even when calculated from the same hypothetical patient outcomes.
The reward is therefore not a neutral label attached to the data. It determines which outcomes the allocation rule values. That choice can change the allocation pattern and the balance between expected patient benefit and the amount of information available for a treatment comparison. The introductory material on response-adaptive randomization motivates adapting allocation; here the focus is how the reward definition shapes that adaptation.
Here \(a\) indexes a treatment, \(Y^a\) denotes the outcome that would be observed under treatment \(a\), and \(R^a\) indicates response under that treatment. For example, when larger values of \(Y^a\) are better and response is defined by a threshold \(c\), \(R^a=I(Y^a\ge c)\). If smaller outcomes are better, the response criterion must instead use the relevant lower threshold. The response definition and reward direction must be fixed clearly before using them to guide allocation.
Three Rewards, Three Possible Rankings
A response-probability reward reduces the outcome to whether a patient crosses a specified threshold. It can be clinically easy to interpret, but it does not distinguish values on the same side of that threshold. A mean-outcome reward uses the outcome scale directly, so large values can matter substantially to the average even if they are uncommon. A utility reward combines outcomes according to prespecified values—for example, assigning a penalty to toxicity. It can represent a broader patient-benefit objective, but the chosen values and tradeoffs need clinical justification.
The rewards are not interchangeable summaries. If \(g\) is nonlinear, then in general \(E[g(Y^a)]\ne g(E(Y^a))\). In particular, a squared-outcome reward has expected value \(E[(Y^a)^2]\), not \((E(Y^a))^2\). Moreover, a reward intended to guide allocation is not automatically the same quantity as the treatment contrast chosen for the trial’s primary analysis.
Worked Example: Three Definitions Reverse the Ranking
Hypothetical data-generating model. Suppose treatment A produces \(Y=10\) with probability 0.60 and \(Y=0\) with probability 0.40. Treatment B produces \(Y=6\) with probability 0.90 and \(Y=0\) with probability 0.10. Let response mean \(Y\ge5\). Suppose toxicity indicators \(T\) have probabilities 0.50 under A and 0.05 under B. Define utility in outcome units as \(U=Y-8T\), so a toxic event carries a prespecified penalty of 8 units. Assume these distributions are stable across patients and treatment assignment is randomized.
Response probability. Only the positive outcome values meet the threshold, so \(P(R^A=1)=0.60\) and \(P(R^B=1)=0.90\). This reward favors B.
Mean outcome. The expected outcomes are \(E(Y^A)=10(0.60)+0(0.40)=6.00\) and \(E(Y^B)=6(0.90)+0(0.10)=5.40\). This reward favors A.
Expected utility. By linearity of expectation, \(E(U^a)=E(Y^a)-8E(T^a)\). Thus \(E(U^A)=6.00-8(0.50)=2.00\), whereas \(E(U^B)=5.40-8(0.05)=5.00\). This reward favors B. No assumption that outcome and toxicity are independent is needed for this calculation, because the utility is the stated difference and expectation is linear.
For illustration, use a simple allocation rule that assigns probability 0.70 to the treatment with the higher estimated reward and 0.30 to the other treatment. With response probability as reward, B receives 0.70; with mean outcome, A receives 0.70; with utility, B receives 0.70. This rule illustrates the consequence of a changed ranking, not a recommended universal allocation procedure. The numerical allocation probabilities are fixed by the illustrative rule, not implied by the reward values themselves.
From Estimated Reward to Allocation
An adaptive design acts on estimated rewards, not on the unknown population quantities directly. A common conceptual sequence is to estimate each treatment’s expected reward from outcomes observed so far, then apply a prespecified rule that converts those estimates into randomization probabilities. The same response data can be summarized as a response probability, an outcome mean, or a utility mean; which summary enters the rule must be specified.
For example, suppose responses are independent Bernoulli outcomes within each treatment, with a stable response probability during the trial. A Beta(1,1) prior for each treatment’s response probability gives a posterior mean equal to \((s+1)/(n+2)\), where \(s\) is the number of responses among \(n\) assigned patients. This posterior mean is an estimate of response probability; it is not the probability that the treatment is best.
Worked Example: Response Reward From Accumulating Data
Hypothetical data. After 20 patients receive each of two treatments, A has 12 responses and B has 18. Assume independent Bernoulli responses within treatment, stable response probabilities, and independent Beta(1,1) priors. Use the posterior mean response probability as the estimated reward, and use the illustrative 0.70/0.30 allocation rule described above.
For A, the posterior distribution is Beta(12+1, 20-12+1)=Beta(13,9). Its posterior mean reward is \(13/(13+9)=13/22=0.5909\). For B, the posterior distribution is Beta(18+1, 20-18+1)=Beta(19,3), with posterior mean reward \(19/(19+3)=19/22=0.8636\).
Because B has the larger estimated response-probability reward, the illustrative rule assigns the next patient to B with probability 0.70 and to A with probability 0.30. This conclusion is specific to response probability as the reward. If the design instead uses mean outcome or utility, it must calculate those expected rewards from the corresponding patient outcomes and apply the rule to that different comparison.
The allocation mechanism and the reward definition are separate design choices. A rule might use a thresholded response estimate, a posterior expected mean, or an estimated utility; it might then use a particular mapping from estimates to assignment probabilities. Changing either choice can alter allocation. When comparing reward definitions, hold the allocation rule and other design features fixed if the aim is to isolate the effect of the reward.
Allocation, Information, and Power
A reward aimed at patient benefit does not necessarily maximize precision for a treatment contrast. To see the connection, suppose \(n_1\) and \(n_0\) independent patients receive treatments 1 and 0, respectively, with fixed sample sizes and outcome variances \(\sigma_1^2\) and \(\sigma_0^2\). The variance of the difference in sample means is \(\sigma_1^2/n_1+\sigma_0^2/n_0\). With equal variances, a strongly unequal allocation generally increases this variance relative to equal allocation, reducing precision and potentially power for a fixed contrast.
For a fixed total \(N=n_1+n_0\), the allocation minimizing this variance under the stated independent-sampling model can be derived by differentiating \(V(n_1)=\sigma_1^2/n_1+\sigma_0^2/(N-n_1)\). Setting the derivative to zero gives the condition below; the positive solution states the variance-minimizing ratio.
This derivation assumes fixed sample sizes, independent outcomes within and across treatment groups, and known variances for the purpose of the allocation calculation. It does not say that this ratio is the best choice for every trial: it concerns variance of a mean difference, not patient benefit or every possible inferential target. It does show why a reward-based rule that concentrates patients on one arm may not allocate patients in the way that best supports a precise treatment contrast.
Worked Example: Unequal Allocation and Approximate Power
Hypothetical comparison. Consider a two-sided test of equal response probabilities at significance level \(\alpha=0.05\), with 100 patients in total. Suppose the true response probabilities are 0.50 under control and 0.80 under the treatment favored by a response reward. Assume independent Bernoulli outcomes and, for this illustration, fixed group sizes. Compare equal allocation with an 80/20 allocation that assigns 80 patients to the favored treatment and 20 to control.
With equal allocation, \(n_1=n_0=50\). Using the standard error under this alternative, \(SE=\sqrt{0.80(0.20)/50+0.50(0.50)/50}=\sqrt{0.0082}=0.09055\). The standardized difference is \(z= (0.80-0.50)/0.09055=3.313\). A normal-approximation two-sided power is approximately \(\Phi(3.313-1.96)+\Phi(-3.313-1.96)=0.912\).
With 80 patients on treatment and 20 on control, \(SE=\sqrt{0.80(0.20)/80+0.50(0.50)/20}=\sqrt{0.0145}=0.1204\). Then \(z=0.30/0.1204=2.492\), giving approximate power \(\Phi(2.492-1.96)+\Phi(-2.492-1.96)=0.702\). The example’s normal approximation uses known response probabilities to calculate the alternative standard error; an actual trial’s test and power calculation may use different variance estimates and account for random allocation counts. The comparison isolates how, in this setting, reducing the control group can decrease power despite concentrating assignment on the treatment with higher response probability.
Choosing and Checking a Reward
Before a trial begins, the reward should match the intended objective closely enough that its allocation consequences are acceptable. A response probability may be suitable when crossing a meaningful threshold is central to the decision. A mean outcome may be suitable when differences across the full outcome range matter. A utility can include harms, burdens, or costs, but its components and weights must be justified and interpreted in their units. If the reward and primary treatment contrast differ, state that distinction rather than implying that allocation directly optimizes the inferential target.
A useful comparison keeps the treatment outcome model, sample size, prior specification, allocation rule, and other design choices fixed while changing only the reward. Examine both the resulting allocation pattern and the operating characteristics relevant to the trial, such as average treatment-specific sample sizes, patient outcomes under the specified model, precision, and power. In an adaptive design, allocation counts vary across trial realizations; a single expected allocation is not a complete description. Simulation can reveal whether a reward definition produces unstable or unexpectedly concentrated allocations under plausible outcome distributions.
| Reward | What it values | Key design question |
|---|---|---|
| Response probability | Meeting a specified response criterion | Is the threshold clinically meaningful, and does it discard important differences within response categories? |
| Mean outcome | Average value on the outcome scale | Is the outcome direction clear, and is the average the intended patient-benefit target? |
| Expected utility | Prespecified combination of outcomes, harms, or costs | Are the components and tradeoff weights defensible and interpretable? |
| Fixed allocation | A prespecified assignment ratio, not an estimated reward | Is a stable comparison or a particular information balance preferred to reward-driven adaptation? |
Diagnostics should check the quantities the rule actually uses. Verify that response coding and outcome directions are correct, that utility calculations apply the intended weights, and that estimates are based on the outcomes observed for the assigned treatments. Compare observed allocation probabilities and arm counts with the specified rule. If different plausible reward definitions produce sharply different allocations, that is a substantive sensitivity finding: the design’s behavior depends on a value choice, not merely on sampling noise.
Common Pitfall and Limitations
Wrong reasoning: “Treatment A has the higher mean outcome, so an adaptive rule based on response probability should allocate more patients to A.” This is incorrect because the rule responds to the reward it was given. In the first worked example, A has the higher mean outcome, while B has the higher response probability and expected utility. The correction is to calculate the prespecified reward for each treatment, then apply the allocation rule to those reward estimates.
Reward sensitivity is not resolved simply by choosing the reward that produces the most unequal allocation or the largest apparent patient benefit. Estimates are uncertain, outcomes can vary, and treatment comparisons need information from both groups. The fixed-count power calculation above illustrates one tradeoff, but does not describe every adaptive trial: in a response-adaptive design, counts depend on previous outcomes, and operating characteristics must be evaluated under the full design. Nor does a single hypothetical distribution establish which reward is clinically preferable. That judgment requires a clearly stated objective and defensible assumptions about outcomes and harms.
Check Your Understanding
Answer each question using the stated reward definition. Solutions follow the questions.
- A treatment produces \(Y=8\) with probability 0.40 and \(Y=2\) with probability 0.60. If response is \(Y\ge5\), what are its response probability and mean outcome?
- For a hypothetical outcome with probabilities \(P(Y=1)=0.50\) and \(P(Y=3)=0.50\), calculate \(E(Y)\) and \(E(Y^2)\). Why are these not interchangeable rewards?
- Treatment C has mean outcome 9 and toxicity probability 0.50; treatment D has mean outcome 7 and toxicity probability 0.10. If \(U=Y-6T\), which treatment has greater expected utility?
- For independent groups with outcome variances \(\sigma_1^2=4\) and \(\sigma_0^2=1\), what ratio \(n_1/n_0\) minimizes the variance of the difference in sample means under the fixed-total assumptions?
Solutions. (1) Only \(Y=8\) meets the threshold, so the response probability is 0.40. The mean is \(8(0.40)+2(0.60)=4.40\) outcome units. (2) \(E(Y)=1(0.50)+3(0.50)=2.00\), whereas \(E(Y^2)=1^2(0.50)+3^2(0.50)=5.00\). They are different expected rewards because squaring is nonlinear. (3) \(E(U^C)=9-6(0.50)=6.00\); \(E(U^D)=7-6(0.10)=6.40\). D has the greater expected utility by 0.40 utility units. (4) The variance-minimizing ratio is \(\sigma_1/\sigma_0=2/1=2\), so \(n_1/n_0=2\), subject to the assumptions in the derivation.