Why the Enrollment Horizon Changes the Decision
An adaptive trial assigns treatments sequentially and uses observed outcomes to guide later assignments. The value of an assignment therefore depends not only on the outcome it may produce for the patient receiving it, but also on whether the information can improve decisions for patients who have not yet enrolled. If enrollment ends after a fixed number of assignments, the number of patients still to come is part of the decision problem.
A finite-horizon objective states what the trial is trying to optimize over a specified, fixed enrollment horizon. It is not itself an allocation rule. An objective might evaluate outcomes for enrolled patients, the quality of a final treatment recommendation, or both. Different objectives can favor different allocation policies even when they use the same outcome model and the same horizon.
A history includes assignments and the outcomes observed so far. A policy maps that history to an arm or to assignment probabilities across arms. For an objective based on the complete trial history, one general expression is \(J_T(\pi)=E_\pi[G(H_T)]\), where \(H_T\) is the history at the end and \(G\) is the chosen terminal criterion. A criterion may instead combine outcomes accumulated during enrollment with a terminal payoff. Its definition should make clear what is valued, when it is valued, and which constraints apply.
Specify the Objective Before Choosing the Policy
A common mathematical form assigns each patient a utility \(u_t\), determined by the assigned arm and that patient’s outcome, and may also include a terminal payoff \(G\). The objective is then the expected value of the specified trial utility under policy \(\pi\). This form is useful for deriving a decision rule; it does not imply that every clinical objective should value outcomes only through an additive sum.
Here \(A_t\) is the arm assigned to patient \(t\), and \(Y_t\) is that patient’s outcome. If the chosen criterion concerns only a terminal recommendation, the stage utilities can be set to zero and the terminal payoff can encode the quality of that recommendation. If the criterion concerns patient outcomes during enrollment, stage utilities can represent those outcomes and the terminal payoff can be zero. A trial may also include constraints, such as minimum allocation shares; those restrictions define which policies are permitted rather than changing what the objective measures.
This distinction matters in practice. A policy is a procedure for assigning treatments; an objective is the standard by which its consequences are judged. A policy that learns quickly may be attractive for a terminal decision, while a policy that prioritizes the outcomes of patients enrolled now may allocate differently. The objective and any constraints should be stated before interpreting an allocation rule as “best.”
Worked Example: A Fixed Horizon and Two Allocation Policies
Suppose a hypothetical trial enrolls \(T=4\) patients and compares arms A and B. For this illustration only, assume independent Bernoulli outcomes with known success probabilities \(p_A=0.70\) and \(p_B=0.50\). Let the objective be the expected number of successes among the four enrolled patients, with no terminal payoff.
Under policy 1, all four patients receive A. Its expected number of successes is \(4(0.70)=2.80\).
Under policy 2, three patients receive A and one receives B. The expected number is \(3(0.70)+1(0.50)=2.60\). Thus policy 1 has the larger value for this particular objective and these known probabilities.
The comparison is conditional on the stated assumptions and objective. There is no uncertainty to learn about in this example, and the policy cannot improve future decisions by exploring B. In an adaptive trial with unknown treatment performance, an assignment can have a different value because its outcome may change subsequent assignments.
Beliefs and the Value of Learning
When arm performance is uncertain, the decision-maker can summarize current information with a belief \(b\), such as a posterior distribution over the arms’ unknown response probabilities. After assigning arm \(a\) and observing outcome \(y\), that belief updates to \(b^{a,y}\). The expected immediate utility of assigning \(a\) under belief \(b\) is denoted \(r(b,a)=E[u_t(a,Y_t)\mid b]\).
For an additive stage-utility objective with terminal payoff \(G\), define \(V_n(b)\) as the maximum expected utility from the current point through the end of the trial when \(n\) patients remain. The terminal condition and recursion are:
The expectation inside the recursion averages over the possible outcome for the patient assigned now, using the predictive distribution under the current belief. The two terms have different roles: \(r(b,a)\) is the expected utility at this assignment, while the continuation value accounts for how the outcome changes later decisions. This is why the rule that chooses the arm with the greatest current expected utility need not maximize the finite-horizon objective.
The recursion follows by conditioning on the first assignment and its outcome. Any admissible policy must choose some arm \(a\) now. Given that choice and the resulting outcome \(Y\), the remaining problem has \(n-1\) patients and updated belief \(b^{a,Y}\). By the definition of \(V_{n-1}\), the best achievable expected continuation utility conditional on that outcome is \(V_{n-1}(b^{a,Y})\). Averaging over \(Y\), adding the immediate expected utility, and maximizing over the possible first actions gives \(V_n(b)\). At \(n=0\), no assignment remains, so only the terminal payoff \(G(b)\) can contribute.
This recursion assumes the belief contains the information needed to predict outcomes and update after an assignment. For example, a model with delayed outcomes may need to include pending outcomes in the state. The recursion also presumes a known horizon and a defined set of feasible actions. If enrollment may stop early, or allocation constraints depend on prior assignments, those features must be represented in the state or feasible-action set.
Worked Example: Two Patients Remaining Can Make Learning Worthwhile
Consider a hypothetical two-arm Bernoulli model. Arm A has known success probability \(p_A=0.50\). Arm B has an unknown success probability \(p_B\) with prior distribution \(\operatorname{Beta}(1,1)\), uniform on \([0,1]\). Outcomes are independent conditional on the arm probabilities. The objective is the expected total number of successes among the remaining patients, with no terminal payoff.
With only one patient remaining, assigning either arm has expected success probability \(0.50\): for B this is the prior mean \(E[p_B]=1/(1+1)=0.50\). No future assignment can use the result, so learning has no continuation value at this horizon.
Now suppose two patients remain. A feasible policy assigns the first patient to B, observes the outcome, and assigns the second patient to whichever arm then has the larger posterior mean. The first assignment has expected success probability \(0.50\).
If the first B outcome is a success, the posterior is \(\operatorname{Beta}(2,1)\), with mean \(2/3=0.667\); assign the second patient to B. If it is a failure, the posterior is \(\operatorname{Beta}(1,2)\), with mean \(1/3=0.333\); assign the second patient to A, whose success probability is \(0.50\). The predictive probability of either first outcome is \(0.50\), so the second patient’s expected success probability under this policy is
The policy’s expected total is therefore \(0.50+0.5833=1.0833\) successes. Assigning A to both patients yields \(2(0.50)=1.000\). The learning policy has higher expected value for this two-patient objective, although its first assignment has no immediate expected-success advantage. The gain comes from using the first outcome to choose the second assignment.
How the Horizon Changes the Value of Information
The preceding calculation can be extended to show the role of the number remaining. Under the same hypothetical model, consider a policy that assigns the first patient to B and then commits all \(H-1\) remaining patients to B after a first success or to A after a first failure. Its expected outcome per later patient is \(0.5833\), as calculated above. Its expected total over \(H\) patients is therefore \(0.50+(H-1)(0.5833)\). By comparison, assigning A to all \(H\) patients gives \(0.50H\).
For \(H=1\), the difference is zero: there is no later patient who can benefit from the information. For \(H=2\), the difference is \(1/12=0.0833\) expected successes. For \(H=5\), this particular policy exceeds always assigning A by \(4/12=0.333\) expected successes. This comparison evaluates one specified policy against another; it does not prove that the policy is optimal. It illustrates why an informative assignment may be more valuable when more assignments remain.
The result should not be read as “more remaining patients always makes exploration optimal.” The value depends on uncertainty, the possible outcomes, the differences in treatment performance, the utility assigned to outcomes, and the available alternatives. If an informative assignment has much lower immediate utility, or if the outcome is unlikely to change a later decision, its information may not repay its cost even with several patients remaining.
Worked Example: A Terminal Recommendation Has a Different Objective
Suppose a hypothetical trial has one patient and its objective is not that patient’s success, but the probability of correctly recommending the arm with the larger success probability after observing the outcome. Arm A has known \(p_A=0.50\); arm B has prior \(p_B\sim\operatorname{Beta}(1,1)\). If no information is collected, the prior probability that B is better is \(0.50\), so randomly choosing between the tied prior means gives probability \(0.50\) of selecting the truly better arm under the prior.
Assign the one patient to B. After a success, the posterior mean for B is \(2/3\), so recommend B; after a failure, it is \(1/3\), so recommend A. The prior-averaged probability of a correct recommendation is
Thus this one-patient design gives a \(0.750\) prior-averaged probability of a correct recommendation under the stated rule and model, compared with \(0.50\) without information. But the assigned patient’s prior-predictive success probability remains \(0.50\). The design’s advantage is for the terminal recommendation, not for expected success on that patient. A decision-maker must state which of these outcomes the objective values.
Comparisons, Diagnostics, and Limitations
| Approach | What it optimizes or uses | Horizon implication |
|---|---|---|
| Myopic allocation | Chooses the arm with the greatest current expected utility | Does not account for how today’s outcome may improve later choices |
| Finite-horizon optimization | Evaluates immediate utility and continuation value through the fixed end | The number remaining enters the value function and can change the preferred action |
| Fixed allocation | Uses a prespecified assignment schedule or probabilities | Does not adapt to observed outcomes, even if many patients remain |
| Terminal-decision design | Values the final recommendation or another end-of-trial criterion | Information can matter even when the current patient’s expected utility is unchanged |
Several checks help detect a mismatch between a proposed policy and its intended objective. First, verify that the horizon in the calculation is the number of assignments actually remaining, not the original planned enrollment at every stage. Second, check that the state used by the decision rule includes the information needed to predict and update outcomes. Third, test boundary cases: with no patients remaining, an assignment cannot provide future learning value; with one patient remaining and no terminal payoff, only immediate utility matters. A computed policy that violates these checks may reflect an indexing error, an omitted terminal criterion, or an incorrect state description.
A common mistake is to assume that choosing the arm with the highest posterior mean must be optimal because it has the highest expected outcome for the next patient. That reasoning ignores the continuation term. The correction is to compare the immediate expected utility plus the expected value after updating the belief, as in the finite-horizon recursion. Conversely, another mistake is to assume that any information is valuable: information that cannot change a later decision, or whose expected benefit is smaller than its cost under the objective, need not improve the objective.
Finite-horizon optimization can be computationally demanding because the number of possible histories and beliefs may grow with enrollment. Its conclusions also depend on the reward model, prior, utility definition, terminal payoff, and constraints. In a clinical trial, optimizing a narrowly specified objective does not by itself establish that the objective is clinically appropriate, that treatment effects are causally identified, or that the resulting allocation is acceptable. Those questions require separate justification.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A trial has six planned assignments. After the fourth assignment, how many patients remain for the current decision, and why does the answer matter?
- In the two-arm Bernoulli example, what is the expected total under assigning A to both patients, and how does it compare with probing B once and then adapting?
- Why can a policy that maximizes expected success for the next patient fail to maximize the expected terminal recommendation accuracy?
- State one assumption needed for the belief \(b\) to be sufficient for the finite-horizon recursion. Give one example of when the state must be expanded.
Solutions. (1) Two patients remain, counting the patient about to be assigned: \(6-4+1=3\) would instead apply before assignment four, so after assignment four there are \(6-4=2\). The correct remaining horizon determines how many later decisions can use new information. (2) Always assigning A gives \(2(0.50)=1.000\) expected successes. Probing B and adapting gives \(0.50+0.5833=1.0833\), an increase of \(0.0833\) under the stated model and objective. (3) The next-patient rule ignores the value of information for later assignments or a terminal decision. The finite-horizon objective includes continuation value, which may favor a different current arm. (4) The belief must summarize the information needed to predict outcomes and update after an assignment. With delayed outcomes, for example, pending assignments and their unobserved outcomes may need to be included in the state.