Implement Sequential Allocation In A Simulation

Why Sequential Order Matters

A response-adaptive trial simulation is not just a data set followed by an allocation calculation. Each participant’s allocation can depend on earlier assignments and on outcomes that have become available, so the order of operations is part of the design. An implementation that calculates all outcomes first and then assigns participants can accidentally let future information influence earlier decisions.

A replicate is one simulated trial under a specified data-generating mechanism and allocation rule. Within a replicate, participants arrive in sequence. At each decision, the simulation identifies the information currently available, calculates the allocation probabilities, assigns the arriving participant, and only then generates that participant’s outcome. If outcomes are delayed, assignment counts and observed-outcome counts need not agree.

Definition: Immediately before participant \(i\) is assigned, let \(N_{k,i-1}\) be the number of participants assigned to arm \(k\) among participants \(1,\ldots,i-1\). Let \(S_{k,i-1}\) be the number of successes on arm \(k\) whose outcomes are available by that decision. A pending outcome contributes to \(N_{k,i-1}\), because assignment has occurred, but it does not contribute to \(S_{k,i-1}\), because the response is not yet observed.

The distinction is essential: \(N_{k,i-1}\) describes the allocation history, whereas \(S_{k,i-1}\) describes observed outcomes. A simulator must represent both quantities when its rule uses both. It must also define when an outcome becomes available; “previous participant” does not automatically mean “previous observed outcome.”

Specify the Mechanism Before Coding

For a concrete illustration, consider two arms, A and B, and a binary outcome, with success coded as 1 and failure as 0. Suppose the allocation rule assigns scores \(q_{k,i}=(1+S_{k,i-1})/(2+N_{k,i-1})\), then normalizes the scores to obtain assignment probabilities. This is an illustrative, explicitly defined rule—not a claim that pending outcomes are failures or a posterior probability for an arm’s response rate. Its denominator uses assignments, including pending assignments, while its numerator uses observed successes only.

p_{A,i}=\frac{q_{A,i}}{q_{A,i}+q_{B,i}}, \qquad p_{B,i}=1-p_{A,i}, \qquad q_{k,i}=\frac{1+S_{k,i-1}}{2+N_{k,i-1}}.

The scores are positive, so the normalized probabilities are defined and sum to one. In a different study, the specified rule might use only participants whose outcomes are observed in both its numerator and denominator. That is a different rule and requires a different count definition. The implementation should follow the rule’s stated information set rather than quietly substituting one denominator for another.

For an outcome model, suppose a participant assigned to arm \(k\) has a hypothetical binary response \(Y_i\sim\text{Bernoulli}(\theta_k)\), where \(\theta_k\) is the arm-specific success probability. For this simple example, assume responses are independent conditional on assigned arms, the response probabilities are fixed during a replicate, and one participant’s assignment does not alter another participant’s response. More elaborate data-generating mechanisms may condition on baseline covariates or time, but those inputs and assumptions must also be specified.

A Replicate-Level Allocation Loop

A reliable loop uses one decision at a time. Baseline covariates, if present, are generated when participants arrive and can enter the allocation rule if it calls for them. The outcome is not generated until after the current assignment is made. The response is then recorded as available immediately or later, according to the simulated reporting schedule.

Initialize the replicate.
Set all arm-specific assignment counts and observed outcome counts to zero. Clear the replicate’s outcome and availability records. Any prespecified initial assignments must be implemented as part of the rule, not added by an undocumented shortcut.

Update the available history.
Before decision \(i\), mark any previously generated responses that are now available. Update observed success counts from those responses; do not count a pending response as observed.

Calculate and use the assignment probabilities.
Evaluate the specified rule using the current history. Draw \(U_i\sim\text{Uniform}(0,1)\). Assign to A if \(U_i\le p_{A,i}\), and to B otherwise. Then increase the assigned arm’s assignment count.

Generate the assigned participant’s outcome.
After assignment, draw the response from the outcome model for that assigned arm. Record its value and when it becomes available. At later decisions, use it only if it has become available.

Finish and preserve the replicate record.
After the final participant and any prespecified follow-up, save the assignment and outcome history needed to describe the simulated trial. Start the next replicate with fresh trial counts and records.

Worked Examples

Worked Example: A Pending Response Still Counts as an Assignment

Suppose participant 1 was assigned to A and had an observed success; participant 2 was assigned to B and had an observed failure; participant 3 was assigned to A but that participant’s response is still pending. Calculate the probabilities before assigning participant 4.

Before participant 4, the assignment counts are \(N_A=2\) and \(N_B=1\). The observed success counts are \(S_A=1\) and \(S_B=0\). Participant 3 contributes to \(N_A\), because the assignment is known, but not to \(S_A\), because the outcome is unavailable.

The scores are \(q_A=(1+1)/(2+2)=2/4=0.50\) and \(q_B=(1+0)/(2+1)=1/3\approx0.3333\). Therefore, \(p_A=0.50/(0.50+0.3333)\approx0.6000\), and \(p_B\approx0.4000\).

If the allocation draw is \(U_4=0.55\), then \(0.55\le0.60\), so participant 4 is assigned to A. Only after that decision is made is participant 4’s response generated. This calculation does not treat participant 3’s pending response as an observed failure; it follows the stated rule, which uses the assignment count in the denominator and observed successes in the numerator.

Worked Example: Assign First, Then Generate the Response

At a decision point, suppose \(N_A=1,S_A=1,N_B=1,S_B=0\), and no responses are pending. Calculate the next allocation probability, then implement one hypothetical assignment and outcome draw. Assume the response probabilities are \(\theta_A=0.70\) and \(\theta_B=0.40\).

The scores are \(q_A=(1+1)/(2+1)=2/3\) and \(q_B=(1+0)/(2+1)=1/3\). Thus \(p_A=(2/3)/(2/3+1/3)=2/3\), or approximately 0.6667.

Let the allocation draw be \(U=0.40\). Since \(0.40\le0.6667\), assign the participant to A. Now, and not before this assignment, generate the response. With an independent response draw \(V=0.82\), define success as \(V\le\theta_A\). Since \(0.82>0.70\), the generated outcome is \(Y=0\), a failure. If this response is available before the next allocation, increase \(N_A\) to 2 and retain \(S_A=1\). If it is delayed, increase \(N_A\) to 2 but leave \(S_A=1\) until the response becomes available.

Worked Example: Normalizing Scores Across Three Arms

For three arms, use the same score rule \(q_k=(1+S_k)/(2+N_k)\), with assignment probabilities proportional to the scores. Suppose before a decision the histories are \(N_A=2,S_A=1\), \(N_B=1,S_B=0\), and \(N_C=0,S_C=0\). Calculate the probabilities and implement a draw \(U=0.60\), using the order A, then B, then C.

The scores are \(q_A=2/4=0.50\), \(q_B=1/3\approx0.3333\), and \(q_C=1/2=0.50\). Their sum is approximately 1.3333. Normalizing gives \(p_A=0.50/1.3333=0.3750\), \(p_B=0.3333/1.3333=0.2500\), and \(p_C=0.50/1.3333=0.3750\). The probabilities sum to 1.

The cumulative intervals for \(U\) are A: \([0,0.3750]\), B: \((0.3750,0.6250]\), and C: \((0.6250,1]\). Since \(U=0.60\), the participant is assigned to B. The assigned arm’s count is then incremented. For example, if the response probability on B is 0.40 and the post-assignment response draw is \(V=0.30\), then \(V\le0.40\) and the generated outcome is a success. It enters \(S_B\) only when available under the specified reporting schedule.

Why the Order Is Correct

The sequential implementation follows directly from the conditional structure of the simulated trial. Given the history available before decision \(i\), the design determines \(P(A_i=k\mid H_{i-1})\), where \(H_{i-1}\) contains only information available at that decision. Given the assignment, the data-generating mechanism determines the response distribution. For a binary outcome, the conditional probability of an observed assignment-response sequence can be written as a product of these decision-specific factors:

P(A_1,Y_1,\ldots,A_n,Y_n) = \prod_{i=1}^{n} P(A_i\mid H_{i-1})\, P(Y_i\mid A_i,X_i),

when each response is represented at its generation time, \(X_i\) denotes any relevant baseline covariates, and the response model has the stated conditional form. If reports are delayed, the history used by the next allocation includes only responses whose availability times have passed. The factorization explains the coding order: form the history, draw the assignment from its conditional distribution, then draw the response conditional on that assignment. Generating the response first and allowing it into \(H_{i-1}\) would simulate a different, information-leaking procedure.

Implementation Checks and Failure Modes

Several simple checks can catch errors before a large simulation is run. At every decision, probabilities should be finite, nonnegative, and sum to 1 within numerical tolerance. The selected arm should agree with the draw and the stated cumulative-interval convention. Assignment counts should increase exactly once per assignment, while observed counts should change only when outcomes become available.

Using the wrong denominator. Confusing assigned participants with participants having observed outcomes changes the allocation rule. Name and store the two counts separately.
Leaking future outcomes. If an outcome is generated before its assignment or used before its availability time, the simulated policy has access to information it could not have had.
Off-by-one indexing. The probabilities for participant \(i\) use the history before participant \(i\)’s assignment. Increment the assignment count only after selecting the arm.
Failing to reset between replicates. Carrying counts, pending outcomes, or observed successes into a new replicate makes it depend on the previous simulated trial.
Inconsistent draw conventions. Define cumulative intervals and boundary handling once. With a continuous uniform draw, exact boundary ties have probability zero theoretically, but a deterministic convention still makes code behavior clear.

An implementation should also be inspected on short, hand-calculable traces like the examples above. Print the decision index, current counts, calculated probabilities, uniform draw, selected arm, generated outcome, and availability time. This makes it possible to see whether the code follows the intended chronology rather than relying only on a final summary.

Alternative Choices and Limitations

A fixed allocation design uses probabilities that do not depend on accumulating outcomes. A response-adaptive design recalculates probabilities from the history specified by its rule. Both can be simulated with the same allocate-then-generate loop; the probability calculation is what differs. Within response-adaptive rules, some use only outcomes already observed, while others also use assignment counts, covariates, or other prespecified history. These alternatives are not interchangeable: the simulation must encode the exact rule being studied.

The illustrative score rule here is deliberately simple. In the presence of delayed outcomes, its assignment-count denominator can make an arm’s score smaller while responses are pending. That behavior is a property of this defined rule, not a general recommendation for handling missing or delayed responses. A simulation’s conclusions are also conditional on its outcome model, response delays, and allocation rule. If the model omits features relevant to the intended setting, a correct sequential implementation can still represent an unrealistic trial.

Common Mistake

Common mistake: “Participant 3 has no observed outcome, so I should remove that participant from both \(N_A\) and \(S_A\).” This is wrong when \(N_A\) is defined as the number assigned to A: assignment is known even when response is pending. In the earlier calculation, the correct values are \(N_A=2,S_A=1\), giving \(q_A=2/4=0.50\), \(q_B=1/3\), and \(p_A=0.60\). If a different rule intends to count only observed outcomes in its denominator, define a separate observed-outcome count explicitly and use that rule consistently.

Check Your Understanding

Work through each question using the stated definitions. Solutions follow the questions.

Before a decision, \(N_A=3,S_A=1,N_B=2,S_B=1\). Under the two-arm score rule in this tutorial, calculate \(p_A\).
A participant is assigned to B, but the outcome is pending. Which counts change immediately, and which change when a success is later observed?
With three arms, normalized probabilities are 0.20, 0.50, and 0.30 in that order. Which arm receives a participant with \(U=0.68\)?
Explain why generating every participant’s outcome at the start of a replicate can be inappropriate for a sequential adaptive allocation simulation.

Solutions. (1) \(q_A=(1+1)/(2+3)=2/5=0.40\); \(q_B=(1+1)/(2+2)=2/4=0.50\). Thus \(p_A=0.40/(0.40+0.50)=4/9\approx0.4444\). (2) Immediately, increase \(N_B\) by 1; do not change \(S_B\). When the pending response is observed, increase \(S_B\) by 1 if and only if it is a success. (3) The cumulative intervals are arm 1 through 0.20, arm 2 above 0.20 through 0.70, and arm 3 above 0.70. Since \(0.20<0.68\le0.70\), assign to arm 2. (4) The allocation rule must use only history available at each decision. Generating and exposing future outcomes before their assignments can allow information from those outcomes to affect earlier allocation probabilities, producing a different procedure from the intended sequential design.