What One Replicate Represents
A simulation of an adaptive trial must reproduce the sequence of events in the trial, not merely generate a final data set. First, participants are assigned according to the current allocation rule. Their outcomes are generated under the specified data-generating scenario. The prespecified interim rule is then applied to the data available at that point. Only after that decision is made should the next-stage assignments and outcomes be generated. The replicate ends when the protocol’s stopping condition is reached and its final decision and other trial-level results are recorded.
A replicate is one complete simulated realization of that process under one chosen scenario and one set of random-number draws. It is a possible trial path, not a summary of how the design performs in general. A useful replicate record allows an analyst to reconstruct what was assigned, what was observed, which rule was triggered, and what decision followed.
For a concrete illustration, consider a hypothetical, two-stage, two-treatment trial with treatments A and B and a binary response. In stage 1, four participants receive each treatment. After all eight outcomes are observed, the trial compares the observed response proportions. If their absolute difference is at least 0.25, all four stage-2 participants are assigned to the treatment with the larger observed proportion. Otherwise, two stage-2 participants are assigned to each treatment. At the end, a treatment meets the illustrative success criterion if its observed response proportion is at least 0.60; the trial recommends the treatment with the larger proportion if at least one treatment meets the criterion, and otherwise records “no treatment meets criterion.” A tie among qualifying treatments is recorded as a tie.
This rule is solely an example of a fully specified algorithm. A real design’s allocation, stopping, and decision criteria must come from its own protocol. In particular, the example’s final response proportion is a decision statistic, not a claim that it provides an unbiased treatment comparison after adaptive allocation.
Specify the Data-Generating Step
For this illustration, suppose each participant’s response is binary and, conditional on assigned treatment, responses are independent with a fixed probability of response. Let \(p_A\) and \(p_B\) denote those probabilities under treatments A and B in the selected scenario. These probabilities are inputs to the simulation, not quantities estimated from the one replicate. For a participant assigned treatment \(j\), where \(j\) is A or B, generate an independent random variable \(U\) uniformly distributed between 0 and 1, and define the response \(Y\) by:
Because \(P(U < p_j)=p_j\), this construction gives \(Y\sim\text{Bernoulli}(p_j)\): the response is 1 with probability \(p_j\) and 0 with probability \(1-p_j\). It also makes the simulation’s outcome-generation step explicit. Other outcome types require an appropriate model and corresponding random generation; the binary example should not be used for outcomes with different distributions.
The illustration assumes that the scenario’s response probabilities are constant across stages; that each participant’s outcome is independent of other participants’ outcomes conditional on treatment; that treatment assignment is implemented as specified; and that outcomes are available when the interim decision is due. It also assumes no missing outcomes, delayed ascertainment, or other event that changes the protocol path. If a realistic scenario includes any of these features, the replicate must represent them and the protocol’s rules for handling them. Silently treating delayed or missing outcomes as observed responses changes the simulated trial.
Here, stage-1 assignment is fixed at four participants per treatment, and stage-2 assignment is determined by the stated interim rule. That assignment policy is part of the example’s protocol. If a different protocol randomizes assignments, the simulator must also generate assignment random numbers using that protocol’s probabilities. It should not confuse random treatment assignment with the random generation of outcomes.
Why the Order of Operations Matters
The adaptation makes later assignments depend on earlier data. Thus stage-2 assignment is not a separate choice made in advance of observing the interim results: it is a function of those results. The simulation must follow that dependency.
A short justification makes the dependency clear. Write \(H_1\) for the complete stage-1 history and \(A_2\) for the stage-2 allocation. Under a deterministic prespecified rule, \(A_2=g(H_1)\) for a specified function \(g\). Consequently, stage-2 outcomes must be generated for the treatments assigned by \(g(H_1)\). Generating them first and then choosing \(A_2\) would reverse the protocol’s information flow: the adaptation could effectively use outcomes that were not yet observed when the allocation was made. For a randomized allocation rule, the same order applies, except that allocation also uses the rule’s specified random draw conditional on \(H_1\).
Store the scenario, protocol version, random-number seed or stream identifier, and the quantities needed to apply the rules.
Assign four participants to each treatment and generate their binary responses under the scenario’s treatment-specific probabilities.
Calculate each observed response proportion and use the specified comparison, including its boundary, to determine stage-2 allocation.
Assign four stage-2 participants according to the chosen branch and generate their responses under the appropriate treatment probabilities.
Calculate final treatment-specific counts and proportions, apply the success and recommendation rules, and store both the results and the path that produced them.
Worked Replicates
Worked Example: A Replicate That Triggers Adaptation
Suppose the hypothetical scenario uses \(p_A=0.50\) and \(p_B=0.65\). For this replicate, use the following independent uniform draws, listed in participant order within each treatment. A response is 1 exactly when the draw is less than the assigned treatment’s response probability.
For stage 1, the draws are \(0.12, 0.73, 0.41, 0.88\) for A and \(0.09, 0.54, 0.22, 0.81\) for B. For A, the first and third draws are below 0.50, giving two responses among four participants: \(\hat{p}_A=2/4=0.500\). For B, the first three draws are below 0.65, giving three responses among four: \(\hat{p}_B=3/4=0.750\).
The interim difference is \(|0.750-0.500|=0.250\), which meets the rule’s “at least 0.25” boundary. The larger interim proportion is for B, so all four stage-2 participants are assigned to B. This branch is determined by the observed stage-1 data, not by the scenario’s true probabilities.
For the four stage-2 participants assigned to B, suppose the draws are \(0.18, 0.72, 0.63, 0.37\). Three draws are below 0.65, so stage 2 adds three responses and one nonresponse for B. The final counts are 2 responses among 4 participants for A and 6 among 8 for B. Therefore, \(\hat{p}_A=2/4=0.500\) and \(\hat{p}_B=6/8=0.750\). Only B meets the illustrative 0.60 criterion, so this replicate records a recommendation of B. Its record should also show that 12 participants were enrolled, four stage-2 participants were allocated to B, and the interim threshold was met.
Worked Example: A Replicate That Takes the Balanced Branch
Keep the same hypothetical scenario, \(p_A=0.50\) and \(p_B=0.65\), and the same protocol. Suppose stage-1 draws for A are \(0.20, 0.61, 0.44, 0.79\), and for B are \(0.15, 0.72, 0.38, 0.91\). A has two responses, so \(\hat{p}_A=2/4=0.500\). B also has two responses, so \(\hat{p}_B=2/4=0.500\). The absolute difference is 0, which is less than 0.25; therefore, stage 2 assigns two participants to each treatment.
Suppose the stage-2 draws are \(0.32, 0.86\) for A and \(0.26, 0.77\) for B. A adds one response, and B adds one response. The final proportions are \(\hat{p}_A=3/6=0.500\) and \(\hat{p}_B=3/6=0.500\). Neither reaches 0.60, so the recorded decision is “no treatment meets criterion.” This path illustrates why the comparison in the interim rule and the final success rule must be applied separately: the first controls allocation, while the second determines the final decision.
Worked Example: Checking a Boundary and the Trial Record
Suppose a stage-1 data table contains 2 responses among 4 participants on A and 3 among 4 on B. The interim proportions are \(2/4=0.500\) and \(3/4=0.750\); their absolute difference is \(0.250\). Because the rule says “at least 0.25,” the correct branch assigns all four stage-2 participants to B. A program that sends this path to the balanced branch has implemented the condition incorrectly, for example by using “greater than 0.25” rather than “at least 0.25.”
The record should now be checked against the protocol: A has 4 participants and 2 responses; B has 8 participants after stage 2; total enrollment is \(4+8=12\); and stage-2 allocation is four to B and zero to A. If the saved record instead reports 6 participants on each treatment, its totals contradict the triggered branch even if the final decision label happens to be correct. The counts, allocation, branch label, and decision must all agree.
Recording, Diagnostics, and Failure Modes
A useful replicate record contains enough information to audit the path without storing every intermediate calculation in an opaque form. For the example, save the scenario identifier and probabilities; the participant-level treatment and response records, or an equivalent sufficient record; stage-specific numbers assigned and responding; interim proportions and branch; final proportions; total sample size; and final decision. Also retain enough information to reproduce the random draws, such as a seed together with the random-number generator and software version. A seed alone may not reproduce results if the generator or the order of random-number calls changes.
Basic checks can be made within the replicate. Confirm that each response is either 0 or 1; the number of responses cannot exceed the number assigned; stage-1 allocation is four per arm; stage-2 allocation totals four; and total enrollment matches the branch. Recalculate the interim comparison from the stored stage-1 counts and verify that the saved branch follows the exact inequality. Recalculate final proportions and confirm that the decision follows the final criterion, including any tie rule. These checks detect implementation errors; they do not establish that the scenario itself is realistic.
The most serious coding failure is to generate all outcomes and assignments in a fixed pattern that ignores the interim rule. Another is to use a generated stage-2 outcome to determine stage-2 allocation, thereby allowing future information to affect an earlier decision. Boundary conditions, strict versus inclusive inequalities, ties, and stop rules deserve explicit tests. If a protocol contains an early stop, for example, a stopped replicate must not generate participants who would never have been enrolled under that protocol.
A single replicate is also a poor place to draw conclusions about how well the design works. A recommendation of B in one path does not demonstrate that B is generally selected reliably, nor does a “no treatment” decision indicate that the design has low power. The path is conditional on both the chosen scenario and its random draws. Summarizing performance requires examining many replicates; that is a distinct task from correctly implementing one.
| Approach | What it does | Why use or avoid it here |
|---|---|---|
| One sequential replicate | Generates one path and applies every protocol rule in order. | Useful for tracing and auditing a path; insufficient for judging overall design performance. |
| Fixed-allocation simulation | Uses allocations set in advance, independent of interim outcomes. | Simpler, but it does not represent a protocol whose allocation adapts to observed data. |
| Analytic calculation | Derives a probability or expected result from a mathematical model. | Can be efficient when the model and adaptation are tractable; a sequential simulator is flexible for complex paths, but does not remove the need for correct assumptions or implementation. |
Simulation cannot repair a misspecified scenario or an ambiguous protocol. If probabilities vary over calendar time, outcomes are correlated, or response ascertainment is delayed, the simple independent Bernoulli model above is inadequate unless those features are explicitly ruled out for the intended scenario. Nor should an analyst infer that a simulation is correct merely because it runs without errors: a valid replicate must reproduce the protocol’s information flow and decision rules.
Common Pitfall
Correction: Generating hidden outcomes in advance can be harmless only if they are never allowed to affect decisions before they would be observed, and if all allocation and outcome dependence is represented correctly. It is safer and easier to audit the sequential procedure: generate the stage-1 data, apply the interim rule, then generate outcomes for the resulting stage-2 assignments. In particular, do not allocate using future outcomes or treat a fixed-allocation data set as though it had followed the adaptive protocol.
For an auditable single path, the central discipline is to keep the scenario, the protocol, and the random draws distinct. The scenario governs how outcomes are generated; the protocol governs assignments and decisions given observed data; and the draws produce the particular realization. Confusing any two can create a plausible-looking record that does not represent the intended trial.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In the illustrative rule, stage-1 proportions are 0.625 for A and 0.375 for B. Which stage-2 allocation branch is taken, and why?
- If a participant assigned to a treatment with response probability 0.40 receives a uniform draw of 0.40, what response is generated under the stated rule?
- Why must the interim rule be applied before generating stage-2 outcomes?
- What is one important difference between a single replicate and a summary of design performance?
- Name two checks that can reveal an implementation error in the replicate record.
Solutions. (1) The absolute difference is \(0.625-0.375=0.250\), so the “at least 0.25” condition is met and all four stage-2 participants go to A. (2) The response is 0 because the rule requires \(U<p_j\), and \(0.40\) is not less than \(0.40\). (3) Stage-2 allocation depends on stage-1 information; generating its outcomes first risks using future information or assigning outcomes to treatments not selected by the rule. (4) A replicate is one random path; design performance requires a summary across many paths, so one decision cannot establish an overall rate or reliability. (5) Examples include checking that stage-2 allocation totals four, recalculating the interim difference and branch from stage-1 counts, verifying final counts and proportions, and confirming that the decision follows the success threshold.