Why Simulate Expected Enrollment?
A MAMS design’s maximum enrollment is fixed by its stage sizes and the assumption that every treatment arm continues. Its actual enrollment can be smaller because interim decisions may stop some arms, or stop the trial, before later stages. Expected enrollment therefore depends not only on the design but also on the assumed treatment effects and outcome variability: these determine how often each continuation decision occurs.
When decision rules are complicated, calculating all possible paths analytically can be difficult. Simulation provides a direct alternative: generate trial outcomes under a specified scenario, apply the design’s interim rules, record how many participants would be randomized, and repeat. The average of these simulated totals estimates expected enrollment for that scenario. It is not a single universal property of the design; different plausible assumptions can produce different estimates.
Specify the Scenario and the Design Before Simulating
A simulation estimates an expectation only relative to a probability model. Specify, at minimum, the scenario’s arm-specific outcome distributions and any covariate or attrition assumptions used in planning. Also specify the design: stage sizes, allocation ratios, which control data are shared, the interim statistics and decision boundaries, and the rule for stopping the trial. Apply the rules as written, including any rounding or minimum-enrollment constraints.
At each stage, generate outcomes from the scenario for the participants randomized to the active arms and control. The generated data must reproduce the dependence structure of the design. For example, two treatment-versus-control statistics that use the same control group are correlated through that shared data. Simulating each comparison as though it had a separate control group changes the decision probabilities and can change expected enrollment.
Let \(A_k\) be the set of treatment arms entering stage \(k\), and let \(I_{jk}\) equal 1 if arm \(j\) enters that stage and 0 otherwise. Suppose stage sizes are fixed: \(n_{jk}\) participants are randomized to treatment arm \(j\) at stage \(k\), and \(n_{0k}\) to a shared control group whenever at least one treatment arm continues. The total randomized enrollment on a path is
Here \(\mathbf{1}\{A_k\ne\varnothing\}\) is 1 when at least one treatment arm enters stage \(k\), and 0 otherwise. This expression counts a shared control cohort once, not once for every treatment comparison. If the actual protocol randomizes control participants under a different rule, the enrollment calculation must follow that rule instead.
Worked Example: Counting Enrollment on One Three-Stage Path
Consider a hypothetical design with two treatment arms, A and B. At stage 1, 40 participants are randomized to each treatment and 40 to shared control. If an arm continues, its stage 2 cohort and the shared control cohort each contain 30 participants; stage 3 cohorts each contain 20. On one simulated path, both arms enter stage 1, only A enters stage 2, and A alone enters stage 3.
Stage 1 enrollment is \(40+40+40=120\). At stage 2, only A and the shared control are randomized, giving \(30+30=60\). At stage 3, A and the shared control contribute \(20+20=40\).
The stage 2 and stage 3 control cohorts are each counted once. Counting 30 stage 2 controls for each treatment, for example, would incorrectly include 30 participants who were not randomized.
From Simulated Outcomes to an Enrollment Estimate
For each replicate, generate the stage 1 outcomes, calculate the interim statistics, and apply the continuation rules. Generate later-stage outcomes only for arms and participants that enter those stages. Continue until the trial ends. Record one total \(N_r\) for replicate \(r\). After \(R\) independently generated replicates under the same scenario, the estimate is
The average is the natural estimator because every \(N_r\) is generated from the same trial design and scenario. In particular, if the simulated totals are independent and each has expectation \(E(N)\), then \(E\{\widehat{E}(N)\}=E(N)\): taking the expectation of the average gives the average of the expectations, which is \(E(N)\). This justification requires the simulation to represent the intended model and each replicate to follow the same protocol. A large number of replicates cannot repair an incorrectly specified model or a wrongly implemented stopping rule.
Worked Example: Averaging Enrollment Across Four Simulated Paths
Consider a hypothetical two-stage binary-outcome design. Stage 1 randomizes 20 participants to control and 20 to each of two treatments, for 60 participants total. For each treatment, continue to stage 2 if its observed response proportion exceeds the control proportion by at least 0.10. Each continuing treatment then receives 30 stage 2 participants, and a single shared control cohort of 30 is randomized if at least one treatment continues. Thus, the possible totals are 60 if neither continues, 120 if one continues, and 150 if both continue.
The following four simulated stage 1 outcome sets are hypothetical draws under one specified scenario. “Responses” gives the number of responders out of 20 in each group. The rule is applied to the observed difference in response proportions.
| Replicate | Control responses | Treatment 1 responses | Treatment 2 responses | Arms continuing | Total enrollment |
|---|---|---|---|---|---|
| 1 | 8 | 12 | 11 | 2 | 150 |
| 2 | 9 | 13 | 10 | 1 | 120 |
| 3 | 8 | 9 | 9 | 0 | 60 |
| 4 | 10 | 11 | 13 | 1 | 120 |
For replicate 1, the control proportion is \(8/20=0.40\); the treatment proportions are \(12/20=0.60\) and \(11/20=0.55\). The differences are 0.20 and 0.15, so both treatments continue. For replicate 2, the proportions are 0.45, 0.65, and 0.50. The differences are 0.20 and 0.05, so only treatment 1 continues. For replicate 3, both treatment proportions are 0.45, compared with 0.40 for control, so neither difference reaches 0.10. For replicate 4, the proportions are 0.50, 0.55, and 0.65; only treatment 2 has a difference of at least 0.10.
The simulated estimate from these four paths is
This calculation demonstrates how to form the estimate, not a precise planning result: four paths are far too few to establish that the scenario-specific expected enrollment is close to 112.5. The estimate describes these simulated outcomes under the stated design and scenario; it does not imply that an actual trial will enroll exactly that many participants.
A Continuous-Outcome Example
The same process applies when decisions use continuous outcomes. The following hypothetical illustration assumes independent observations within and between randomized groups, normally distributed outcomes with common known standard deviation \(\sigma=10\) outcome units, and a stage 1 rule that continues the treatment if \(Z\ge 0.5\). For equal stage 1 group sizes of 25, define \(Z=(\bar{x}_1-\bar{x}_0)/\{10\sqrt{1/25+1/25}\}\), where \(\bar{x}_1\) and \(\bar{x}_0\) are the treatment and control sample means. This example uses the same control observations in the comparison and does not represent two separate controls.
Worked Example: Applying a Continuous-Outcome Interim Rule
Suppose stage 1 randomizes 25 participants to treatment and 25 to control. A continuing treatment receives 40 participants at each arm in stage 2, with no stage 2 control unless the treatment continues. Under a hypothetical scenario, one simulated stage 1 dataset gives a treatment mean 4 outcome units above the control mean. The statistic is
Since \(1.414\ge 0.5\), the treatment continues. This path enrolls 50 participants at stage 1 and 80 at stage 2, for \(N_1=130\). A second simulated dataset gives a stage 1 mean difference of \(-2\) outcome units:
Since \(-0.707<0.5\), the treatment stops and this path enrolls \(N_2=50\). The average for these two illustrative paths is \((130+50)/2=90\) participants. As in the binary example, this small average is only an illustration of the calculation; the simulated stage 1 differences are not a substitute for repeated simulation under the full scenario.
Simulation Workflow and Checks
Record the outcome distributions, treatment effects, stage sizes, allocation, shared-control structure, interim statistics, boundaries, and stopping rules.
Generate outcomes for the groups entering the first stage, calculate the prescribed interim statistics, and apply the rules. Continue this process for later stages only when the simulated decisions call for them.
Sum the randomized cohort sizes actually used on that path. Count shared control once per cohort, and use the protocol’s rule for any stopped or empty stage.
Record one total \(N_r\) for each replicate, then calculate their arithmetic mean as the scenario-specific estimate.
Useful checks catch implementation errors before interpreting the average. First, every path should respect the design’s enrollment limits: its total must not exceed the maximum possible enrollment, and it should not be below the enrollment required before its stopping point. Second, manually trace a few paths and confirm that the software applies every boundary, shared-control rule, and stage size correctly. Third, check that the generated outcomes have the intended scenario characteristics and dependence structure. For example, when an interim statistic uses a shared control group, the same generated control data must be used in each relevant comparison.
A common error is to simulate each arm’s continuation decision independently, then count a new control cohort for every treatment that continues. This can distort the joint stopping pattern and overcount enrollment. The correction is to generate the trial data jointly as required by the design, make decisions from those data, and count each actually randomized cohort once. Another error is to simulate outcomes under one treatment-effect scenario but report the average as if it applied to all scenarios. Expected enrollment is scenario-specific; repeat the calculation under each planning scenario of interest.
What Simulation Adds—and What It Does Not
If stage decisions are simple and their probabilities can be derived reliably, a direct calculation of expected enrollment can be more efficient and easier to audit. For a fixed-stage-size design, the expectation can be written as
This follows by taking expectations of the pathwise enrollment expression and using the fact that the expectation of an indicator is the probability of its event. Simulation estimates the probabilities through the observed continuation patterns without requiring a closed-form derivation. It can accommodate complicated decision rules, but it introduces simulation variability and depends on correct outcome generation and implementation. It also does not determine whether the design has adequate power, controls error rates, or is operationally feasible; those questions require their own analyses.
The main limitation is that the estimate answers only the question posed by the simulation model. If assumed effects or outcome variability are unrealistic, the estimated average may not describe enrollment in practice. Likewise, if actual randomization, attrition, or stopping behavior differs from the rules simulated, the estimate targets the wrong process. Report the scenario and enrollment assumptions alongside any result, and avoid treating a simulated average as a guaranteed recruitment total.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A stage has 25 participants in each of two continuing treatment arms and 25 in one shared control cohort. How many participants are randomized at that stage?
- Four simulated paths enroll 80, 110, 80, and 150 participants. Calculate their average total enrollment.
- Why must a simulation using two comparisons with a shared control use the same simulated control data in both comparisons?
- A replicate’s total exceeds the design’s maximum possible enrollment. What should be checked first?
Solutions. (1) The total is \(25+25+25=75\) participants; the shared control is counted once. (2) The average is \((80+110+80+150)/4=420/4=105\) participants. (3) Both comparisons use the same randomized control cohort, so their statistics are dependent through the shared observations. Generating separate controls would change that dependence and could change the interim decisions and enrollment. (4) Check the stage-size and stopping-rule implementation, especially whether shared controls were counted more than once or an arm was incorrectly included after stopping. A path total above the design maximum signals an inconsistency in the simulation or the stated maximum.