Simulating Expected Sample Size and Trial Duration

Why Record Recruitment and Duration in Every Replicate?

Operating characteristics such as power describe what a design does statistically, but they do not by themselves show how much recruitment or calendar time the design typically requires. In a multi-arm multi-stage (MAMS) trial, interim decisions can stop the study early or drop some experimental arms while others continue. Consequently, the final number of participants and the time to the final outcome can vary from one simulated trial to another, even when the design and treatment scenario are fixed.

The key bookkeeping rule is to record incremental recruitment at each stage and then calculate the cumulative total separately for each replicate. Duration must likewise be calculated from the simulated recruitment and decision timeline, using the protocol’s assumptions about accrual and follow-up. Averaging these replicate-level quantities estimates their expected values under the specified scenario.

Definition: For replicate $r$, let $n_r$ be the total number of participants recruited over the trial and let $T_r$ be the trial duration, measured from first recruitment to availability of the final primary outcome. With $R$ simulated replicates, the estimated expected sample size and expected duration are $$ \widehat{E}(n)=\frac{1}{R}\sum_{r=1}^{R}n_r, \qquad \widehat{E}(T)=\frac{1}{R}\sum_{r=1}^{R}T_r. $$ These averages estimate expectations under the design, treatment scenario, and accrual and follow-up model used in the simulation.

Define What Is Counted Before Simulating

The sample-size measure should match the question being asked. Here, $n_r$ means the total number of participants recruited and randomized in replicate $r$. It is not the number of outcome observations available at an interim analysis, and it is not the number recruited at the last stage alone. If a protocol distinguishes randomized participants from participants with observed outcomes, those are different quantities and should be recorded separately. The examples below assume each planned participant is recruited and randomized; they do not model recruitment failure or missing outcomes.

A shared control group must be counted once, not once for every comparison using that group. For a two-stage design with two experimental arms, suppose stage 1 recruits 30 participants to control and 30 to each experimental arm. Stage 1 therefore adds $30+30+30=90$ participants. At stage 2, recruit 30 additional control participants if at least one experimental arm continues, and 30 additional participants to each experimental arm that continues. The second-stage increment is therefore 0, 60, or 90, depending on the interim decision. The maximum cumulative sample size is $90+90=180$.

Duration requires its own explicit model. For the examples, assume an aggregate accrual rate of 30 participants per month, no recruitment while stage 1 outcomes mature, and a two-month outcome follow-up after recruitment to each stage. Once stage 1 recruitment is complete, its two-month follow-up and an immediate interim review precede any stage 2 recruitment. If no arm continues, the trial ends after stage 1 follow-up. These are hypothetical assumptions, not universal MAMS rules. A protocol that permits overlapping recruitment, has a delayed review, or uses different follow-up times needs a different duration calculation.

Worked Example: Calculating Replicate-Level Totals

Consider four hypothetical replicates of the design just defined. In replicate 1 both experimental arms continue; in replicate 2 only one continues; in replicate 3 neither continues; and in replicate 4 only one continues. Calculate each cumulative sample size and duration, then estimate their averages.

Stage 1 contributes 90 participants in every replicate. Stage 2 contributes 90 when both arms continue, 60 when one continues (30 shared control plus 30 to that arm), and 0 when neither continues. Thus the cumulative totals are $180, 150, 90,$ and $150$, respectively. In particular, replicate 4 adds 60 at stage 2 and has $90+60=150$ participants overall.

Stage 1 recruitment takes $90/30=3$ months. With its two-month follow-up, the first-stage period is 5 months. If no arm continues, duration is 5 months. If one arm continues, stage 2 recruitment takes $60/30=2$ months, followed by two months of outcome follow-up, giving $5+2+2=9$ months. If both continue, stage 2 recruitment takes $90/30=3$ months, giving $5+3+2=10$ months. The replicate durations are therefore 10, 9, 5, and 9 months.

The estimated mean sample size is $$ \widehat{E}(n)=\frac{180+150+90+150}{4} =\frac{570}{4}=142.5\text{ participants}. $$ The estimated mean duration is $$ \widehat{E}(T)=\frac{10+9+5+9}{4} =\frac{33}{4}=8.25\text{ months}. $$ Both are averages across the four simulated trials, not values that every trial is expected to attain. The sample-size estimate is below the design maximum of 180 because not all replicates have both arms continuing.

Why the Mean Sample Size Depends on Continuation

The replicate average estimates expected sample size because each $n_r$ is a draw from the sample-size distribution induced by the simulation. A short derivation makes the shared-control count explicit. Let $I_j$ equal 1 if experimental arm $j$ continues to stage 2 and 0 otherwise, for $j=1,2$. Let $I_{\text{any}}$ equal 1 if at least one arm continues. With 90 participants at stage 1, 30 additional control participants whenever any arm continues, and 30 additional participants per continuing experimental arm, the total in a replicate is $$ n_r=90+30I_{\text{any}}+30I_1+30I_2. $$ Taking expectations and using linearity of expectation gives $$ E(n_r)=90+30P(I_{\text{any}}=1) +30P(I_1=1)+30P(I_2=1). $$ This argument does not assume that the two continuation indicators are independent. The shared control contribution is governed by the probability that at least one arm continues, not by the sum of the individual continuation probabilities. That distinction prevents counting the shared cohort twice.

Worked Example: Expected Sample Size From Continuation Probabilities

Suppose a hypothetical treatment scenario has $P(I_1=1)=0.60$, $P(I_2=1)=0.50$, and $P(I_{\text{any}}=1)=0.80$. These probabilities might be estimated from a large simulation or specified for an illustrative calculation. Find the expected sample size implied by them.

Using the expression above, $$ E(n_r)=90+30(0.80)+30(0.60)+30(0.50) =90+24+18+15=147\text{ participants}. $$ The expected stage 2 control contribution is 24 participants, since 30 are recruited with probability 0.80. The expected experimental contributions are 18 and 15. The result is an expectation, so it need not be a possible realized total for a single replicate; each realized total in this design is 90, 150, or 180.

For a consistency check, the stated probabilities imply $P(\text{both continue})=0.60+0.50-0.80=0.30$. Thus the probabilities of both, exactly one, and neither continuing are 0.30, 0.50, and 0.20. Direct averaging over these outcomes gives $0.30(180)+0.50(150)+0.20(90)=147$ participants, the same result.

Scenario Comparisons and Duration

Calculate $n_r$ and $T_r$ separately within each treatment scenario. Different treatment effects can change how often arms continue, and hence can change both expected recruitment and duration. Use the same design and recruitment assumptions across scenarios when making a like-for-like comparison. A duration estimate is not determined by sample size alone: recruitment rate, stage timing, follow-up, and whether recruitment overlaps with follow-up all matter.

Worked Example: Comparing Two Hypothetical Scenarios

Using the same design and timing assumptions, suppose four hypothetical replicates under scenario A have sample sizes $180, 150, 150,$ and $90$, with durations $10, 9, 9,$ and $5$ months. Under scenario B, suppose the corresponding replicate summaries are $180, 180, 150,$ and $180$ participants, with durations $10, 10, 9,$ and $10$ months. Every sample size is at most the design maximum of 180.

For scenario A, $$ \widehat{E}_A(n)=\frac{180+150+150+90}{4}=142.5\text{ participants}, \qquad \widehat{E}_A(T)=\frac{10+9+9+5}{4}=8.25\text{ months}. $$ For scenario B, $$ \widehat{E}_B(n)=\frac{180+180+150+180}{4}=172.5\text{ participants}, \qquad \widehat{E}_B(T)=\frac{10+10+9+10}{4}=9.75\text{ months}. $$ In these illustrative results, the simulated average recruitment and duration are higher under scenario B. The comparison describes these simulated scenarios; it does not establish that a treatment effect causes a particular real-world trial duration.

Checks, Alternatives, and Common Failure Modes

A basic audit should reconcile stage increments with cumulative totals in every replicate. For the specified example design, stage 1 must contribute 90; stage 2 can contribute only 0, 60, or 90; and the cumulative total must be 90, 150, or 180. A total of 270 is impossible unless the design has been changed and the new recruitment plan is stated. Check also that the shared control cohort is recorded once, that the duration clock uses a consistent start and end definition, and that each continuation decision triggers the intended recruitment and follow-up schedule.

Simulation is not the only way to obtain an expected sample size. If continuation probabilities are available, the indicator calculation above gives an analytic expectation for this simple design. If all possible decision paths and their probabilities can be enumerated, expected duration can also be calculated by weighting each path’s duration by its probability. Simulation is particularly useful when the decision process, accrual pattern, or timing is complicated, but it does not remove the need to specify those features correctly. For a simple design, an analytic calculation can be a useful independent check.

Confusing stage increments with cumulative totals. Add stage 1 and stage 2 recruitment to obtain the overall number; do not report the cumulative total as the stage 2 increment.
Counting shared control twice. If two arms continue, one shared stage 2 control cohort is still one cohort.
Using the wrong maximum. Derive the maximum from the largest permitted recruitment path under the stated design, and check every replicate against it.
Inferring duration from sample size alone. Apply the actual accrual, decision, and follow-up schedule; equal totals can take different times under different schedules.
Changing assumptions between scenarios without saying so. Keep design and timing fixed for a treatment-scenario comparison, or explicitly identify what changed.

An average also conceals the range of possible outcomes. A design with the same estimated mean sample size can have a different distribution of trial sizes and durations. The mean is useful for summarizing expected resource use, but it does not describe every possible path or guarantee that a particular trial will finish near the average.

Key takeaway: For every simulated replicate, record incremental recruitment by stage, apply interim decisions to determine which cohorts are recruited, and calculate the complete trial duration from an explicit calendar model. Average the resulting totals within each scenario, and verify that every replicate respects the design’s recruitment limits.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In the example design, what is the total sample size if exactly one experimental arm continues?
If both arms continue, what is the stage 2 increment and what is the cumulative total?
Suppose four replicate sample sizes are 90, 150, 180, and 150. What is their estimated mean sample size?
Under the stated timing assumptions, how long does a replicate take if no arm continues? Explain the components.
Why does the expected sample-size expression use the probability that at least one arm continues for the stage 2 control contribution?

Solutions. (1) Stage 1 contributes 90, and stage 2 adds 30 control plus 30 to the continuing arm, so the total is 150 participants. (2) Stage 2 adds 30 control plus 30 to each of two arms, or 90; the cumulative total is $90+90=180$ participants. (3) The mean is $(90+150+180+150)/4=570/4=142.5$ participants. (4) Stage 1 recruitment takes $90/30=3$ months and its follow-up takes 2 months, for 5 months total. (5) The shared control cohort is recruited once whenever one or more arms continue, so its expected contribution is 30 multiplied by the probability of that union event, not by the sum of arm-specific continuation probabilities.