Sensitivity of Expected Enrollment to Assumed Effects

Why Vary the Assumed Effects?

Expected enrollment is not a fixed property of a MAMS design alone. It also depends on the assumed treatment effects, because those assumptions affect the probabilities that treatment arms continue at interim analyses. A design may therefore have substantially different expected enrollment under optimistic, modest, or weak-effect assumptions, even when its stage sizes and decision rules are unchanged.

A sensitivity analysis makes that dependence visible. In a one-way sensitivity analysis, vary one treatment-effect assumption while holding the others and the design fixed. In a multi-parameter sensitivity analysis, vary two or more assumed effects together. These analyses answer different questions: the first shows how enrollment responds to one assumption locally; the second shows how combinations of assumptions can alter the result.

Expected enrollment must also be distinguished from maximum enrollment. For a fixed design, changing the assumed effects changes the probabilities of reaching later stages, but it does not change the stage sizes or the largest enrollment possible. Maximum enrollment changes only if the design is recalculated—for example, if stage sizes are reselected to meet a target operating characteristic under new effect assumptions.

Expected and Maximum Enrollment for a Fixed Design

Definition: Let $A_k$ be the set of treatment arms still active at stage $k$, and let $n_k(S)$ be the number of participants enrolled at stage $k$ when the active-arm set is $S$. The expected enrollment is the sum, over stages and possible active-arm sets, of the probability of that state multiplied by its enrollment: $$ E(N)=\sum_{k=1}^{K}\ \sum_{S} P(A_k=S)\,n_k(S). $$ The maximum enrollment for the fixed design is the largest total enrollment over all feasible paths through its stages and stopping rules.

The formula counts participants enrolled at a stage only when that stage is reached and in the configuration that actually occurs. In particular, a stage can be reached with some arms stopped, so its enrollment may depend on which arms remain. If stage enrollment is the same for every possible active-arm set, the inner sum reduces to the stage enrollment multiplied by the probability that the stage is reached.

The formula follows by writing total enrollment as a sum of stage-specific contributions. For each stage $k$, let $N_k$ equal its enrollment if reached and zero otherwise. Then $N=\sum_k N_k$, so linearity of expectation gives $E(N)=\sum_k E(N_k)$. Splitting each $E(N_k)$ over the possible active-arm sets gives the stated expression. This argument does not require independence between stages or arms.

E(N_k)=\sum_S P(A_k=S)n_k(S) \quad\Longrightarrow\quad E(N)=\sum_k E(N_k).

To evaluate sensitivity to treatment effects, keep the design fixed and recalculate the probabilities $P(A_k=S)$ under each assumed effect value or combination. Those probabilities come from the design’s specified outcome model and decision rules. Shared controls, correlated test statistics, and joint stopping decisions can make the arm-continuation events dependent; in that case, use joint probabilities rather than multiplying marginal probabilities.

One-Way Sensitivity: Vary One Effect at a Time

A one-way analysis begins with a reference set of assumptions. It changes one treatment effect across plausible values while keeping the other treatment effects, outcome variability, allocation, stage sizes, decision boundaries, and all other modeling assumptions fixed. The resulting enrollment differences can then be attributed to the changed effect assumption, conditional on those fixed choices.

The following hypothetical example uses a deliberately simple design. Two experimental arms share a control. Stage 1 enrolls 90 participants. At stage 2, enrollment is 60 if both experimental arms continue and 40 if exactly one continues. If neither continues, there is no stage-2 enrollment. The fixed design’s maximum is therefore $90+60=150$. For illustration only, suppose the two arms’ continuation events are independent. In an actual shared-control design, their joint probabilities should be calculated using the appropriate dependence structure.

Worked Example: Varying One Arm’s Effect Assumption

Suppose the assumed effect for arm 2 gives a continuation probability of $p_2=0.50$. Examine three assumed effects for arm 1, represented by hypothetical continuation probabilities $p_1=0.30,\ 0.50,$ and $0.70$. These probabilities are assumed outputs of the design’s decision-rule calculations; they are not universal mappings from effect size to continuation probability.

Under the independence assumption, the probability that both arms continue is $p_1p_2$, and the probability that exactly one continues is $p_1(1-p_2)+(1-p_1)p_2$. Thus $$ E(N)=90+60p_1p_2+40\{p_1(1-p_2)+(1-p_1)p_2\}. $$

For $p_1=0.30$, the probabilities are $0.30(0.50)=0.15$ for both arms and $0.30(0.50)+0.70(0.50)=0.50$ for exactly one. Therefore, $$ E(N)=90+60(0.15)+40(0.50)=90+9+20=119. $$

For $p_1=0.50$, the corresponding probabilities are $0.25$ and $0.50$, giving $$ E(N)=90+60(0.25)+40(0.50)=90+15+20=125. $$

For $p_1=0.70$, they are $0.35$ and $0.50$, giving $$ E(N)=90+60(0.35)+40(0.50)=90+21+20=131. $$

In this illustrative sensitivity analysis, expected enrollment ranges from 119 to 131 participants as the assumed continuation probability for arm 1 changes. The maximum remains 150 in every case because the design has not changed. The direction of the trend is a feature of the supplied probabilities, not a general rule: an arm’s continuation probability can increase or decrease with its assumed effect depending on the design’s efficacy and futility rules.

Multi-Parameter Sensitivity: Vary Effects Together

When several treatment effects are uncertain, varying only one at a time can miss important combinations. For example, two effects may jointly affect the probability of reaching a stage with both arms active, or the trial may enroll quite differently when one assumed effect is strong and the other weak. A multi-parameter analysis evaluates the joint operating probabilities for each specified combination of effect assumptions.

Even in the simple independent example, expected enrollment is not just a sum of separate arm-specific effects on enrollment. Algebraically, $$ E(N)=90+40(p_1+p_2)-20p_1p_2. $$ The product term reflects that continuing both arms leads to a different stage-2 enrollment than continuing just one. With dependent decisions, the expected enrollment is still calculated from the possible joint outcomes, but the product $p_1p_2$ must be replaced by the actual probability that both continue.

Worked Example: Comparing Joint Effect Assumptions

Use the same fixed design: stage 1 enrolls 90, stage 2 enrolls 60 if both arms continue and 40 if exactly one continues. The following hypothetical effect pairs produce the stated arm-continuation probabilities under the design’s rules. Assume independence solely for this example.

Assumed effect pair	$p_1$	$p_2$
Lower effects for both arms	0.25	0.40
Moderate effects for both arms	0.55	0.70
Higher effect for arm 1, lower for arm 2	0.80	0.35

For the first pair, $P(\text{both})=0.25(0.40)=0.10$, while $P(\text{exactly one})=0.25(0.60)+0.75(0.40)=0.45$. Hence $$ E(N)=90+60(0.10)+40(0.45)=114. $$

For the moderate pair, $P(\text{both})=0.55(0.70)=0.385$, and $P(\text{exactly one})=0.55(0.30)+0.45(0.70)=0.48$. Thus $$ E(N)=90+60(0.385)+40(0.48)=90+23.10+19.20=132.3. $$

For the third pair, $P(\text{both})=0.80(0.35)=0.28$, and $P(\text{exactly one})=0.80(0.65)+0.20(0.35)=0.59$. Therefore $$ E(N)=90+60(0.28)+40(0.59)=90+16.8+23.6=130.4. $$

The moderate pair has the highest expected enrollment of these three, even though the third pair has the largest continuation probability for arm 1. The reason is that expected enrollment depends on the joint pattern of arms continuing and on the enrollment attached to each pattern. These values do not establish a general ranking of weak, moderate, and strong effects; they illustrate why the full combination must be assessed.

When Maximum Enrollment Changes

For the fixed design in the examples, the maximum is 150 under every effect assumption: both arms can continue to stage 2, and that path enrolls $90+60$. A sensitivity table that reports a changing maximum while claiming to keep all stage sizes and allocation rules fixed needs explanation. The probability of reaching the maximum may change, but the maximum itself does not.

A different question is what happens if a new effect assumption is used to recalculate the design. Stage sizes may then change to meet a target power or other operating constraints. The new design can have a different maximum, and its expected enrollment must be recalculated using its own stage sizes and stopping probabilities. Those are design-comparison results, not the effect of changing an assumption within one fixed design.

Worked Example: Comparing Recalculated Designs

Suppose a hypothetical design search produces two candidate two-stage designs that meet the same prespecified operating constraints under their respective assumed effects. The stage totals and continuation probabilities below are hypothetical design-search outputs. In each design, stage 2 enrolls the listed total if both arms continue and two-thirds of that total if exactly one continues. Assume independent continuation for this arithmetic example.

Under the first assumption set, stage 1 enrolls 90, stage 2 enrolls 72 if both continue and 48 if one continues; $p_1=0.50$ and $p_2=0.60$. The probability both continue is $0.30$, and exactly one continues with probability $0.50(0.40)+0.50(0.60)=0.50$. Thus the maximum is $90+72=162$, while $$ E(N)=90+72(0.30)+48(0.50)=90+21.6+24=135.6. $$

Under the second assumption set, stage 1 enrolls 72, stage 2 enrolls 48 if both continue and 32 if one continues; $p_1=0.60$ and $p_2=0.65$. The probability both continue is $0.39$, and exactly one continues with probability $0.60(0.35)+0.40(0.65)=0.47$. The maximum is $72+48=120$, and $$ E(N)=72+48(0.39)+32(0.47)=72+18.72+15.04=105.76. $$

The second recalculated design has both a lower maximum and a lower expected enrollment in this hypothetical comparison. The arithmetic does not prove that stronger assumed effects always permit a smaller design. That conclusion would require the design-search procedure, target operating characteristics, and all other assumptions to be specified and checked.

Diagnostics, Comparisons, and Limitations

Before interpreting sensitivity results, check what was held fixed and how the continuation probabilities were obtained. A reliable analysis should make clear whether the model, decision boundaries, allocation, stage sizes, and assumptions about dependence were fixed. If any changed, the reported difference combines multiple sources of variation.

Check joint probabilities. Shared-control statistics are often correlated. Multiplying marginal continuation probabilities is justified only under independence; otherwise calculate probabilities for joint active-arm states.
Check for non-monotonicity. Efficacy and futility stopping can pull continuation probabilities in different directions as an effect changes. Do not assume expected enrollment must rise or fall monotonically.
Check the maximum separately. For a fixed design, verify that the maximum is the total along the largest feasible enrollment path. If it varies across effect assumptions, determine whether the design was re-sized.
Check the range of assumptions. A sensitivity analysis only describes the values examined. Implausibly narrow or one-sided effect values can make the design appear more robust than it is.

One-way sensitivity analysis is easier to interpret because it isolates one assumption, but it can conceal interactions among treatment effects. Multi-parameter sensitivity analysis makes those combinations visible, at the cost of requiring more operating-characteristic calculations and careful interpretation. Averaging enrollment across scenarios answers a different question: it combines scenario-specific expected enrollments using scenario weights. A sensitivity analysis instead displays how enrollment changes under the assumptions themselves; it need not assign probabilities or weights to those assumptions.

Finally, expected enrollment is an average over repeated trials under a specified model, not a guarantee about the enrollment of a particular trial. It does not replace reporting the maximum, and a low expected enrollment does not establish that power, error control, or other design constraints are adequate. Those properties must be checked separately for each design and assumption set.

Key takeaway: For a fixed MAMS design, treatment-effect assumptions change expected enrollment through the probabilities of reaching later stages and active-arm configurations; they do not change the maximum enrollment. A maximum changes when the design itself is recalculated.

Check Your Understanding

Answer the questions, then compare your reasoning with the brief solutions.

A fixed design has maximum enrollment of 240. An effect assumption changes the probability of reaching its final stage. Can the maximum change? Explain.
In the two-arm example, let $p_1=0.40$ and $p_2=0.50$, with independence. Stage 1 enrollment is 90; stage 2 enrolls 60 if both continue and 40 if exactly one continues. Calculate expected enrollment.
Why can a one-way sensitivity analysis fail to describe enrollment when both treatment effects vary?
Under what condition is it valid to calculate the probability that both arms continue as $p_1p_2$?
A reported maximum changes across effect assumptions, but the report says the design was fixed. What should be checked?

Solutions. (1) No. The maximum is determined by the fixed design’s stage sizes and feasible paths; only the probability of attaining it changes. (2) $P(\text{both})=0.40(0.50)=0.20$; $P(\text{exactly one})=0.40(0.50)+0.60(0.50)=0.50$. Therefore $E(N)=90+60(0.20)+40(0.50)=122$ participants. (3) The joint effect combination can change the probabilities of the different active-arm states, including through interactions or dependence, so varying each effect separately may miss those outcomes. (4) When the two continuation events are independent under the assumed model. (5) Check whether stage sizes, allocation, or other design features were recalculated, and verify that maximum enrollment was computed over the same feasible paths.