Scenario Grids for Sample-Size Planning

Why Plan Across Effect Scenarios?

A MAMS design is planned before the arm-specific treatment effects are known. Yet those effects influence whether arms continue at interim analyses and, in turn, how many participants are enrolled. A calculation under just one assumed effect vector can therefore give a narrow picture of expected enrollment: it may describe the global null well while missing the possibility that one treatment works and the others do not.

A scenario grid is a collection of specified arm-specific effect configurations under which a proposed design is evaluated. Its purpose is not to predict which configuration will occur. It is to reveal how design performance and expected enrollment change across plausible configurations, including patterns that a single common-effect assumption would conceal.

The effect scale must be meaningful for the outcome and estimand being planned. For example, \(\delta_j\) could be a mean difference for a continuous outcome or an absolute risk reduction for a binary outcome, with the sign chosen so positive values mean benefit. Effects on different scales should not be placed together as though they were directly comparable.

Core Configurations in a Grid

Three patterns are especially useful. In the global-null configuration, every treatment effect is zero: \(\boldsymbol{\delta}=(0,\ldots,0)\). In a global-alternative configuration, every arm has its specified beneficial effect. Often this is a common target effect, \((\delta^*,\ldots,\delta^*)\), but the term can also describe a prespecified vector of beneficial effects that differ by arm. A partial-benefit configuration has benefit for some arms but not others; for example, \((\delta^*,0,0)\) represents one beneficial arm and two null arms.

A useful grid includes more than the two extremes of “all ineffective” and “all effective.” Include configurations reflecting plausible heterogeneity: one active arm, several active arms, effects near an interim threshold, and, where relevant, effects of differing sizes. Values should be chosen for scientific and planning reasons, not just because a regular numerical sequence is convenient.

The probabilities used to evaluate a scenario also depend on assumptions beyond the effects. These include the outcome distribution, variance or event rate, stage sizes, allocation ratios, missingness assumptions, and the dependence between treatment-control comparisons caused by a shared control group. State which quantities are held fixed across the grid. If an important nuisance parameter is uncertain, it may be appropriate to evaluate more than one value of it, but that should be identified as an additional planning dimension rather than disguised as an effect scenario.

From a Scenario to Expected Enrollment

For each vector \(\boldsymbol{\delta}\), determine the probabilities of the interim decisions under the specified design and data model. The resulting expected enrollment is conditional on that scenario. In a simple two-stage design with no early trial-wide efficacy stop, let \(n_1\) be the stage-one sample size per arm, including control; let \(n_{T2}\) be the stage-two sample size added to each continuing treatment arm; and let \(n_{C2}\) be the stage-two shared-control sample size added if at least one arm continues. Let \(M\) be the number of treatment arms that continue. Then total enrollment is \(N=(K+1)n_1+n_{T2}M+n_{C2}I(M>0)\), where \(I(M>0)\) equals 1 if any arm continues and 0 otherwise.

Taking expectations under the scenario-specific distribution of \(M\) gives the calculation used below. It accounts both for the expected number of continuing treatment arms and for the chance that the shared-control cohort is enrolled at stage two.

This expression follows by linearity of expectation: the expected treatment-arm additions are \(n_{T2}E(M)\), and the expected control addition is \(n_{C2}P(M>0)\). The joint distribution of the arm decisions matters when calculating \(P(M>0)\) or \(E(M)\), especially when comparisons share a control. In more complex designs, define total enrollment for each possible decision path and take its expectation over paths; do not assume the simple two-stage expression covers early efficacy stops, varying stage sizes, or other rules it does not represent.

Summarizing Across Scenarios

A scenario-by-scenario table is usually the clearest primary summary. If a single average is also useful, assign explicit weights \(w_s\) to scenarios, where \(w_s\geq 0\) and \(\sum_s w_s=1\). The weighted expected enrollment is \(\sum_s w_s E(N\mid\boldsymbol{\delta}_s)\). These weights represent a chosen planning distribution over scenarios; they are not probabilities that the corresponding hypotheses or effects are true unless a justified probability model is explicitly being used for that purpose.

Diagnostics, Alternatives, and Limitations

Before interpreting a grid, check that every effect vector uses the same arm ordering, outcome scale, and direction; that the global-null vector is genuinely all zero; and that partial-benefit vectors include the scientifically relevant combinations of active and inactive arms. Verify that each scenario's decision probabilities form a valid distribution and that enrollment is calculated using the actual allocation and shared-control rules. Probabilities computed as if treatment-control comparisons were independent can be misleading when they share control observations.

A useful diagnostic is whether the grid reveals patterns hidden by a common-effect assumption. If all arms are assigned the same effect in every non-null scenario, the grid cannot show how enrollment behaves when only one arm continues. Also inspect scenarios near an interim decision boundary: modest changes in effects can alter continuation probabilities and expected enrollment appreciably. The introductory material on sensitivity of expected enrollment addresses how assumed effects influence the expected-size calculation; a scenario grid makes that sensitivity visible across several coordinated effect patterns.

A single design calculation at one effect vector is simpler, but answers a narrower question. A one-arm-at-a-time sensitivity analysis can identify the influence of each arm, but may miss combinations of effects that matter when shared-control decisions are dependent. A weighted average compresses results into one number, but depends on chosen weights and can hide unfavorable scenarios. For planning, these approaches are complementary: show the scenario-specific results first, then add a weighted summary if it has a clear interpretation.

A grid is not an exhaustive description of uncertainty. Its conclusions are conditional on its chosen effect values and other model assumptions, and a coarse grid can miss sharp changes in decision probabilities. Scenarios also do not by themselves establish power, error rates, or clinical usefulness; those require their own prespecified operating-characteristic criteria. Finally, expected enrollment is an average over possible decision paths, not a guarantee of the enrollment in a particular trial.

A common mistake is to call the global alternative “the one scenario in which every arm has the same effect,” then use it as the only non-null case. That choice can omit a likely and important pattern: one effective arm with the other arms inactive. The correction is to define the global alternative precisely for the planning question and include partial-benefit vectors separately. Another mistake is to treat scenario weights as probabilities that the effects are true. Unless the weights come from an explicitly justified probability model, describe them as planning weights and report the unweighted scenario results as well.