Building Scenario Grids for Trial Simulation

Why Build a Scenario Grid?

A trial simulation answers a conditional question: what would the trial’s results look like under a specified set of assumptions? A scenario grid makes those assumptions explicit and shows which combinations will be examined. It is not a list of possible results, and it does not assign probabilities to the assumptions. Its purpose is to make the design’s sensitivity to decision-relevant conditions visible before simulation begins.

This tutorial uses a hypothetical trial with a continuous efficacy endpoint, for which lower values are better, and a binary safety endpoint. Treatment effect is the mean reduction in the efficacy endpoint relative to control. Baseline risk is the control-arm probability of a safety event. Variability is the residual standard deviation of the continuous efficacy endpoint. Recruitment is summarized by the average number of participants enrolled per month. These factors describe distinct aspects of the trial; in particular, the safety event risk and efficacy endpoint variability need not be mathematically linked.

The earlier material on decision-relevant operating characteristics establishes why the simulation should be tied to decisions. Here the task is narrower: define a useful and coherent range of assumptions to evaluate those characteristics. A grid is only as informative as its factor definitions, value ranges, and coverage of consequential combinations.

Define Factors and Their Ranges

For each factor, specify its meaning, scale, units, and levels. Levels should ordinarily include a central, clinically or operationally plausible value and values that examine important departures from it. A boundary value probes a meaningful edge of the plausible range or a limiting case, such as no treatment effect. A stress-test value deliberately represents a difficult but interpretable condition, such as slower-than-expected recruitment. These labels do not imply that the values are equally likely.

• Treatment effect: define its direction and units. In this example, \(\delta\) is the control mean minus the treatment mean on the efficacy endpoint, so positive values favor treatment. A zero value is a useful boundary case.
• Baseline risk: define the event and the population to which the control-arm risk refers. A risk of 0.10 means an expected 10% event probability under control, not a 10% treatment effect.
• Variability: specify the outcome and the scale of its standard deviation. A larger residual standard deviation generally makes a given mean difference harder to distinguish from sampling variation.
• Recruitment: specify whether the value is an average rate, a target, or a time-varying operational assumption, along with units and the time period it covers. A rate alone does not describe every feature of recruitment, such as site start-up or seasonal changes.

Factor values must be checked both individually and in combination. Risks must lie between zero and one, standard deviations must be positive, and recruitment rates must be nonnegative. Clinical coherence matters too: a large treatment benefit paired with a particular population or risk profile should not be included merely because the combination is mathematically possible. If the factors describe separate endpoints, say so explicitly rather than implying a dependency that the model does not contain.

Why Vary Factors Together?

A grid that changes one factor at a time can be easy to read, but it can miss combinations that matter. For example, a design might perform adequately when variability is high but benefit is also large, and when benefit is small but variability is low, yet be inadequate when small benefit and high variability occur together. A factorial grid makes such combinations explicit. It also allows the team to identify interactions: a change in one factor may alter the consequence of another.

The rationale can be seen with a simple precision calculation. Suppose the continuous endpoint is analyzed by comparing two independent groups with equal sample size \(n\), and assume a common within-group residual standard deviation \(\sigma\). The estimated treatment benefit is the control sample mean minus the treatment sample mean. Because the two sample means are independent, the variance of their difference is the sum of their variances:

This justification assumes independent participants across arms, equal group sizes, and a common residual variance. Under those assumptions, holding \(n\) fixed while increasing \(\sigma\) increases the standard error in direct proportion. Thus treatment effect and variability should not be inspected only in separate, isolated scenarios: their combination determines how large the signal is relative to sampling variation.

Recruitment Assumptions Must Match the Design

Recruitment can affect trial duration, the number enrolled during a fixed window, or whether a target sample size is attainable on schedule. The grid should make clear which of these is being held fixed. If the trial has a fixed recruitment duration, different rates imply different expected enrollment. If the sample size is fixed, different rates may instead imply different recruitment durations. Treating rate and duration as if both were fixed without checking their arithmetic can create impossible scenarios.

Choose Grid Size Without Losing Important Conditions

A full factorial grid is transparent but can grow quickly. If there are \(k\) factors, with \(m_i\) levels for factor \(i\), the number of combinations is

For the four factors with three levels each, \(N_{\text{scenarios}}=81\). Adding a fifth factor with four levels would increase the count to \(81\times4=324\). This arithmetic is useful before simulation because it makes the computational and review burden visible.

When a full grid is not practical, reduce it deliberately. Preserve the central plausible scenario, clinically meaningful boundary conditions, and combinations that challenge the decision. A reduced grid might pair a small benefit with high variability, or slow recruitment with a fixed enrollment window. Do not remove combinations simply because they are inconvenient or because each factor looks acceptable in isolation. If a combination is impossible or clinically incoherent, document the reason and omit it rather than allowing it to appear as an unexplained gap.

The main alternative, a one-factor-at-a-time set of scenarios, varies each factor around a reference scenario while holding the others fixed. It can efficiently show local sensitivity to individual factors, but does not reveal all interactions. A full factorial grid covers every specified combination but may be unnecessarily large. A selected, decision-focused grid lies between these approaches: it is smaller than the full factorial design while retaining combinations with a clear clinical or operational rationale. No approach makes the selected values representative of their probability in practice unless a separate probability model is specified.

Diagnostics, Pitfalls, and Limitations

Before using a grid, check its definitions, arithmetic, and coverage. A useful review asks whether every factor has units and a stated reference, whether boundary and stress-test values have a reason, whether the combinations are feasible, and whether the recruitment rate agrees with the trial’s duration or target enrollment. It should also check whether an apparent factor is redundant with another assumption or whether a key dimension has been omitted.

• Unclear treatment-effect direction: A value of 4 is meaningless without a scale and direction. State whether positive values favor treatment and give the endpoint units.
• Inconsistent recruitment constraints: A scenario cannot both enroll 120 participants in 20 months at 6 per month and achieve a target of 180 without another stated mechanism. Keep the rate, duration, and enrollment target consistent.
• Impossible or incoherent combinations: Mathematical bounds are necessary but not sufficient. A valid probability does not automatically make the assumed population or endpoint relationship plausible.
• Calling scenario levels probabilities: Three levels do not imply equal likelihood. The grid is a set of conditions for evaluation, not a probability distribution over the true state of the world.
• Relying only on central assumptions: A central scenario cannot establish robustness. Examine boundary and stress conditions that could change the decision.
• Assuming one-factor-at-a-time checks cover interactions: They do not. Explicitly include combinations where multiple unfavorable assumptions occur together if those combinations are decision-relevant.

A scenario grid is necessarily finite. It can miss conditions between its chosen levels, omit relevant factors, or include assumptions that later prove implausible. Its conclusions are therefore conditional on the specified scenarios. A grid also does not, by itself, determine which conditions are most likely; assigning probabilities or propagating parameter uncertainty requires additional modeling. Those limitations are reasons to document the rationale and boundaries of the grid, not reasons to treat it as a prediction.