Summarizing Stopping Look and Sample Size Distributions

Why Summarize the Whole Stopping Pattern?

A group sequential design is not described completely by its maximum planned information or its overall power. Under a particular data-generating scenario, some simulated trials may stop at the first look, others may continue, and others may reach the maximum number of looks. The stopping pattern shows where trial paths end; the rejection pattern shows at which analysis the design rejects \(H_0\); and the sample-size distribution describes how much information or how many participants the design uses across paths.

These summaries answer different planning questions. A high probability of stopping early may reduce average sample-size use, but it does not by itself indicate high power: early stopping may be for efficacy or futility. Likewise, a design's average sample size does not say how often it uses close to its maximum. Each summary must be tied to the simulated scenario and to the stopping and analysis rules that generated the paths.

Path-Level Quantities and Their Summaries

Suppose the design has \(K\) planned analyses. For one simulated trial path, let \(J\) be the look at which the trial stops, so \(J\in\{1,\ldots,K\}\). The final look is an administrative stopping point: a path that has not stopped earlier ends at look \(K\). Let \(N_j\) denote the sample size at look \(j\), and let \(N_J\) be the sample size at which that path stops. If the schedule is specified in information rather than participants, the same summaries can be formed using the information at each look.

The conditional stopping probability has a different denominator from the unconditional stopping probability. At look \(j\), only paths that reach that look can stop there. A rejection at look \(j\) is also an event on a path that stops at \(j\); a path cannot reject at multiple looks if the design stops upon rejection. Consequently, analysis-specific rejection events are disjoint, and overall rejection probability is their sum.

These summaries rely on a clearly specified simulation scenario and correctly implemented design. Each simulated path must be generated under the scenario's assumptions, evaluated using the planned looks and stopping rules, and followed through to its actual stopping point. For a simulated frequency to estimate a probability, the paths should be independent replicates from the same specified scenario. Counts should include every completed path exactly once. If several scenarios are simulated, report the summaries separately for each one.

Estimating Stopping and Rejection Probabilities

For \(R\) simulated paths, let \(C_j\) be the number that stop at look \(j\), and let \(A_j\) be the number that reject \(H_0\) at look \(j\). The estimated unconditional stopping probability is \(C_j/R\). The estimated conditional stopping probability is \(C_j/\sum_{\ell=j}^{K}C_\ell\), because the denominator counts paths that reached look \(j\). The estimated analysis-specific rejection probability is \(A_j/R\), not \(A_j/C_j\). The latter is the rejection fraction among paths stopping at that look, a useful conditional description but not the probability of rejection on a randomly selected trial path.

Since each path stops once, the stopping counts must satisfy \(\sum_{j=1}^{K}C_j=R\). Since rejection at look \(j\) requires stopping at look \(j\), \(0\le A_j\le C_j\). The simulated overall rejection probability, or power under an alternative scenario, is \(\sum_{j=1}^{K}A_j/R\). Under a null scenario, the corresponding total estimates the design's Type I error probability for the specified null data-generating conditions.

Summarizing the Sample-Size Distribution

If stopping at look \(j\) entails sample size \(N_j\), the simulated distribution of \(N_J\) is obtained by assigning each path the sample size at its stopping look. With a fixed schedule of increasing sample sizes, its mean can be calculated directly from the stopping probabilities. Its median and other quantiles can be found by accumulating probability across the ordered sample sizes. A complete summary often reports the look-specific stopping frequencies, sample sizes at those looks, the mean and median sample size, and the maximum planned sample size. The mean describes average use; the maximum describes the planned upper end, not typical use.

To justify the second expression, write the sample size on a path that stops at \(J\) as successive increments. The increment from \(N_{j-1}\) to \(N_j\) is included precisely when the path reaches look \(j\), or \(J\ge j\). Thus,

Taking expectations and using \(E[\mathbf{1}(J\ge j)]=P(J\ge j)\) yields the stated expression. This form emphasizes that expected sample-size savings depend on how often paths avoid the additional sample required to reach later looks. If sample size varies between paths even at the same look, summarize the actual path-level sample sizes instead of treating \(N_j\) as a fixed value.

Interpreting, Comparing, and Checking the Summaries

Stopping distributions are most informative when the reason for stopping is retained. If efficacy and futility rules can both stop a trial, report the mutually exclusive stopping category assigned to each path, such as efficacy, futility, or reaching the final analysis without an earlier stop. The category definitions must reflect the implemented rule, including what happens if more than one criterion appears to be met at a look. Counts for mutually exclusive categories should add to the number of paths. Do not combine different reasons into a single “early stop” frequency if the distinction matters for the planning question.

Analysis-specific rejection probabilities complement stopping frequencies. A stopping table can show that many paths end early, but only the rejection table indicates how much rejection probability is contributed at each analysis. Comparing scenarios can reveal a shift in the timing of rejection or in average sample-size use, even when their overall rejection probabilities are similar. Comparisons are meaningful only when the scenarios and design rules are stated; a simulated frequency is not a universal property of the design.

Several checks can expose errors in summaries. Confirm that stopping counts sum to \(R\), that each path has one stopping look, that rejection counts do not exceed stopping counts, and that the total rejection probability agrees with the sum of analysis-specific probabilities. Check that conditional stopping denominators count paths reaching the look rather than all paths. Verify that the sample size assigned to each path matches its actual stopping look, and that reported units are clear. A discrepancy may indicate duplicated or missing paths, overlapping stopping categories, an incorrect denominator, or a mismatch between the stopping rule and the summary code.

A common alternative is to report only the maximum sample size or only the expected sample size. The maximum is essential for describing the planned upper limit, but it cannot show how often that limit is reached. The mean gives a compact measure of average use, but conceals the shape of the distribution: two designs can have the same mean and quite different probabilities of early stopping or of using nearly the maximum. Look-specific frequencies and selected quantiles add that context. These summaries do not replace the overall power or Type I error estimate; they describe different features of the design.

All simulation summaries are estimates under the assumed data-generating model and implementation. They do not establish that the assumptions are appropriate for a future trial, and they can be misleading if paths are generated or classified incorrectly. Report the scenario, number of simulated paths, look schedule, stopping categories, and the definition of sample size used. Interpret small differences cautiously, especially when they are not accompanied by the uncertainty inherent in a finite simulation.