Cumulative Test Statistics at Interim Looks

What Is Cumulative at an Interim Look?

At an interim look, investigators have data from participants or units enrolled so far. The test statistic at that look is ordinarily calculated using all data accrued to date, not just the observations added since the preceding look. This distinction matters: the new observations contribute fresh information, but the cumulative statistic summarizes the evidence in both the old and new observations.

Suppose the monitoring schedule has looks at cumulative sample sizes \(n_1<n_2<\cdots<n_K\). Let \(n_k\) denote the total number of observations available at look \(k\), and let \(m_k=n_k-n_{k-1}\) denote the number added since look \(k-1\), with \(n_0=0\). The \(m_k\) observations form the new data block at look \(k\). A cumulative statistic uses observations \(1,\ldots,n_k\); a block statistic uses only observations \(n_{k-1}+1,\ldots,n_k\).

The cumulative statistic is the one associated with the test at that look. The new-block statistic describes the latest data alone. The cumulative statistic may incorporate the block statistic, but it is not generally equal to it, nor is it generally the unweighted sum of earlier and new statistics. The monitoring schedule and the rule for comparing look-specific statistics with thresholds are separate from the definition of the statistic itself.

A Normal-Mean Model and the Update Identity

To make the distinction precise, consider independent observations \(X_1,X_2,\ldots\) from a normal population with known standard deviation \(\sigma\). The monitoring question is whether the population mean exceeds a specified null value \(\mu_0\):

Assume the observation distribution, independence, known \(\sigma\), and the ordered look sizes are appropriate for the analysis. At look \(k\), the sample mean \(\bar{x}_k\) is computed from all \(n_k\) observations. The cumulative one-sided Z statistic is

For the new block at look \(k\), let \(\bar{x}_{B,k}\) be its mean. Its null-centered standardized statistic is

The block statistic is defined here on the same scale and against the same null value as the cumulative statistic. It is not the cumulative test at look \(k\). Since the cumulative sample mean is a weighted average of the earlier and newly added observations, the two statistics satisfy an exact update identity.

For the first look, \(n_0=0\), so the statistic is calculated directly from the first block. At later looks, the first term carries forward the contribution of the observations already collected, while the second term adds the contribution of the new block. The weights depend on the relative block sizes. This identity is an algebraic consequence of combining the data; it does not say that a cumulative statistic is the same as a block statistic.

Calculating from the Accumulated Data

The update identity is useful for seeing how a new block contributes, but the direct cumulative calculation is often the safest check: combine all observations available at the look, calculate their mean, and use the cumulative sample size in the standard error. Both methods should agree when the model and inputs are the same.

Why Look Statistics Need Not Move in One Direction

A larger cumulative sample does not guarantee a larger test statistic. New observations may support the alternative, oppose it, or be close to the null. Consequently, the cumulative sample mean and its standardized statistic can move either way. Sample size affects the standard error, but the observed effect estimate also changes as data accrue.

Diagnostics, Alternatives, and Limitations

A useful calculation check is to compute the cumulative statistic in two ways: directly from the combined data and by updating the previous statistic with the new-block contribution. A disagreement usually signals a bookkeeping or arithmetic error. Check that the cumulative sample size is the sum of all block sizes, that the cumulative mean weights each block by its size, and that the standard error uses the cumulative sample size.

The main tempting alternative is to use only the latest block's statistic as though it were the cumulative test. That discards the earlier observations and answers a different question. A block-only analysis may be useful for describing what the newest data show, but it should not silently replace the prespecified cumulative analysis. Conversely, simply repeating the first-look statistic at later looks ignores the new data. Neither shortcut produces the cumulative statistic defined here.

The simple identity shown here relies on a one-sample normal-mean model with known \(\sigma\), independent observations, and the same null value used for each block and cumulative calculation. With an estimated standard deviation, a different endpoint, or a more complex model, the test statistic and its updating calculation must be derived for that setting; this formula cannot be assumed to apply unchanged. The definition of cumulative data remains straightforward, but the exact arithmetic may differ.

Finally, a cumulative statistic is not itself a calibrated repeated-testing procedure. Repeatedly comparing it with an ordinary single-look cutoff can inflate the overall type I error rate. The earlier material on repeated-testing thresholds explains why decision boundaries require calibration. Here the central task is to identify correctly which data enter each look-specific statistic.

• Wrong: “The look-2 statistic is the statistic from the 50 new observations.” Correction: That is a block statistic; the look-2 cumulative statistic uses all observations through look 2.
• Wrong: “The cumulative statistic must increase because the sample size increased.” Correction: The new block can move the estimate toward or away from the null, so the statistic can rise or fall.
• Wrong: “I can add the two Z statistics directly.” Correction: Their contributions have sample-size-dependent weights; use the derived update identity or recompute from the combined data.