Sensitivity to Interim Timing

Why Interim Timing Changes the Rule’s Behavior

A promising-zone rule uses the interim treatment-effect estimate to decide whether to increase the planned sample size. Moving the interim analysis earlier or later changes how noisy that estimate is. Even when the true treatment effect, zone cutoffs, and sample-size limits stay the same, the chance of landing inside the zone can therefore change. An early look generally gives a noisier estimate, but it does not follow that the probability of entering the zone must always increase or decrease: the result depends on where the true effect lies relative to both cutoffs.

A useful timing comparison isolates this one design choice. Keep the effect assumptions and sample-size limits fixed, vary the interim information fraction, and recalculate the probability of a zone decision and its consequences for expected sample size. These are operating characteristics of the re-estimation rule. They are not, by themselves, the probability of rejecting the null hypothesis or a demonstration of type I error control.

Model and Assumptions for a Timing Comparison

Let \(\Delta\) denote the true treatment effect, measured in the outcome’s units, and let \(\hat{\Delta}_t\) be its interim estimate at information fraction \(t\). For the calculations below, use the normal approximation \(\hat{\Delta}_t\sim N(\Delta,SE_t^2)\), where \(SE_t=SE_0/\sqrt{t}\), and \(SE_0\) is the standard error at the original planned final sample size. This relation assumes information accumulates proportionally with sample size and that the variance used in planning is fixed. For example, it follows for a difference in means with known common variance and balanced allocation when the number of participants is proportional to information.

Assume the rule expands the sample when \(L\leq\hat{\Delta}_t\leq U\), where \(L\) and \(U\) are fixed effect-scale cutoffs with \(L<U\). If the estimate is outside that interval, participants continue to the original planned total \(N_0\); inside it, the total is increased to a fixed maximum \(N_{\max}\). Thus the comparison does not include early stopping, changing the zone cutoffs, or changing either sample-size limit. The estimate is assumed to follow the stated normal model at each timing, and the timing comparison holds \(\Delta\), \(SE_0\), \(L\), \(U\), \(N_0\), and \(N_{\max}\) fixed.

This is an operating-characteristic calculation for the zone decision. If the variance is estimated with substantial uncertainty, information does not accrue proportionally to sample size, or the interim estimate has a different distribution, the normal model may be inadequate. Those features should be represented in an appropriate design-level calculation or simulation rather than silently treated as satisfying these assumptions.

Probability of Entering the Zone and Expected Sample Size

Standardizing the zone boundaries gives a direct expression for the entry probability. Because \(\hat{\Delta}_t\) is normal with mean \(\Delta\) and standard deviation \(SE_0/\sqrt{t}\), the probability is the standard normal probability between the standardized cutoffs:

where \(\Phi\) is the cumulative distribution function of a standard normal random variable. Once the entry probability is known, the sample-size calculation follows from the two possible totals. The total is \(N_{\max}\) with probability \(P(\text{zone}\mid\Delta,t)\), and \(N_0\) otherwise. Therefore:

This expression concerns the final total under the stated rule. It does not mean every participant is recruited at the interim: recruitment continues after the interim in both cases, with the larger continuation used only after zone entry.

Why the Effect’s Location Matters

The standardized interval changes at both ends as timing changes. A later look reduces the standard error, so the standardized cutoffs move farther from zero in the direction determined by their distance from \(\Delta\). The resulting probability depends on both distances, \(L-\Delta\) and \(U-\Delta\). It is not generally a monotone function of \(t\). In particular, a true effect near the middle of the zone can have a high chance of remaining inside it with a precise estimate, while a noisy estimate can either spread probability into the zone or spread it out of the zone. For a true effect outside the zone, precision may instead make estimates more consistently fall on that same outside side.

The next example holds the effect and all other design assumptions fixed across the timing comparison, and shows why an intermediate timing need not behave like a simple step between early and late looks. The sample-size expectation uses the same \(N_0=200\) and \(N_{\max}=300\) as above.

Diagnostics, Alternatives, and Limitations

A clean sensitivity analysis changes the information fraction while holding the other inputs fixed. Check that the interim standard error follows the assumed information relationship; that the effect, cutoffs, and sample-size limits are identical across rows; and that the expected total lies between \(N_0\) and \(N_{\max}\). If a calculation yields an expected total outside those limits, or a zone probability outside \([0,1]\), there is an arithmetic or implementation error. Also inspect the standardized endpoints: if they have been calculated using the wrong sign or the wrong standard error, the resulting probability can look plausible while being wrong.

In a real design, compare analytic calculations with simulation when the normal approximation, known variance, or proportional information assumption is questionable. Simulation can represent the planned data-generating model, interim timing, actual re-estimation rule, and sample-size caps together. Monte Carlo uncertainty should be reported when simulated operating characteristics are used. A conditional-power rule is another alternative: unlike fixed effect cutoffs, it bases the decision on a conditional probability of the final result under specified assumptions. That rule can also be timing-sensitive, but it is a different decision rule and should not be substituted into a fixed-zone calculation without changing the design being evaluated.

Zone-entry probability and expected sample size do not determine power. To evaluate the probability of a final rejection, one must specify the final testing procedure and calculate its behavior under the adaptive design. In particular, one should not assume that a conventional fixed-sample final test automatically retains its nominal type I error after the sample size depends on an unblinded interim estimate. That inferential question requires separate evaluation; a timing comparison of expansion probabilities does not resolve it.

Finally, the normal calculations here are conditional on a specified true treatment effect. They do not account for uncertainty about which effect is the right planning value, variance misspecification, deviations from the assumed information growth, or operational delays and recruitment constraints. Sensitivity to timing is useful because it reveals how the rule behaves under its stated assumptions, not because it removes those limitations. Report the assumptions and the rule’s full operating characteristics together.