Why Planned and Realized Information Can Differ
Event-driven monitoring schedules analyses by the number of observed events, but a target event count is a planning device, not a guarantee that an analysis will occur at exactly the intended information. Events may accrue between reviews, the event process may be discrete, and the information contributed by an event depends on the risk sets and treatment allocation. As a result, an analysis may take place with more or less information than planned.
This matters because a group sequential design calibrates its boundaries jointly across looks. An information departure can change the dependence between the sequential statistics and, consequently, the probability of crossing at least one boundary. It can also change power: when the treatment effect is nonzero, more information generally makes the standardized mean farther from zero. A boundary itself is a threshold on a standardized statistic; it is not automatically a guarantee of the originally planned error rate or power after the monitoring schedule changes.
For a logrank analysis, information is the variance scale used to standardize the score, not simply the event count. With approximately equal allocation and similar risk-set balance, logrank information is often roughly proportional to the number of events. In that special case, the event-count ratio is a useful approximation to the information ratio. With unequal allocation, changing risk sets, or stratification, event counts alone may be misleading; use the variance-based information estimate for the analysis.
What Changes in the Sequential Distribution?
Under the canonical joint-distribution approximation, the standardized logrank statistics \(Z_k\) at ordered information times \(I_1<\cdots<I_K\) have unit marginal variance. Under \(H_0\), their means are zero, and their pairwise correlation is \(\operatorname{Corr}(Z_i,Z_j)=\sqrt{I_i/I_j}\) for \(i<j\). Under a local alternative with a constant favorable effect represented by \(\theta>0\), the approximate mean at look \(k\) is \(\theta\sqrt{I_k}\), while the covariance structure remains the canonical one. These approximations presume valid score standardization, suitably independent increments, and an effect model for which the information scale describes the score’s mean and variance.
Thus, a change in information has two distinct implications. First, it changes the mean under an alternative, and hence power. Second, it changes the correlations among looks if their relative information times change, and hence the overall probability of crossing one or more boundaries. Under the null, each individual \(Z_k\) is still approximately standard normal; that fact alone does not determine the probability of crossing across the whole sequence.
Why Joint Calibration Is Sensitive to Timing
The reason can be seen with two looks. Let the first and second standardized statistics have correlation \(\rho\), and let the upper boundaries be \(b_1\) and \(b_2\). Write \(\Phi_2(a,b;\rho)\) for the probability that a bivariate standard normal pair with correlation \(\rho\) is no greater than \(a\) and \(b\). The probability of crossing at least one upper boundary is
The correlation is determined by the information ratio, \(\rho=\sqrt{I_1/I_2}\). If the second look occurs later than planned while the first look stays fixed, \(I_1/I_2\) decreases. For positive upper boundaries, decreasing the correlation reduces the joint probability that both statistics exceed their respective boundaries, so the probability of their union increases. This gives a specific example of why a later final look with unchanged boundaries can increase attained Type I error. It does not imply that every information departure has the same direction of effect: an early look moved later relative to a fixed final look increases the correlation, and may reduce the union probability.
This argument assumes the two-look canonical bivariate-normal approximation and unchanged boundaries. With more looks, the same principle applies, but the attained crossing probability must be evaluated over the full realized joint distribution; it cannot be inferred from one marginal tail probability or a single information ratio.
Worked Example: A Boundary Calibrated at Planned Information
Suppose a hypothetical two-look, one-sided design plans information fractions \(t_1^*=0.65\) and \(t_2^*=1.00\). Its first upper boundary is \(b_1=2.50\), and the target overall Type I error is \(0.025\). Find the approximate second boundary under the canonical null model.
The planned correlation is \(\rho^*=\sqrt{0.65/1.00}=0.8062\). The second boundary must solve \(1-\Phi_2(2.50,b_2;0.8062)=0.025\). A numerical bivariate-normal calculation gives \(b_2\approx1.996\), or about \(2.00\) to two decimal places.
As a check on the numerical solution, using \(b_2=1.99\) gives an overall crossing probability of about \(0.0253\), slightly above the target. A boundary of \(2.02\) gives about \(0.0238\)–\(0.0239\), below the target. Therefore the calibrated value is near \(2.00\); \(1.99\) is too low for the stated target.
The boundary \(2.00\) is calibrated to the planned information ratio and the stated two-look design. It should not be interpreted as guaranteeing an overall error rate of \(0.025\) if the actual information times differ.
Worked Consequences for Error and Power
Worked Example: A Final-Look Information Overrun
Continue the hypothetical two-look design above, with the first look still at \(0.65I_K^*\), but suppose the final analysis is delayed until information reaches \(1.10I_K^*\). The original boundaries \(2.50\) and approximately \(2.00\) are retained. What changes under the null?
The realized correlation becomes \(\rho=\sqrt{0.65/1.10}=\sqrt{0.5909}=0.7687\), lower than the planned \(0.8062\). The attained error is now \(1-\Phi_2(2.50,2.00;0.7687)\), not the planned-probability calculation at correlation \(0.8062\).
Both boundaries are positive. For fixed positive thresholds, the probability of their joint upper-tail event decreases when the correlation decreases. Therefore, the probability of crossing at least one boundary increases when the correlation falls. The unchanged boundaries consequently have an attained Type I error above the value they would have at the planned correlation (allowing for the rounding of the displayed second boundary). The individual null tail probability at the final look remains approximately \(P(Z_2\ge2.00)=0.0228\); it is the joint probability across looks that changes.
The conclusion is about this specified departure: the first-look information is fixed and the final information increases. It is not a general rule that every event overrun inflates error. A departure at an earlier look can change correlation in the opposite direction.
Worked Example: How an Event Shortfall Changes Approximate Power
Consider a hypothetical equal-allocation logrank design planned for \(400\) events. Use the rough information approximation \(I\approx D/4\), where \(D\) is the event count, so planned information is \(100\). Suppose the favorable log-hazard-ratio magnitude is \(\theta=0.20\) per square root information unit and the final upper boundary is \(b=2.00\). Under the normal approximation, the final-look statistic has mean \(\theta\sqrt I\) and variance \(1\).
At the planned information, the mean is \(0.20\sqrt{100}=2.00\), so the final-look rejection probability is \(P(Z\ge2.00)=1-\Phi(2.00-2.00)=0.5000\).
Suppose monitoring instead stops at \(360\) events. The rough information is \(360/4=90\), and the standardized mean is \(0.20\sqrt{90}=1.897\). The final-look power is \(P(Z\ge2.00)=1-\Phi(2.00-1.897)=1-\Phi(0.103)\approx0.4590\).
The event shortfall reduces this approximate final-look power from \(0.5000\) to \(0.4590\), because the alternative mean moves closer to the fixed boundary. This calculation assumes the stated information approximation and effect model; it is not a full sequential power calculation when earlier looks can also reject.
Worked Example: An Event Overrun and Final-Look Power
Use the same hypothetical equal-allocation design, effect magnitude, and boundary as above, but suppose the final analysis occurs after \(440\) events. Under the rough approximation, \(I=440/4=110\). The alternative mean is \(0.20\sqrt{110}=2.098\).
The final-look rejection probability is \(1-\Phi(2.00-2.098)=1-\Phi(-0.098)=\Phi(0.098)\approx0.5390\).
Thus the information overrun raises this approximate final-look power from \(0.5000\) to \(0.5390\) while the boundary remains fixed. The calculation concerns power at the final look only. In a multi-look design, attained overall power also depends on earlier boundaries, the realized information fractions, and their joint distribution.
Responding to Departures and Diagnosing Problems
When actual information differs from plan, first report the realized information at each analysis, not just the event totals. Then assess the operating characteristics under the realized schedule using the planned boundaries. If the design’s assumptions support the canonical approximation, the joint crossing probability can be evaluated from the realized information times; power can be evaluated under the prespecified alternative. When discrete events, ties, or other features make the approximation questionable, simulation or a method suited to the actual data-generating process may be needed.
An alternative is to specify boundaries or an error-spending strategy as a function of information and calibrate them to the information times used for monitoring. The important distinction is whether that strategy was specified and valid for the monitoring procedure. Replacing a boundary after seeing an interim result simply because the observed statistic is inconvenient is not a valid recalibration. Any adaptive change must be justified by a prespecified procedure that controls the relevant error rate.
Useful diagnostics include checking whether information fractions are increasing, whether the recorded variance agrees with the event and risk-set data, and whether the realized correlation pattern is consistent with the canonical model. Large discrepancies between event-count fractions and variance-based information fractions signal that event counts are poor proxies for information. A sensitivity analysis should compare at least the planned schedule with plausible overrun and shortfall schedules, reporting both the attained crossing probability under \(H_0\) and power under the design alternative.
| Approach | What it addresses | Main caution |
|---|---|---|
| Retain planned boundaries and evaluate realized operating characteristics | Shows how the implemented schedule performs without changing thresholds | The attained error rate or power may differ from the design target |
| Use a prespecified information-based spending rule | Relates boundary calibration to the information actually used at each look | The rule and its error control must be established for the monitoring procedure |
| Reconstruct or simulate the realized monitoring process | Can assess operating characteristics when approximations or event timing are consequential | Results depend on the validity of the model and implementation |
Common Mistakes and Limits
- Equating event fraction with information fraction. This is only an approximation in settings such as balanced risk sets and equal allocation. Use the variance-based information for the logrank statistic when available.
- Assuming a standardized statistic’s marginal null distribution settles the error question. Each look can be approximately standard normal under \(H_0\), yet the probability of at least one crossing depends on the joint distribution and the monitoring schedule.
- Assuming more information always changes Type I error in one direction. The direction depends on which look moved and how the information ratios changed. Evaluate the full realized schedule.
- Reporting a recalibrated boundary without checking its joint crossing probability. A numerical boundary must be checked against the target error rate using the relevant joint distribution; rounding can matter when the target is stringent.
These calculations are approximations, not a substitute for verifying the data and analysis plan. The canonical model may be less accurate with sparse or highly discrete event information, material departures from the assumed increment structure, or a treatment effect that does not match the model used for power. The introductory material on Type I error diagnostics provides a useful context for checking calibration; here the central concern is that the information times used in those checks may differ from the planned times.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- A two-look design has planned information \(I_1=64\) and \(I_2=100\). What is the canonical null correlation between the two standardized statistics?
- At a single final look, a fixed upper boundary is \(1.96\). Under the standard-normal null approximation, does increasing information by itself change the marginal Type I error at that look?
- In a two-look design, the first information remains fixed while the second information increases. What happens to the canonical correlation, and why can this affect overall Type I error?
- With equal allocation, use \(I\approx D/4\). If the planned event count is \(320\) and the actual count is \(288\), calculate the planned and actual information and their ratio.
Solutions. (1) \(\rho=\sqrt{64/100}=0.8000\). (2) No. The marginal null distribution is still approximately standard normal, so the upper-tail probability remains \(1-\Phi(1.96)\approx0.0250\); sequential error across multiple looks is a separate joint-probability question. (3) The correlation \(\sqrt{I_1/I_2}\) decreases. The probability of crossing at least one boundary depends on the joint distribution, so the overall error can change even though each marginal null distribution is unchanged. (4) Planned information is \(320/4=80\); actual information is \(288/4=72\); the actual-to-planned information ratio is \(72/80=0.9000\), a 10% shortfall under this approximation.