Why These Calculations Need an Audit
In a group sequential logrank analysis, an event count is often used to schedule an interim look. It is tempting, then, to treat that count as if it were the information available for the test. They are related, but they are not the same quantity. Information depends on who remains at risk in each treatment group when events occur, and on how those events contribute to the logrank variance.
Errors in this bookkeeping can affect the standardized logrank statistic, the realized information fraction, and ultimately which boundary is applied at an interim look. The introductory material on updating risk sets and realized information fractions gives the monitoring context; here the emphasis is on checking the calculations that feed it.
The event count through a look is \(D=\sum_j d_j\). Unlike \(V\), \(D\) records only how many events occurred; it does not account for treatment allocation within the risk sets. The variance formula above is the usual hypergeometric tied-event variance. For untied events, \(d_j=1\), so it reduces to \(V_j=Y_{1j}Y_{0j}/Y_j^2\). A specified analysis may use another standard tie handling convention, such as a software implementation with an approximation. In that case, use that convention consistently at every look.
Why Information Is Not an Event Count
Under the null hypothesis of equal event hazards, conditional on the risk set and on \(d_j\) events at a tied event time, the number of treatment-group events has a hypergeometric variance. Its expectation is \(d_jY_{1j}/Y_j\), and its variance is \(V_j\) as defined above. This gives a direct justification for the score and information contributions.
The last factor is the finite-population correction for tied events. Information therefore depends on the sizes and balance of the risk sets, as well as on the number of events at each time. With balanced groups and single events, each event may contribute information near \(1/4\), rather than one full unit. With unequal or changing risk sets, contributions vary further. Thus even when two looks have the same number of events, they need not have the same information.
These calculations assume correctly recorded event times, group assignments, and censoring; risk sets are formed consistently from the same data snapshot; and censoring is non-informative for the event process under the analysis assumptions. Under the null, the conditional event allocation used by the logrank variance must also be appropriate. These assumptions do not repair errors in the data: a person incorrectly included in a risk set changes the expected events and the variance.
Worked Example: Three Events, Less Than One Unit of Information
Suppose a hypothetical two-group trial uses the standard tied-event variance. At the first event time, \(Y_1=30\), \(Y_0=30\), \(d=2\), and both events are in the treatment group. At a later event time, after the intervening exits have been accounted for, \(Y_1=20\), \(Y_0=28\), \(d=1\), and that event is also in the treatment group. Calculate the total event count, score, information, and \(Z\).
At the first time, the expected number of treatment events under the null is \(2(30/60)=1\). The score contribution is \(U_1=2-1=1\). The tied-event variance contribution is
At the second time, \(Y=20+28=48\), so the expected number of treatment events is \(1(20/48)=0.4167\). Thus \(U_2=1-0.4167=0.5833\). As this is a single event, its variance contribution is
There are \(D=2+1=3\) events. The cumulative score and information are \(U=1+0.5833=1.5833\) and \(V=0.4915+0.2431=0.7346\). Therefore \(Z=1.5833/\sqrt{0.7346}\approx 1.847\). With this score orientation, a positive score means more treatment events than expected under the null. The key distinction is that three events have contributed about 0.735 units of information, not three units.
Risk Sets Must Be Reconstructed at Each Event Time
The risk set at an event time consists of people still under observation and event-free immediately before that time, according to the analysis’s prespecified convention for handling events and censoring tied at the same time. It is not the randomized sample size, nor is it necessarily the risk set from the previous look. People who had an earlier event or were censored before the event time must not be carried forward. Conversely, people who are still at risk must not be dropped merely because they did not have an event at an earlier time.
For an interim analysis, this reconstruction must use the data available at the stated cutoff. Events after that cutoff cannot enter the event count, score, or variance for that look. A useful audit is to trace, for each group, how the preceding risk set changes through events and censoring before calculating the next event contribution.
Worked Example: Baseline Group Sizes Are Not the Current Risk Set
In a hypothetical trial, 20 people were randomized to each group. By a particular event time, earlier events and censoring leave 8 treatment participants and 16 control participants at risk. At this time, two participants in the control group have events. Compare the correct score and variance contribution with the calculation that incorrectly uses the original 20-per-group allocation.
Using the current risk set, \(Y=8+16=24\). The expected treatment events are \(2(8/24)=0.6667\), and the observed treatment events are zero. Therefore \(U_j=0-0.6667=-0.6667\). The tied-event variance is
If the original group sizes were incorrectly substituted, the expected treatment events would be \(2(20/40)=1\), giving an erroneous score contribution of \(-1\). The corresponding incorrect variance would be
Both errors arise from using the allocation at randomization rather than membership in the risk set at the event time. The correct values are determined by the 8 and 16 participants actually at risk immediately before the two events.
Interim Cutoffs Must Match Across Quantities
An interim cutoff defines which event and censoring records are available for that analysis. The event count, each event-time risk set, the cumulative score, the cumulative variance, and the reported information fraction must all refer to that same cutoff. Mixing values from different snapshots does not produce a valid interim statistic, even if each component was correctly calculated at its own snapshot.
A related error is to use the event-count fraction as though it were automatically the information fraction. An event-driven schedule may target a certain number of events, but the realized information fraction is based on the variance information defined for the design. If the planned and realized information differ, follow the prespecified monitoring approach; do not substitute whichever fraction gives a preferred boundary.
Worked Example: Do Not Mix Two Interim Snapshots
Consider a hypothetical trial with a planned final total of 100 events and final logrank information of 100 units. At the first data cutoff, 40 events have occurred and the cumulative score and information are \(U=4\) and \(V=16\). At a later cutoff, 50 events have occurred and the cumulative values are \(U=7\) and \(V=25\). Calculate the valid statistics and identify two invalid mixed calculations.
At the first cutoff, \(Z=4/\sqrt{16}=1.000\). At the later cutoff, \(Z=7/\sqrt{25}=1.400\). The event-count fractions are \(40/100=0.400\) and \(50/100=0.500\), while the information fractions, using the stated final information, are \(16/100=0.160\) and \(25/100=0.250\). These fractions differ because events do not each supply one unit of logrank information.
Mixing the first-cutoff score with the later information gives \(4/\sqrt{25}=0.800\), which is not the statistic at either cutoff. Mixing the later score with the earlier information gives \(7/\sqrt{16}=1.750\), also not a valid statistic for either look. The score and its variance must be calculated from the same set of event records and risk sets.
Audit Checks, Alternatives, and Limitations
The most reliable response to a suspicious statistic is to reconcile its underlying data rather than adjust the final \(Z\) value. Useful checks include:
- Reconcile events: the total \(D\) must equal the sum of the event counts \(d_j\) included through that cutoff.
- Rebuild risk sets: group-specific counts should reflect all prior events and censoring, while remaining consistent with the chosen same-time convention.
- Check variance contributions: each \(V_j\) should be nonnegative, and cumulative \(V\) should not decrease across nested, consistently calculated looks.
- Check snapshot alignment: event counts, score, variance, and the information denominator used for a fraction must have clearly documented cutoffs and meanings.
A common mistake is to reason, “The trial has reached half its planned events, so it has reached half its information.” That may be a useful planning approximation under particular assumptions, but it is not a definition. The correction is to calculate information from the logrank variance at each event time and use the information convention specified for monitoring. Another mistake is to use the original randomization ratio as the event-time allocation; the correction is to use the current risk set, which can differ after events and censoring.
Event-count monitoring is operationally convenient: events can provide a clear trigger for scheduling an analysis. Information-based monitoring is more directly connected to the variance of the logrank statistic, and therefore to its standardization and boundary calculations. Neither eliminates the need for accurate event and censoring records. Event counts alone are insufficient to reconstruct the score or information; doing so requires event-time risk sets and treatment-specific event counts. Likewise, this audit does not resolve problems caused by a misspecified event model or by informative censoring. It checks the arithmetic and data alignment on which the planned analysis depends.
Check Your Understanding
Answer each question, then check the solutions below.
- At one event time, \(Y_1=15\), \(Y_0=25\), and one treatment participant has an event. Find the score and information contributions.
- Why can two interim looks with the same number of events have different logrank information?
- A reported cumulative information value decreases between two looks using nested data and the same variance method. Name one calculation feature to investigate.
- At a cutoff, a report uses the score from that cutoff and a variance calculated from a later data freeze. Is the resulting \(Z\) a valid statistic for either look? Explain.
Solutions. (1) \(Y=40\), so \(U_j=1-15/40=0.6250\) and, for a single event, \(V_j=15(25)/40^2=0.2344\). (2) The group-specific risk sets and tied-event patterns may differ; information is a sum of variance contributions, not a count of events. (3) Check whether the cutoffs, risk-set updates, and variance conventions were kept consistent; each added event-time contribution should be nonnegative. (4) No. The score and variance must come from the same cutoff, because otherwise their ratio does not standardize the score for either analysis.