Repeated RMST Analyses at Interim Looks

Why Repeated RMST Analyses Are Dependent

A trial may estimate the treatment-control difference in restricted mean survival time (RMST) at several interim looks. Those estimates are not separate studies: earlier participants usually contribute to more than one look, and the estimates often use the same truncation horizon. Consequently, the estimated contrasts—and statistics calculated from them—are generally dependent. Treating the looks as independent can misstate the chance that at least one analysis crosses a prespecified efficacy threshold.

The target at each look should first be made explicit. For a fixed horizon \(\tau\), the RMST in group \(g\) is the expected survival time accumulated up to \(\tau\), equivalently the area under that group’s survival curve through \(\tau\). Let \(\widehat{\mu}_{g,k}(\tau)\) be its estimate at look \(k\). Define the treatment-minus-control contrast as \(\widehat{\Delta}_k(\tau)=\widehat{\mu}_{T,k}(\tau)-\widehat{\mu}_{C,k}(\tau)\). Positive values therefore favor treatment when larger survival time is beneficial. The units are time, such as months.

The RMST estimate at a look is commonly obtained by integrating the Kaplan–Meier estimate to \(\tau\). This analysis requires appropriate survival and censoring data: in particular, censoring must be independent of the event time under the assumptions of the estimator, at least conditional on variables used to account for it. At an interim look, follow-up may not extend to \(\tau\) for every participant. The estimator and its variance must then be supported by the observed follow-up and censoring pattern; an unobserved tail cannot be treated as known merely because the target horizon was prespecified.

How Shared Participants Create Covariance

A useful large-sample description writes each estimated contrast as a sum of participant-level influence contributions. These contributions represent how individual observations affect the estimate, and their variance determines the estimator’s sampling variance. The exact contributions for RMST depend on the survival estimator and censoring pattern. At two looks, shared participants contribute to both estimates, so their contributions induce covariance.

Here \(I_{gk}\) is the set of participants in group \(g\) contributing at look \(k\), \(n_{gk}\) is its size, \(O_i\) denotes participant \(i\)'s observed survival and censoring information, and \(\phi_{g,k}\) is a mean-zero influence contribution. Under independent participants and independent randomized groups, the covariance between two contrasts is approximately the sum of the within-group covariances of their shared participants’ contributions, scaled by the two sample sizes. If contributions change with follow-up, that change must be reflected in the covariance; simply counting shared participants is not enough.

A transparent special case shows the source of dependence. Suppose each look uses nested participant sets, the influence contribution for a participant is unchanged between looks, and each arm has \(m_k\) participants at look \(k\), with \(m_k<m_l\). Suppose also that the contribution variance in each arm is \(\sigma_\phi^2\), and the arms are independent. Then the shared participants are exactly the \(m_k\) participants per arm from the earlier look. The covariance is:

The covariance uses \(m_l\) in its denominator because both averages include the earlier participants, while the later average is based on the larger set. This is a simplifying approximation, not a universal RMST covariance formula. Staggered entry, unequal follow-up, changing censoring patterns, or look-specific estimator contributions can make the dependence more complicated.

Joint Null Simulation for Repeated Monitoring

A joint null simulation reproduces the complete planned monitoring process under a specified null model. Each simulated trial should generate participant-level event and censoring data, follow the planned enrollment and look schedule, apply the same RMST horizon and estimator at every look, and record whether the monitoring rule signals at any look. Repeating this process estimates the probability of at least one signal under that null model. Preserving the same participants across looks is essential: simulating each look as if it were a new independent trial removes the dependence the simulation is meant to represent.

There is an important distinction between equal RMST at \(\tau\) and identical survival curves. The null hypothesis of interest for an RMST contrast is often \(H_0:\Delta(\tau)=0\). Two groups can have equal RMST through \(\tau\) while their survival curves differ within that interval. A simulation that imposes identical survival curves represents a stronger null than equality of RMST alone. It may be a useful reference model, but it does not automatically establish the repeated false-positive risk for every survival configuration satisfying \(H_0:\Delta(\tau)=0\). The null model and any nuisance assumptions should therefore be stated, and plausible null configurations considered when the design requires protection across them.

For each simulated trial, define \(Z_k=\widehat{\Delta}_k/SE(\widehat{\Delta}_k)\), with the direction chosen so large positive values favor treatment. For a simple upper-bound rule with common critical value \(c\), a false efficacy signal occurs if \(\max_k Z_k\ge c\) under the simulated null. If \(B\) trials are simulated and \(R\) meet that condition, the estimated repeated signal probability is \(\widehat{q}=R/B\). The Monte Carlo standard error is approximately \(\sqrt{\widehat{q}(1-\widehat{q})/B}\); it measures simulation noise, not uncertainty about whether the null model or monitoring plan is appropriate.

Assessing and Calibrating a Monitoring Rule

Joint simulation can assess a prespecified rule or help calibrate one. For calibration, choose the boundary so the estimated probability of at least one null signal is at or below the desired overall level for the null models the design is intended to cover. In a simple equal-boundary example, one can simulate the joint null statistics at candidate values of \(c\), estimate \(P(\max_k Z_k\ge c)\), and select a value meeting the target. A separate set of simulations should then check the selected boundary; reusing only the simulations that selected it can make the reported performance look more certain than it is.

The simulation should also diagnose the source of unexpected results. Check that each replicate uses the intended randomization ratio, accrual and look definitions; that event and censoring times are generated with the intended dependence and distributions; and that each RMST estimate uses the same \(\tau\), estimator, and contrast direction as the planned analysis. Confirm that the standard errors are computed consistently and that the simulated joint dependence resembles the intended design. If estimates are unstable because few participants remain at risk near \(\tau\), the issue is not repaired by a larger number of simulation replicates: more Monte Carlo replicates reduce simulation error, not statistical instability in each trial.

A multivariate normal approximation using estimated marginal variances and correlations can be computationally convenient, especially when its assumptions are credible. Direct participant-level simulation can represent nonlinear estimation, event accrual, censoring, and look timing more faithfully, but it depends on the validity of its data-generating models. Neither approach is automatically robust to misspecified survival or censoring distributions. In either case, report the null model, the monitoring rule, the number of replicates, the observed crossing count, and Monte Carlo uncertainty.

Common Pitfalls and Scope

• Assuming independence across looks. Cumulative analyses reuse participants. Simulate the same evolving trial or use a justified joint covariance model.
• Changing the target without noticing. If the horizon \(\tau\) or contrast definition changes between looks, the estimates may no longer target the same quantity. Prespecify and apply the target consistently if the aim is repeated monitoring of one RMST contrast.
• Equating equal RMST with identical survival. Equality of area through \(\tau\) does not require equality of the curves. Match the simulated null to the hypothesis being evaluated.
• Confusing Monte Carlo precision with design validity. A very small Monte Carlo standard error cannot compensate for a null model that fails to represent plausible trial data.
• Ignoring follow-up near the horizon. Sparse risk sets or heavy censoring near \(\tau\) can undermine the estimate and its variance. Inspect follow-up support and estimator behavior, not just the simulated rejection count.

The purpose here is to understand and assess the dependence of repeated RMST analyses, not to turn their sequence into a final inferential procedure. The resulting joint distribution is a necessary ingredient for evaluating repeated monitoring, while the inferential interpretation of the reported sequence requires additional choices.