Efficacy Monitoring Before and After Effect Onset

Why the Timing of an Interim Look Matters

A group sequential efficacy statistic is interpreted against a boundary designed to control the probability of a false efficacy declaration across repeated looks. With a delayed treatment effect, however, the timing of a look also determines how much of the accumulated score could reflect benefit. Before the effect begins, the treatment and control hazards may be equal: events still contribute information and random variation, but they do not produce a positive expected score for treatment benefit. After onset, the expected score can begin to favor treatment, provided the chosen weights give those later events appropriate influence.

Thus, an early statistic that does not cross an efficacy boundary is not automatically evidence that the treatment will not help. It may simply have been observed before benefit could emerge. The interpretation changes if the statistic is sufficiently far in the opposite direction, but even then a formal conclusion about harm requires an appropriate prespecified inferential rule.

Model, Score, and Assumptions

Let $T$ denote the treatment arm and $C$ the control arm. At time $t$, let $Y_T(t)$ and $Y_C(t)$ be the numbers at risk, with $Y(t)=Y_T(t)+Y_C(t)$. Let $N_T(t)$ and $N_C(t)$ count observed events through time $t$, and let $N(t)=N_T(t)+N_C(t)$. The hazards are $\lambda_T(t)$ and $\lambda_C(t)$. For the delayed-effect model considered here, assume $\lambda_C(t)=\lambda_0(t)$, while $\lambda_T(t)=\lambda_0(t)$ before a prespecified onset time $\tau$ and $\lambda_T(t)=\theta\lambda_0(t)$ after $\tau$, where $0<\theta<1$ describes a beneficial post-onset treatment effect.

The interpretation below relies on randomized treatment assignment, independent censoring conditional on treatment arm (and any factors used in the analysis), and event and censoring processes that are appropriately represented by the analysis. The chosen weight $w(t)$ is fixed in advance and predictable from the observed history; for a straightforward positive-benefit interpretation, take $w(t)\geq 0$. These conditions support the usual weighted logrank score and its null variance approximation. If the weight, onset time, or analysis population is changed after inspecting interim outcomes, the stated sequential calibration need not apply.

Definition: The weighted logrank score favoring treatment benefit through time $t$ is $$ U(t)=\int_0^t w(s)\left\{\frac{Y_T(s)}{Y(s)}\,dN(s)-dN_T(s)\right\}. $$ Its usual null variance estimate is $$ V(t)=\int_0^t w(s)^2\frac{Y_T(s)Y_C(s)}{Y(s)^2}\,dN(s). $$ When $V(t)>0$, the standardized statistic is $Z(t)=U(t)/\sqrt{V(t)}$. With this sign convention, positive values favor treatment because fewer treatment events than expected under the null make $U(t)$ positive.

At each event time, $Y_T/Y$ is the treatment fraction of the risk set. Under the null hypothesis of equal hazards, it gives the expected fraction of events assigned to the treatment arm, conditional on one event occurring in that risk set. The score adds the weighted difference between that expected treatment count and the observed treatment count. A common weight is $w(t)=1$, giving the ordinary logrank score; a prespecified time-varying weight can give relatively greater influence to events in selected time regions.

Why the Expected Score Is Different Before and After Onset

The key distinction is in the expected score increment, not in whether events occur. Conditional on the risk set just before time $t$, a short-interval approximation gives $E[dN_T(t)]\approx Y_T(t)\lambda_T(t)\,dt$ and $E[dN(t)]\approx [Y_T(t)\lambda_T(t)+Y_C(t)\lambda_C(t)]\,dt$. Substituting both quantities into the score increment gives

\begin{aligned} E[dU(t)\mid\text{risk set}] &\approx w(t)\left[ \frac{Y_T(t)}{Y(t)} \{Y_T(t)\lambda_T(t)+Y_C(t)\lambda_C(t)\} -Y_T(t)\lambda_T(t)\right]dt\\ &=w(t)\frac{Y_T(t)Y_C(t)}{Y(t)} \{\lambda_C(t)-\lambda_T(t)\}\,dt. \end{aligned}

The risk-set factor $Y_TY_C/Y$ is essential: the null expected treatment count is the treatment fraction of the total expected event count, not the treatment expected count alone. Before onset, $\lambda_T=\lambda_C$, so this expected increment is zero. After onset, $\lambda_T=\theta\lambda_C$, and the increment becomes

E[dU(t)\mid\text{risk set}] \approx w(t)\frac{Y_T(t)Y_C(t)}{Y(t)} (1-\theta)\lambda_C(t)\,dt.

For nonnegative weights and $\theta<1$, the post-onset expected increment is nonnegative. This does not mean every observed increment is positive: random variation can move the score either way. It means that, under the stated model, the score’s expected direction can favor treatment only where the hazards differ and the weight gives those events influence.

The variance can accumulate both before and after $\tau$. In particular, under the null, its short-interval expectation is approximately $w(t)^2\{Y_T(t)Y_C(t)/Y(t)\}\lambda_0(t)\,dt$. Consequently, a pre-onset look can have a nonzero standard error and a well-defined $Z$-statistic even though the delayed-effect model gives it no positive expected score drift. An early look is not devoid of data; it may be poorly timed to reveal the particular benefit being anticipated.

Worked Interpretations at Interim Looks

Worked Example: A Look Before the Effect Begins

Hypothetical data. Suppose the prespecified onset is at 12 months and an interim look occurs at 8 months. The weight is $w(t)=1$. At each of four observed event times, 100 people are at risk, split evenly between arms, and exactly one event occurs. Across the four events, one is in the treatment arm and three are in control.

For each event, the null expected treatment count is $Y_T/Y=50/100=0.5$. The total null expected treatment count is therefore $4(0.5)=2$, while the observed treatment count is 1. The score is

$$ U=2-1=1. $$

Each event contributes $w^2(Y_TY_C/Y^2)=1(50\cdot50/100^2)=0.25$ to the variance estimate. Thus $V=4(0.25)=1$, and $Z=1/\sqrt{1}=1.00$. The one-sided standard normal p-value for benefit at this single look is approximately $1-\Phi(1)=0.1587$.

This statistic does not cross a hypothetical efficacy boundary of 2.40. More importantly, under the delayed-effect model the hazards are still equal at this look, so a positive score of this size can arise from chance variation before the anticipated benefit begins. It neither establishes benefit nor rules out benefit after 12 months. The p-value is a single-look reference value, not a sequentially adjusted p-value.

Worked Example: Checking the Post-Onset Drift With Unequal Risk Sets

Hypothetical calculation. Just after onset, suppose $Y_T=40$, $Y_C=60$, $Y=100$, the control hazard over a short interval is $\lambda_C(t)\,dt=0.10$, the post-onset hazard ratio is $\theta=0.60$, and $w(t)=1.5$. The expected treatment event count is $40(0.60)(0.10)=2.40$, and the expected control event count is $60(0.10)=6.00$. Hence the expected total is 8.40.

Under the null allocation, the expected treatment count corresponding to this total is $(40/100)(8.40)=3.36$. The expected score increment is therefore $1.5(3.36-2.40)=1.44$. The same result follows from the risk-set formula:

1.5\left(\frac{40\cdot60}{100}\right)(1-0.60)(0.10) =1.5(24)(0.40)(0.10)=1.44.

For comparison, the corresponding null-variance increment in expectation is approximately $1.5^2(40\cdot60/100)(0.10)=5.40$. This is a variance contribution, not a positive score drift. The unequal risk sets matter: using $Y_T$ alone in place of $Y_TY_C/Y$ would give the wrong expected score increment.

Worked Example: Crossing a Prespecified Efficacy Boundary

Hypothetical interim result. At a later look, suppose the cumulative weighted score is $U=4.50$ and its variance estimate is $V=2.25$. The design’s one-sided efficacy boundary for this look is $b=2.80$, calibrated as part of the group sequential design under the null.

The standardized statistic is

Z=\frac{4.50}{\sqrt{2.25}}=\frac{4.50}{1.50}=3.00.

Because $3.00>2.80$, this look crosses its prespecified efficacy boundary. Under the design’s assumptions and calibration, the trial meets its efficacy stopping rule at this look. The ordinary single-look one-sided normal p-value is approximately $1-\Phi(3.00)=0.00135$, but the boundary-crossing decision—not that unadjusted p-value by itself—is the relevant sequential conclusion. The statistic is evidence favoring treatment under the specified weighted logrank analysis; it does not by itself describe the size of a clinically meaningful benefit.

Worked Example: A Negative Statistic Is Not Automatically a Harm Conclusion

Hypothetical interim result. Suppose another look gives $U=-1.80$ and $V=1.44$. Then $Z=-1.80/\sqrt{1.44}=-1.50$. The statistic is in the direction of more weighted treatment events than expected under the null allocation, so its direction is contrary to treatment benefit.

For the one-sided benefit alternative $Z>0$, the single-look p-value is $1-\Phi(-1.50)=0.9332$; this is not evidence for benefit. If a one-sided harm test had been prespecified, the corresponding nominal lower-tail p-value would be $\Phi(-1.50)=0.0668$, which does not meet a 0.05 threshold. A design that specified only an upper efficacy boundary has not thereby established a formal harm rule. In particular, failure to meet the efficacy boundary is not equivalent to rejecting benefit or declaring harm.

Sequential Interpretation, Diagnostics, and Limitations

For repeated looks, the $Z$-statistics are correlated because each cumulative statistic includes earlier events. Prespecified alpha-spending boundaries account for this repeated testing under the null; the appropriate efficacy decision compares the current $Z_k$ with its current upper boundary $b_k$, rather than comparing every look with an ordinary fixed-sample cutoff. The information time and boundary calibration must correspond to the planned monitoring design. A nominal p-value at one look does not replace this comparison.

For delayed effects, it is also useful to identify which event-time periods contribute to the score accumulated at each look. If a look precedes $\tau$, the delayed-effect model predicts no treatment-control hazard difference in that period. If a look falls after $\tau$, its cumulative score still includes the pre-onset contribution, whose expected drift was zero under the model, as well as the post-onset contribution. The overall statistic should not be described as if all accumulated information arose after benefit began.

Before interpreting a statistic, check that the observed risk sets and event times are consistent with the analysis, that censoring and follow-up have not created an important departure from the assumed comparison, and that the specified weight and onset time were retained. Sparse risk sets, few events, or a near-zero variance estimate can make a normal approximation unreliable or $Z$ unstable. Unexpected weight behavior, a different plausible onset time, or evidence that censoring is informative should prompt careful review rather than an unqualified boundary interpretation.

The weighted logrank statistic targets a weighted comparison of event experience; it is not a direct estimate of a constant treatment effect when hazards change over time. Its sensitivity depends on the weight, event timing, risk sets, and actual onset. If the effect begins later than anticipated, a look expected to show post-onset separation may still contain little such evidence. If the effect begins earlier, more of the accumulated score may reflect benefit. The introductory material on timing interim looks after a delayed effect motivates aligning looks with the expected onset; the present interpretation still requires reporting what the observed score and its design-specific boundary actually show.

A useful distinction is between three statements: “the efficacy boundary was crossed,” “the efficacy boundary was not crossed,” and “the data provide evidence in the opposite direction.” Only the first is an efficacy stopping conclusion under the specified upper-boundary rule. The second is a non-crossing result, not proof of no benefit. The third requires attention to the direction and magnitude of the statistic and to whether a formal lower-tail rule or harm test was prespecified.

Common Mistake

Common mistake: “The early $Z$-statistic did not cross the efficacy boundary, so the treatment has no benefit.” This reasoning confuses lack of an efficacy declaration at one look with evidence that the treatment effect is absent. Before a delayed effect begins, the expected score drift is zero even under the delayed-benefit model; an early look can therefore be uninformative about that later benefit. The correction is to report the boundary result accurately, relate the look to the prespecified onset and weight, and reserve a conclusion against benefit for evidence and a decision rule that support it.

Check Your Understanding

Answer the questions using the treatment-benefit-positive score convention defined above.

Under the delayed-effect model, what is the expected score drift before onset when treatment and control hazards are equal?
With $Y_T=30$, $Y_C=70$, $\theta=0.50$, $w=1$, and $\lambda_C(t)\,dt=0.02$ after onset, calculate the approximate expected score increment.
If an early look has $Z=0.40$ and does not cross its efficacy boundary, what can and cannot be concluded?
At a later look, suppose $U=2.40$, $V=1.00$, and the prespecified upper boundary is 2.50. Does the efficacy stopping rule trigger?
Why is a negative $Z$-statistic not, by itself, proof of treatment harm?

Solutions. 1. The expected increment is zero because $\lambda_T=\lambda_C$, so the hazard difference in the score-drift formula is zero. 2. The risk-set factor is $Y_TY_C/Y=(30\cdot70)/100=21$. The expected increment is $21(1-0.50)(0.02)=0.210$. 3. The efficacy boundary was not crossed; the result does not establish benefit, but it also does not rule out later benefit, especially if the look was before onset. 4. $Z=2.40/\sqrt{1.00}=2.40$, and $2.40<2.50$, so the efficacy stopping rule does not trigger at this look. 5. A negative statistic points in the direction of more weighted treatment events, but a formal harm conclusion requires an appropriate prespecified lower-tail test or decision rule; non-crossing of an upper efficacy boundary alone does not establish harm.