Two-Look Bias From Stage Selection

Why Separate the Two Stages?

When a trial stops early for efficacy, its reported estimate comes from the interim analysis; otherwise, the estimate comes from the final analysis. These two estimates are selected by different outcomes of the interim data. The unconditional bias of the reported estimate therefore has two parts: one from estimates reported after crossing the efficacy boundary, and another from final estimates reported after not crossing it.

The first contribution is typically positive for an upper efficacy boundary: estimates must be high enough to trigger stopping. The continuation contribution is typically negative because, among trials that did not cross, the interim estimate tends to be lower than expected under the selection event, and the final estimate is correlated with it. The net bias is the sum of these contributions, not either contribution considered alone.

Two-Look Normal Model

Assume a one-sided efficacy design with two scheduled analyses. Let \(I_1\) and \(I_2\) be the cumulative information at the first and second looks, with \(0<I_1<I_2\). Let \(\hat\theta_1\) and \(\hat\theta_2\) estimate the same effect \(\theta\), and assume the canonical normal model

This model follows from normally distributed score increments with independent increments and known information. In particular, the final score contains the interim score plus an independent increment; the cumulative estimates are therefore correlated, not independent. The design stops for efficacy if \(Z_1\ge c\), where \(c\) is a fixed upper boundary. Write \(A=\{Z_1\ge c\}\) for the early-stop event. Assume no other stopping rule changes which estimate is reported.

Define the interim standardized error \(X=Z_1-\theta\sqrt{I_1}\), so \(X\sim N(0,1)\), and define \(a=c-\theta\sqrt{I_1}\). The early-stop event is equivalently \(X\ge a\). Let \(\phi\) and \(\Phi\) denote the standard normal density and cumulative distribution function, respectively.

Deriving the Two Contributions

The unconditional bias can be split by the event that determines which estimate is reported:

Here \(\mathbf 1_A\) is 1 when \(A\) occurs and 0 otherwise. Each expectation is an unconditional contribution: it includes both the size of the estimation error and the probability of the event on which that error is reported.

For the early-stop term, \(\hat\theta_1-\theta=X/\sqrt{I_1}\). For a standard normal variable, integrating its density gives \(E[X\mathbf 1(X\ge a)]=\phi(a)\). Thus the early-stop contribution is \(\phi(a)/\sqrt{I_1}\).

For the continuation term, the normal model gives

Since \(X\) has variance 1 and the pair is jointly normal, the conditional mean of the final estimation error given \(X\) is its covariance with \(X\), multiplied by \(X\): \(E_\theta(\hat\theta_2-\theta\mid X)=(\sqrt{I_1}/I_2)X\). This is the regression identity for a jointly normal pair. Consequently, the continuation contribution is \((\sqrt{I_1}/I_2)E[X\mathbf 1(X<a)]\). Integrating the standard normal density gives \(E[X\mathbf 1(X<a)]=-\phi(a)\), so this contribution is \(-(\sqrt{I_1}/I_2)\phi(a)\).

In fact, because \(X=Z_1-\theta\sqrt{I_1}\), \(\operatorname{Cov}(\hat\theta_2-\theta,X)=\operatorname{Cov}(Z_2,Z_1)/\sqrt{I_2}\). Since \(\operatorname{Cov}(Z_2,Z_1)=\sqrt{I_1/I_2}\), the covariance is \(\sqrt{I_1}/I_2\). Therefore the conditional mean is \((\sqrt{I_1}/I_2)X\), and the continuation contribution is \(-(\sqrt{I_1}/I_2)\phi(a)\). The decomposition is

The covariance calculation is included explicitly because confusing the covariance of \(Z_1,Z_2\) with that of the effect estimates leads to an incorrect continuation term. The final result also shows why the two parts have opposite signs.

Worked Calculations

Interpretation, Diagnostics, and Alternatives

The decomposition is useful because it shows how selection operates across the full design. At the interim look, crossing the boundary selects positive estimation errors. Among trials that continue, errors below the boundary are overrepresented, and correlation between the interim and final estimates carries some of that selection into the final estimate. The continuation contribution is therefore an offset, not a second positive “selection bonus.”

Several checks help detect errors in a calculation:

• Check the event direction. For an upper boundary, early stopping is \(X\ge a\), whose truncated first moment is \(+\phi(a)\). Continuation is \(X<a\), whose first moment is \(-\phi(a)\).
• Check the information scale. The early coefficient is \(1/\sqrt{I_1}\); the continuation coefficient is \(\sqrt{I_1}/I_2\). They have units of the effect, not units of the test statistic.
• Check the arithmetic independently. Multiply each coefficient by the same positive \(\phi(a)\), retain the negative sign for continuation, and then add the results.
• Check limiting cases. If the final analysis has much more information, the continuation coefficient tends toward zero. If \(I_2\) approaches \(I_1\), the formula's net bias approaches zero, consistent with the two cumulative estimates becoming nearly the same.

A common mistake is to report only the positive early-stop contribution as the design's unconditional bias. That ignores estimates reported after continuation. Another is to treat \(\hat\theta_1\) and \(\hat\theta_2\) as independent; the continuation term depends on their correlation and would then be lost. The correction is to define the reported estimate by its stopping event and calculate both unconditional expectations under the joint normal model.

This calculation differs from simply using a fixed-final-look estimate: that estimate is not observed for trials that stop early. It also differs from describing the mean estimate among early-stopped trials alone, which is conditional on \(A\) and does not include the probability of stopping. Direct evaluation of the two event-weighted expectations is the appropriate approach for the unconditional bias asked for here. Numerical integration or simulation can check the formula, but neither removes the need to specify the stopping rule and joint model correctly.

The result has limits. It assumes a known information schedule, a single upper efficacy boundary, normally distributed cumulative estimates, and a common effect parameter across looks. It does not automatically apply if information is random, the boundary is two-sided, there are futility rules, or the estimator is not canonical normal. In such cases the event decomposition remains valid, but the two contributions must be recalculated for the actual joint distribution and reporting rule.