Estimation When an Arm Stops Early for Efficacy

Why Efficacy Stopping Changes the Estimate

An arm that stops early for efficacy has, by definition, produced an interim result that crossed a favorable boundary. Even if the estimator would be unbiased before looking at the stopping decision, the observed estimate is now selected for being large. This creates a tendency for the reported treatment effect to overstate the underlying effect. It is the estimation consequence of the same selection event that made stopping possible.

The size and direction of the adjustment depend on when the arm stopped and what it had to do to reach that stopping stage. An arm stopping at the first interim has crossed one boundary. An arm stopping later has typically failed to cross earlier efficacy boundaries and then crossed a later one. Treating those two paths as the same event discards relevant information.

Definition: Let \(\hat\theta_k\) be an estimate of a treatment effect \(\theta\) at stage \(k\), and let \(E_k\) denote the event that the arm stops for efficacy at stage \(k\). The stage-specific selection bias is \(E_\theta(\hat\theta_k\mid E_k)-\theta\). It describes the conditional mean excess of the estimate among trials with that stopping event, at the specified true effect.

For clarity, examples below use a treatment-minus-control effect in outcome units, with a normal sampling model and known standard errors. In a MAMS trial, estimates from cumulative data at successive stages are generally correlated. The event \(E_k\) must therefore represent the complete path: no earlier efficacy stop, followed by crossing the efficacy boundary at stage \(k\). If futility rules or other selection criteria also operate, they belong in the path as well.

One Efficacy Boundary: The Source of the Upward Bias

Suppose an estimate \(X\) has the model \(X\sim N(\theta,\sigma^2)\), where \(\sigma\) is known, and the arm stops when \(X\geq c\). This model assumes a fixed analysis time and a prespecified boundary; it does not account for additional unmodelled choices made after seeing the data. Conditional on stopping, \(X\) follows a lower-truncated normal distribution.

Write \(a=(c-\theta)/\sigma\), and define the upper-tail inverse Mills ratio by \(\lambda(a)=\phi(a)/[1-\Phi(a)]\), where \(\phi\) and \(\Phi\) are the standard normal density and distribution functions. The conditional mean is

E_\theta(X\mid X\geq c) =\theta+\sigma\lambda\!\left(\frac{c-\theta}{\sigma}\right).

The added term is positive for every finite boundary. It is the mean selection bias at the true \(\theta\). The formula also shows that bias is not a constant subtraction: it depends on the treatment effect, standard error and boundary.

Justification: If \(Z=(X-\theta)/\sigma\), then \(Z\sim N(0,1)\) and \(X\geq c\) is equivalent to \(Z\geq a\). The standard normal tail identity \(E[Z\,1\{Z\geq a\}]=\phi(a)\), divided by \(P(Z\geq a)=1-\Phi(a)\), gives \(E(Z\mid Z\geq a)=\lambda(a)\). Substituting \(X=\theta+\sigma Z\) yields the stated result.

Worked Example: Expected Inflation at an Early Efficacy Look

Suppose a hypothetical first-stage treatment-effect estimate has standard error \(0.10\) outcome units, and the arm stops for efficacy if the estimate is at least \(0.20\) units. At a hypothetical true effect of \(0.10\) units, calculate the mean estimate among arms that stop.

Here \(a=(0.20-0.10)/0.10=1\). The inverse Mills ratio is \(\lambda(1)=\phi(1)/[1-\Phi(1)]\approx0.2420/0.1587=1.525\). Therefore the conditional bias is \(0.10(1.525)=0.1525\) units, and

E_{\theta=0.10}(X\mid X\geq0.20) =0.10+0.1525=0.2525\text{ units}.

This is a mean over hypothetical trials that cross this boundary when the true effect is \(0.10\) units. It does not say that every observed estimate will exceed \(0.2525\), nor that the true effect in a particular trial is \(0.2525\).

Condition on the Stage and the Complete Path

Let \(Y\) collect the estimates or independent stage increments observed through stage \(k\), with joint density \(f_\theta(y)\). If \(E_k\) is the event that the specified efficacy stopping path occurred, the conditional likelihood is

L_c(\theta;y) =\frac{f_\theta(y)}{P_\theta(E_k)},\qquad y\in E_k.

The denominator depends on \(\theta\). Consequently, the conditional score is the ordinary score minus the derivative of the log probability of the selection event:

\frac{\partial}{\partial\theta}\log L_c(\theta;y) = \frac{\partial}{\partial\theta}\log f_\theta(y) - \frac{\partial}{\partial\theta}\log P_\theta(E_k).

A conditional maximum likelihood estimate sets this score to zero. The event probability must match the actual stopping rule, including earlier non-crossings. For cumulative normal estimates, its calculation must also respect their correlation, which arises because later analyses reuse earlier participants' data. The earlier material on conditional likelihood and correlation-aware selection adjustment provides the corresponding general framework; here the emphasis is on the special information conveyed by an efficacy stopping stage.

Sometimes it is useful to condition on the observed interim estimate as well as the fact that the arm reached a later efficacy look. Then the later-stage data are truncated at a threshold determined by that interim value. This is a valid conditional analysis, but it deliberately treats the interim estimate as fixed rather than using its likelihood contribution to estimate \(\theta\).

Worked Example: A Later Efficacy Stop Conditional on the First-Stage Estimate

Consider a hypothetical two-stage trial. Let the first-stage estimate be \(X_1=0.10\) units, with stage-1 efficacy boundary \(0.50\) units, so the arm continues. Let an independent second-stage increment estimate \(U_2\) have standard error \(0.20\) units. For this example, the cumulative estimate is the equally weighted average \((X_1+U_2)/2\), and the stage-2 efficacy boundary is \(0.30\) units. The observed increment is \(U_2=0.60\) units.

The observed cumulative estimate is \((0.10+0.60)/2=0.35\) units, so it crosses the stage-2 boundary. Given \(X_1=0.10\), crossing requires

\frac{0.10+U_2}{2}\geq0.30 \quad\Longleftrightarrow\quad U_2\geq0.50.

Assume \(U_2\sim N(\theta,0.20^2)\), independently of \(X_1\). Conditional on \(X_1\) and on stage-2 crossing, \(U_2\) has a normal distribution truncated below at \(L=0.50\). Its conditional likelihood has score

\frac{U_2-\theta}{0.20^2} -\frac{1}{0.20}\lambda\!\left(\frac{0.50-\theta}{0.20}\right)=0.

For the observed \(U_2=0.60\), the conditional maximum likelihood estimate solves \(0.60=\theta+0.20\lambda((0.50-\theta)/0.20)\). Solving numerically gives approximately \(\hat\theta=0.274\) units: at \(\theta=0.274\), the argument is \(1.13\), \(\lambda(1.13)\approx1.630\), and \(0.274+0.20(1.630)\approx0.600\). This estimate uses the second-stage increment conditional on the first-stage estimate. It is not the full-path conditional maximum likelihood estimate, which would instead use the joint data and condition on both the first-stage non-crossing and the stage-2 crossing.

A Practical Plug-In Correction

A simpler approximation starts from the one-boundary conditional bias and substitutes the unadjusted estimate for the unknown \(\theta\). For observed estimate \(x\), standard error \(\sigma\), and efficacy boundary \(c\), this gives

\hat\theta_{\mathrm{plug\text{-}in}} =x-\sigma\lambda\!\left(\frac{c-x}{\sigma}\right).

This is a plug-in bias correction, not generally the conditional maximum likelihood estimate. It estimates the bias at the observed effect and subtracts it. For a later stopping stage, applying this one-boundary formula to the cumulative estimate alone is not generally appropriate: the earlier non-crossings and correlations change the selection event.

Worked Example: Correcting a First-Look Estimate by Plug-In

In a hypothetical trial, an arm has an estimated treatment effect of \(0.300\) outcome units with standard error \(0.125\) units. The standardized efficacy boundary is \(2.0\), and the observed standardized estimate is \(0.300/0.125=2.4\). Thus the effect-scale boundary is \(c=2.0(0.125)=0.250\) units.

The plug-in ratio is \(\lambda((c-x)/\sigma)=\lambda(-0.4)=\phi(0.4)/\Phi(0.4)\approx0.368270/0.655422=0.5619\). Using the unrounded ratio in the arithmetic, the estimated bias is \(0.125(0.561883)\approx0.0702\) units, and the corrected estimate is

0.300-0.125(0.561883) =0.2298\text{ units}.

The adjustment lowers the estimate because crossing a high efficacy boundary selects positive estimation error. It is an approximate bias correction under the one-boundary normal model, not a confidence interval or a guarantee that the corrected estimate equals the true effect.

Diagnostics, Alternatives, and Limitations

Before applying a correction, verify that the model and selection event describe how the arm actually stopped. In particular, check that the analysis uses the correct efficacy boundaries, stage-specific standard errors and data reuse structure. In a multistage design, compare the recorded path with the modeled event: a stage-2 efficacy stop ordinarily requires non-crossing at stage 1 as well as crossing at stage 2. Omitting either part changes the conditioning probability and can misstate the adjustment.

The main alternatives answer somewhat different questions. A plug-in correction is quick and transparent for a single threshold, but ignores uncertainty in the bias estimate and is only an approximation. Conditional maximum likelihood uses the probability of the complete selection event in its likelihood; it can be more faithful to the stopping mechanism, but may require numerical integration and can be unstable when the selected event is rare. Estimation from an independent validation sample avoids conditioning the estimation data on the efficacy decision, but may be unavailable or less precise. None of these methods automatically resolves model misspecification.

Useful warning signs include a correction calculated from the wrong stage boundary, a claimed stage-specific adjustment that uses only the final crossing while ignoring earlier stages, or a numerical selection probability inconsistent with a simple probability bound. For instance, \(P(A\cap B)\geq P(B)-P(A^c)\); if a computed joint stopping probability is below this bound, the integration or event definition is wrong. Also check that reported results retain the treatment-effect units and that the adjustment has the expected direction for an upper efficacy boundary.

Known or adequately modeled variance. The displayed formulas use a known standard error. Estimating variance, changing allocation, or adapting information requires the corresponding joint model.
Correct sampling distribution. Normality may be exact for a Gaussian contrast or an approximation in other settings; severe departures can alter tail selection behavior.
Prespecified stopping rule. Unrecorded discretion or additional analyses create selection not represented by the nominal boundary.
Interpret the target carefully. Conditioning on a stopping path describes estimates among trials with that path. It is not the same as unconditional performance across all possible paths.

Key takeaway: Efficacy stopping selects favorable estimates. A correction must reflect the stage-specific path that produced the stop; the one-boundary inverse-Mills formula is useful when its assumptions and selection event genuinely apply.

Check Your Understanding

Answer each question, then compare with the concise solutions below.

An estimate has standard error \(0.08\) units and stops at an upper boundary \(0.16\) units. If the true effect is \(0.08\) units, what is its mean selection bias?
Why is the event for stopping at stage 2 generally more than just crossing the stage-2 boundary?
For observed estimate \(x=0.20\) units, standard error \(0.10\) units, and boundary \(c=0.20\) units, calculate the one-boundary plug-in correction.
What does conditioning on the observed first-stage estimate do in the later-stage worked example?
Name one reason a one-boundary correction may be unreliable for a MAMS arm that stops at a later stage.

Solutions. (1) Here \(a=(0.16-0.08)/0.08=1\), so the bias is \(0.08\lambda(1)\approx0.08(1.525)=0.1220\) units. (2) The arm must also have avoided an efficacy stop at each earlier look; that history is part of the selection event. (3) The standardized difference is zero, so \(\lambda(0)=\phi(0)/0.5\approx0.7979\). The corrected estimate is \(0.20-0.10(0.7979)=0.1202\) units. (4) It treats the first-stage estimate as fixed and corrects the second-stage increment conditional on crossing its resulting threshold; it does not use the first-stage value as a likelihood contribution for \(\theta\). (5) The formula may omit earlier non-crossings and the correlation induced by reuse of interim data, so it does not represent the complete stopping path.