Estimation After Futility Stopping

Why Estimation After Futility Stopping Is Different

A trial that stops for futility has observed interim evidence below a prespecified threshold. This selection matters: even if the treatment effect is positive, the estimate among trials that stop for futility tends to be lower than the true effect. The observed estimate is not automatically invalid, but its interpretation depends on whether the analysis uses the ordinary likelihood or conditions on the stopping event.

This tutorial focuses on an early futility stop at a single interim look. It uses the canonical normal model, so the distinction between the two likelihoods can be made precisely. The introductory material on bias-adjusted estimation establishes why stopping can affect estimates; here the key question is how that selection enters the likelihood after a futility stop.

The estimate \(\hat{\theta}\) has the same units as \(\theta\), while \(Z\), \(f\), and \(\theta\sqrt{I}\) are dimensionless. The assumptions of known \(I\) and a normal statistic are important: estimating a standard error or introducing nuisance parameters changes the likelihood and is not covered by this calculation.

Ordinary and Conditional Likelihoods

If the observed interim statistic is \(z\), the ordinary likelihood is proportional to \(\phi(z-\theta\sqrt{I})\), where \(\phi\) is the standard normal density. Its maximum occurs at \(\hat{\theta}=z/\sqrt{I}\). The stopping rule does not alter this likelihood when the observed statistic itself is used and the rule is a deterministic function of the data: the indicator that the rule permitted this observation does not depend on \(\theta\).

A different analysis conditions on the fact that the trial stopped for futility. Under the stated rule, that event is \(Z\leq f\), whose probability under parameter \(\theta\) is \(\Phi(f-\theta\sqrt{I})\), with \(\Phi\) the standard normal cumulative distribution function. The conditional density of the observed statistic is therefore the original density divided by this event probability.

Taking logs and differentiating gives the conditional-likelihood score. Writing \(t=\theta\sqrt{I}\) makes the equation easier to solve:

The ratio \(\phi(a)/\Phi(a)\) is the inverse Mills ratio for a lower-truncated normal distribution. When a finite solution exists, it is greater than \(z\), because the ratio is positive. Thus the conditional estimate shifts the observed statistic upward to account for having selected an observation from the lower tail.

Why the Conditional Score Has a Unique Root Below the Boundary

A short justification clarifies both the numerical solution and the special case at the boundary. Define \(a=f-t\) and \(\lambda(a)=\phi(a)/\Phi(a)\). The score is \(S(t)=z-t+\lambda(f-t)\). Its derivative is

The variance of a standard normal variable truncated above at \(a\) is \(1-\lambda(a)\{a+\lambda(a)\}\), which is positive for finite \(a\). Hence \(S'(t)<0\): the score is strictly decreasing. As \(t\) tends to negative infinity, \(S(t)\) tends to positive infinity. As \(t\) tends to positive infinity, the inverse Mills ratio satisfies \(\lambda(f-t)\sim t-f\), so \(S(t)\) tends to \(z-f\). If \(z<f\), this limit is negative. Continuity and strict decrease therefore give exactly one finite root.

At the exact boundary \(z=f\), the limiting score is zero rather than negative. In fact, for every finite \(t\),

So the conditional likelihood continues increasing as \(t\) increases and has no finite maximizer over an unrestricted parameter space. A rounded statistic reported as equal to \(f\) may have an unrounded value below \(f\); the analysis should use the unrounded value where available. If the parameter space is restricted, a maximum may occur at a permitted endpoint, but that is a different, constrained analysis.

What Futility Selection Does to the Estimate

The downward selection effect can also be seen in the conditional mean of the ordinary estimate. Let \(\eta=\theta\sqrt{I}\). For a normal variable \(Z\sim N(\eta,1)\), truncation at \(Z\leq f\) gives

Dividing by \(\sqrt{I}\) shows that the conditional mean of \(\hat{\theta}\) is below \(\theta\) by \(\phi(f-\eta)/\{\sqrt{I}\Phi(f-\eta)\}\). This is a statement about the average ordinary estimate over repeated trials that stop for futility at this look; it is not a claim that any particular observed estimate is below the true effect. Nor does it say that the conditional maximum-likelihood estimate removes all bias.

Contrast With Efficacy Stopping and Completion

Futility and efficacy stopping select opposite tails. In the same one-look normal model, an efficacy stop at an upper boundary \(Z\geq e\) conditions on a different event, with probability \(1-\Phi(e-\theta\sqrt I)\). Its conditional likelihood has that upper-tail probability in the denominator, rather than the futility probability \(\Phi(f-\theta\sqrt I)\). The selection effect is generally upward for the ordinary estimate, so the direction of a conditional adjustment is opposite to the futility case.

A trial that reaches its maximum sample size is a third situation, not another name for either early stop. Its data include the fact that the trial continued past earlier looks, and any stopping-aware likelihood must account for that path. The one-look futility denominator above is not the appropriate likelihood for that outcome. Likewise, one should not infer that completion at the planned maximum removes all selection from earlier monitoring.

Choosing and Checking an Estimation Approach

The ordinary estimate and conditional estimate answer different likelihood questions. The ordinary estimate maximizes the likelihood of the observed statistic under the sampling model. The conditional estimate maximizes the distribution of that statistic given the futility event. Neither is universally preferable without specifying the inferential target. Conditional likelihood directly incorporates the observed selection event; the ordinary likelihood preserves the usual sampling model and may be appropriate for unconditional inference. Bias-corrected estimators based on repeated-sampling bias are another approach, but their definition depends on whether bias is conditional on the stopping outcome or averaged across outcomes.

Useful checks for a futility calculation are straightforward. Confirm that \(z\leq f\), use the lower-tail denominator \(\Phi(f-t)\), keep \(I\) and the effect units consistent, and verify that the computed root satisfies the score equation. For \(z<f\), the root should be finite and exceed \(z\). If a solver returns a value below \(z\), or if the denominator is replaced by an upper-tail probability, the setup or computation is wrong. At \(z=f\), do not report a finite unrestricted conditional maximum-likelihood estimate.

The central limitation is that a one-look truncated-normal calculation is only as credible as its model and conditioning event. Incorrect information, a misspecified distribution, data-dependent boundaries, or unaccounted-for changes to the stopping rule can invalidate the stated likelihood. In a multi-look design, the relevant selection event may involve survival through earlier looks as well as the current futility decision; applying a single-threshold formula without that justification can misstate the adjustment.