Estimating Effects for Arms Dropped at Futility

What Can Be Estimated After Futility Stopping?

When an arm is dropped for futility, its later-stage outcomes are not collected under the usual trial design. The trial nevertheless contains interim information about that arm, and the stopping decision itself tells us that the interim result fell on the futility side of a rule. Those facts can support an estimate of the arm’s treatment effect, but only under a stated model for the interim data and the rule that selected the arm for stopping.

The target here is the arm’s underlying treatment effect, not the unobserved result it would have produced had it continued. For example, if the treatment effect is a mean difference, the parameter is the population mean outcome under the treatment minus the population mean outcome under control. The interim estimate is evidence about that parameter; it is not itself the parameter, and it does not reveal what a later-stage estimate would have been.

Definition: Let $\theta$ be an arm’s treatment effect and let $Y$ be its interim estimate, with known standard error $\sigma$. Assume $Y\sim N(\theta,\sigma^2)$. For a fixed futility boundary $c$, suppose the arm is dropped when $Y<c$. A selection-adjusted conditional-likelihood estimate uses the distribution of $Y$ conditional on the observed drop event, $Y<c$, rather than treating $Y$ as if it had been observed without selection.

This normal model is a working model for the interim contrast. It assumes the estimator is unbiased before selection, its standard error is correctly specified, and its sampling distribution is adequately normal. The boundary must be known for the analysis, and the rule must really correspond to $Y<c$, or to an equivalent fixed threshold after standardization. A boundary determined by other data can be treated as fixed only when that conditioning is appropriate; if the other data or other arms’ statistics are correlated with $Y$, their joint selection mechanism may need to be modeled.

The Conditional Likelihood and Its Correction

Write $\phi$ and $\Phi$ for the standard normal density and cumulative distribution function. Since the arm is dropped only when $Y<c$, the conditional density of its observed interim estimate is

f(y\mid Y<c;\theta)= \frac{\phi((y-\theta)/\sigma)} {\sigma\,\Phi((c-\theta)/\sigma)},\qquad y<c.

The denominator is the probability of being dropped under the assumed effect: $P(Y<c\mid\theta)=\Phi((c-\theta)/\sigma)$. It matters because an observed low interim estimate is not an unselected draw from the normal distribution. Define the inverse Mills ratio $\lambda(a)=\phi(a)/\Phi(a)$. Differentiating the conditional log likelihood with respect to $\theta$ gives the score equation below.

\ell(\theta;y)= -\log(\sigma)-\frac{1}{2}\left(\frac{y-\theta}{\sigma}\right)^2 -\log\Phi\left(\frac{c-\theta}{\sigma}\right)+\text{constant}, \qquad \frac{\partial\ell}{\partial\theta} =\frac{y-\theta}{\sigma^2} +\frac{1}{\sigma}\lambda\left(\frac{c-\theta}{\sigma}\right).

Setting the score to zero yields the conditional maximum-likelihood equation. It moves the estimate upward from the observed interim value: among arms known to have fallen below the boundary, a particular low value is less surprising if the underlying effect is larger than that value.

Result: Under the stated model and rule, the conditional maximum-likelihood estimate $\hat{\theta}_{\mathrm{cond}}$ solves $$ \hat{\theta}_{\mathrm{cond}} =y+\sigma\lambda\left(\frac{c-\hat{\theta}_{\mathrm{cond}}}{\sigma}\right). $$ For a finite observed value $y<c$, the solution is above $y$. It remains an estimate based on interim information and the model; it is not an estimate of an unobserved later-stage outcome.

A short justification also explains why the equation has a well-behaved solution in this simple setting. The conditional mean of $Y$ given $Y<c$ is $m(\theta)=\theta-\sigma\lambda(a)$, where $a=(c-\theta)/\sigma$. Differentiating gives

m'(\theta)=1-\lambda(a)\{a+\lambda(a)\} =\frac{\operatorname{Var}(Y\mid Y<c;\theta)}{\sigma^2}>0.

The conditional mean therefore increases strictly with $\theta$. It ranges from arbitrarily low values toward $c$ as $\theta$ increases, so each observed $y<c$ corresponds to a unique solution of $m(\theta)=y$, equivalently the score equation above. Numerical methods such as Newton’s method or a bracketed root finder can solve it. In practice, evaluating the log of $\Phi(a)$ carefully is important when $a$ is far into a tail.

Worked Example: Interpreting a Drop Event

Suppose a hypothetical arm has interim estimate $y=-0.10$ outcome units, standard error $\sigma=0.50$ outcome units, and is dropped when $Y<0.20$. Under the candidate value $\theta=0$, calculate the probability of dropping and the conditional density at the observed value.

The standardized boundary is $(0.20-0)/0.50=0.40$, so $P(Y<0.20\mid\theta=0)=\Phi(0.40)\approx0.6554$. The observed value is indeed below the boundary, as required by the drop event.

Its ordinary normal density under $\theta=0$ is \[ \frac{1}{0.50}\phi\left(\frac{-0.10-0}{0.50}\right) =\frac{1}{0.50}\phi(-0.20) \approx 2(0.3910)=0.7821 \] per outcome unit. Conditional on being dropped, the density is \[ \frac{0.7821}{0.6554}\approx1.193 \] per outcome unit. This is a likelihood contribution, not a probability at the exact point $y=-0.10$; a density can exceed one. The calculation illustrates the selection adjustment but does not by itself estimate $\theta$.

Solving for the Adjusted Estimate

Worked Example: Conditional Estimate for a Dropped Arm

In a hypothetical trial, an arm’s interim effect estimate is $y=0.80$ treatment-effect units with known standard error $\sigma=1$ unit. The arm is dropped if $Y<1.50$ units. Find the conditional maximum-likelihood estimate.

Substitute the observed estimate, standard error, and cutoff into the score equation: \[ \theta=0.80+\lambda(1.50-\theta). \] This is implicit because the inverse Mills ratio depends on the unknown $\theta$. A numerical root finder gives $\hat{\theta}_{\mathrm{cond}}\approx1.794$ units. At this value, the standardized boundary is $1.50-1.794=-0.294$, and $\lambda(-0.294)\approx0.994$. The right side is therefore approximately $0.80+0.994=1.794$ units, agreeing with the root to the shown precision.

The raw interim estimate is $0.80$ units, while the conditional estimate is about $1.794$ units. This substantial shift reflects the assumed rule and model: conditional on falling below $1.50$, an estimate of $0.80$ is interpreted relative to the selected lower tail. It does not establish that the true effect is positive, nor does it recover the later-stage result that would have been observed had the arm continued.

The correction can be large when the observed value is far below a boundary that itself is high relative to the fitted effect. That is not a reason to accept the adjusted value uncritically. It is a reason to inspect the likelihood, the boundary, the standard error, and the assumptions that made a one-dimensional truncation model reasonable.

Worked Example: A Check Using a Known Conditional Mean

Consider a hypothetical normal model with $\sigma=1$, drop rule $Y<0$, and candidate effect $\theta=1$ unit. The standardized boundary is $(0-1)/1=-1$. Since $\lambda(-1)=\phi(1)/\Phi(-1)\approx0.2420/0.1587=1.525$, the conditional mean among dropped arms is \[ E(Y\mid Y<0;\theta=1)=1-1.525=-0.525 \] units, to three decimal places.

Suppose the observed interim value is $y=-0.525$ units. The conditional score equation is \[ \theta=-0.525+\lambda(-\theta). \] At $\theta=1$, the right side is approximately $-0.525+1.525=1.000$ unit, so the conditional estimate is $1.000$ unit (up to rounding). The example shows how a negative interim estimate can be compatible with a positive underlying effect when conditioning on a particularly restrictive drop event. It also shows why the correction depends strongly on the assumed model and selection rule.

What the Estimate Does Not Recover

Once an arm stops, later outcomes for that arm are generally absent by design. No adjustment to its interim estimate can manufacture those observations. The conditional estimate targets $\theta$ under the interim sampling model; it does not estimate the realized later-stage sample mean, the treatment effect in the patients who would have enrolled later, or an alternative outcome under continued follow-up unless additional assumptions define those targets and link them to observed data.

The simple formula is also not automatically valid for a full MAMS selection path. If the same control group contributes to several interim contrasts, their estimates can be correlated. If stopping depends jointly on several arms, on conditional power, or on other random quantities, then the relevant selection event may not be just $Y<c$. In those settings, conditioning on the complete selection event requires the corresponding joint distribution and its selection probability. Treating a correlated, multiarms decision as an independent one-arm truncation can give a misleading correction.

Diagnostics, Alternatives, and Common Pitfalls

Before reporting a conditional estimate, verify that the analysis matches the actual stopping rule and data. Useful checks include:

Boundary check: confirm the observed estimate is on the drop side and express the rule in the same units and direction as $Y$.
Standard-error check: confirm that $\sigma$ corresponds to the interim estimator and accounts for the trial’s allocation and contrast construction.
Model check: assess whether a normal approximation and the assumed treatment-effect scale are defensible at the interim sample size.
Selection check: identify whether the decision also depends on other arms, shared controls, or random quantities not fixed in the conditional analysis.
Numerical check: evaluate the score at the proposed root and verify it is near zero; compare a root finder’s result with the conditional likelihood over a local grid.

A non-finite or highly unstable numerical estimate, a root that does not satisfy the score equation, or a correction that changes dramatically under small plausible changes in the cutoff or standard error signals that the result needs scrutiny. In particular, the correction is not a substitute for sensitivity analysis when the tail probability of dropping is very small under the fitted effect.

Approach	What it uses	Main caution
Raw interim estimate	The observed $y$, without conditioning on the drop	Does not account for the fact that the arm was selected for stopping
Conditional likelihood	The interim estimate and its distribution conditional on the specified drop event	Depends on a correct model and a correctly represented selection event
Unconditional design likelihood	The sampling model together with the trial’s stopping and observation process	Requires modeling the full design and the data actually observed; it cannot supply unobserved outcomes without assumptions

A common mistake is to say, “The arm was stopped for futility, so its treatment effect must be near zero,” or to treat the raw interim estimate as an unbiased estimate after observing that it triggered stopping. The correction is not to assume the effect is zero. Rather, specify a model and stopping rule, then account for selection in the likelihood. Conversely, a large upward adjustment is not proof that a dropped arm is effective: it is an estimate conditional on assumptions, with uncertainty that must be evaluated using appropriate inferential methods.

Key takeaway: An arm dropped for futility can still have a model-based, selection-adjusted estimate of its underlying effect from interim data. The estimate corrects for conditioning on the drop event; it cannot recover unobserved later outcomes, and it is only as credible as the interim model and selection mechanism used to compute it.

Check Your Understanding

Answer each question, then compare with the solutions below.

Under $Y\sim N(\theta,1)$, an arm is dropped if $Y<0.5$. If $\theta=0$, what is its drop probability?
If a dropped arm has observed $y=-0.2$, cutoff $c=0$, and known $\sigma$, what direction does the conditional-likelihood correction move the estimate, and what equation defines it?
Can a conditional estimate based on interim data recover the later-stage estimate that was never observed? Explain.
For $y=1$, $c=2$, and $\sigma=0.5$, write the conditional score equation for the estimate.
Why might a one-arm truncation correction be inappropriate when several arms share a control group?

Solutions. (1) The drop probability is $\Phi((0.5-0)/1)=\Phi(0.5)\approx0.6915$. (2) It moves upward from $y$; the estimate solves $\theta=-0.2+\sigma\lambda((0-\theta)/\sigma)$. (3) No. It estimates the underlying effect under the interim model and does not reveal an unobserved later-stage result. (4) It solves $\theta=1+0.5\lambda((2-\theta)/0.5)$. (5) Shared-control contrasts can be correlated, and the joint stopping event may not have the one-dimensional conditional probability used in the formula; the joint selection mechanism may need to be modeled.

Approach	What it uses	Main caution
Raw interim estimate	The observed \(y\), without conditioning on the drop	Does not account for the fact that the arm was selected for stopping
Conditional likelihood	The interim estimate and its distribution conditional on the specified drop event	Depends on a correct model and a correctly represented selection event
Unconditional design likelihood	The sampling model together with the trial’s stopping and observation process	Requires modeling the full design and the data actually observed; it cannot supply unobserved outcomes without assumptions