Likelihood-Based Intervals After Early Stopping

What a likelihood interval uses after a stop

An early-stopped trial supplies fewer observations than planned, and the observed estimate may be unusually far from the null because the stopping rule selected the time of analysis. A likelihood-based interval starts with the probability model for the data actually observed. Its construction is direct; its coverage, however, must be assessed under the sequential design rather than assumed from a fixed-sample formula.

We use a canonical Gaussian model with a known variance and deterministic information at each planned look. Write \(I_k\) for the cumulative information at look \(k\), and \(S_k\) for the cumulative score. The independent increments satisfy \(\Delta S_j=S_j-S_{j-1}\sim N(\theta\Delta I_j,\Delta I_j)\), where \(\Delta I_j=I_j-I_{j-1}\), \(I_0=S_0=0\), and \(\theta\) is the treatment effect in the outcome's units. Thus \(S_k\sim N(\theta I_k,I_k)\). The stopping rule is fixed in advance and depends on the observed data, not on unknown \(\theta\).

This likelihood uses all observed increments through the stop. The event that the realized path met a stopping boundary is not an extra parameter-dependent factor in the likelihood of that full path: the rule determines whether that path is observed, but does not alter its sampling density. This statement does not mean that the stopping rule is irrelevant to coverage. It shapes the distribution of the estimate and information at the time of stopping.

Deriving the likelihood-ratio interval

Completing the square in the log likelihood gives a particularly simple result:

For a nominal 95% likelihood-ratio interval, the usual one-degree-of-freedom cutoff is \(c=\chi^2_{1,0.95}\approx3.841\). Since \(\sqrt{3.841}\approx1.96\), solving the likelihood-ratio inequality gives:

This is exactly the familiar fixed-information Gaussian interval evaluated at the observed information. The algebra is exact for the stated Gaussian model. What is not automatic is 95% coverage across repetitions of a sequential trial: \(K\) and the estimate at \(K\) are jointly affected by the stopping rule. In particular, a cutoff justified by a fixed-sample reference distribution is not, by itself, a guarantee of sequential coverage.

Why inversion of stopping-aware tests can differ

An interval obtained by inverting tests is built from a different question: for each candidate value \(\theta_0\), would the prespecified test reject \(H_0:\theta=\theta_0\) given the observed stopped path? A stopping-aware test calibrates its rejection rule to the sequential sampling plan. It accounts for the opportunities to cross boundaries at multiple looks, rather than applying a fixed-sample reference distribution at whichever look happened to stop the trial.

For example, suppose a two-sided test rejects a candidate \(\theta_0\) if a standardized score crosses a specified boundary at any look. At look \(j\), the statistic is \(Z_j(\theta_0)=(S_j-I_j\theta_0)/\sqrt{I_j}\). If the design's early boundary is \(b_1\), then at an early stop the candidate is retained when \(|Z_1(\theta_0)|<b_1\). Inverting this condition produces an interval centered at the observed estimate with half-width \(b_1/\sqrt{I_1}\), provided the test uses symmetric boundaries and no other observed-look condition further restricts the set.

The boundary is a feature of the test design, not a universal likelihood-ratio constant. A sequence of boundaries must be calibrated to the design's desired error rate; the resulting confidence set inherits that test calibration. By contrast, the usual likelihood interval uses the fixed cutoff 3.841. Consequently, likelihood-based and test-inversion intervals can have different widths even when they use the same observed estimate and information.

What stopping does—and does not—change in the likelihood

The likelihood calculation above uses the full observed path. If only a summary is available—for example, the stopping stage and endpoint statistic, but not the earlier increments—the likelihood must reflect the probability law of that recorded summary. Likewise, a likelihood conditional on stopping at look \(k\) is a different object from the likelihood of the full path. In general, conditioning on \(K=k\) introduces a parameter-dependent normalizing probability, because \(P_\theta(K=k)\) can vary with \(\theta\). One should not silently replace the full-path likelihood with a conditional likelihood, or the reverse.

The distinction matters because a likelihood formula and a confidence procedure are not the same thing. The likelihood describes relative support for parameter values under the chosen data model. A confidence interval is a procedure with a coverage property under repeated sampling. A nominal likelihood-ratio cutoff may be convenient and informative, but its sequential coverage must be checked for the stopping rule and the data actually recorded. Inverting a stopping-aware test targets coverage through its test calibration; it may produce less symmetric limits or limits with different widths at different stopping stages.

Practical checks and limitations

Before reporting a likelihood-based interval, verify that the assumptions behind the calculation fit the design: the Gaussian increment model, known variance, deterministic information increments, and correctly recorded path through the stop. If the variance is estimated, the displayed likelihood and cutoff no longer give the stated interval; uncertainty in the variance needs its own treatment. If information increments are random or depend on the observed outcomes, the deterministic-information argument also needs revision.

Useful diagnostics include checking that the reported \(S_K\), \(I_K\), and \(\hat{\theta}=S_K/I_K\) agree; confirming which looks and stopping boundaries were actually used; and comparing the interval's claimed coverage with the sequential design's error calibration. Large differences between the nominal likelihood interval and a stopping-aware interval are not an arithmetic error by themselves. They can signal that the selection induced by stopping is consequential.

The principal alternatives answer related but distinct questions. A standard likelihood-ratio interval summarizes likelihood support with a chosen cutoff. An interval from inverting stopping-aware tests is tied directly to a sequential rejection rule and its calibration. Conditional intervals address inference given a specified stopping stage, while unconditional intervals average coverage across stopping stages; neither is interchangeable with an unadjusted fixed-sample interval. The appropriate choice depends on the inferential target, the recorded data, and the error-control properties required.