Worked Normal-Endpoint Analysis at an Early Stop

Why the Early Stop Changes the Analysis

An estimate observed just after an efficacy boundary has been crossed is not an ordinary random sample from the original sampling distribution: the rule selected it because it was sufficiently large. The usual estimate remains easy to calculate, but an interval or estimate that ignores this selection can give a misleading account of the information in the stopped trial. This tutorial works through the distinction for a normal endpoint, with known variance and a prespecified upper efficacy boundary.

The focus is a trial that stops at its first interim look after crossing the boundary. We calculate the ordinary estimate, a stopping-adjusted estimate defined as conditionally median-unbiased, and an equal-tailed confidence interval with coverage conditional on stopping at that look. This is one well-defined analysis, not a claim that there is a unique adjustment appropriate for every purpose.

Model and assumptions: At the first look, the endpoint estimate \(\hat{\theta}_1\) is normally distributed as \(N(\theta,1/I_1)\), where \(\theta\) is the treatment effect in endpoint units and \(I_1\) is the prespecified information. Equivalently, \(\hat{\theta}_1\sim N(\theta,\sigma^2/n_1)\) when the endpoint observations have known variance \(\sigma^2\) and sample size \(n_1\). The upper efficacy rule is to stop at this look if \(Z_1=\sqrt{I_1}\hat{\theta}_1\ge c_1\), for a fixed boundary \(c_1\). The model, information, and boundary are treated as correctly specified, and the endpoint estimate is available without missing-data or other analysis complications.

Write \(S_1\) for the event that the trial stops at the first look. In this tutorial, \(S_1\) means precisely \(Z_1\ge c_1\). The ordinary maximum likelihood estimate based on the observed endpoint is \(\hat{\theta}_1\). The stopping-adjusted estimate below is instead defined by its median property under the distribution conditional on \(S_1\).

Definition: A conditional median-unbiased estimate \(\tilde{\theta}\), for an observed first-look estimate \(x\) with \(x\ge c_1/\sqrt{I_1}\), solves \(P_{\theta}(\hat{\theta}_1\le x\mid S_1)=0.5\). An equal-tailed conditional \(100(1-\alpha)\%\) confidence interval consists of parameter values whose conditional distribution assigns probability between \(\alpha/2\) and \(1-\alpha/2\) to the observed estimate or a smaller value. It conditions on stopping at the first look.

The Conditional Sampling Distribution

For concise calculations, standardize both the observation and the parameter: \(Y=\sqrt{I_1}\hat{\theta}_1\), \(\delta=\sqrt{I_1}\theta\), and \(c=c_1\). Under the model, \(Y\sim N(\delta,1)\), and the observed early-stop event is \(Y\ge c\). Let \(\Phi\) denote the standard normal cumulative distribution function and let \(Q(u)=1-\Phi(u)\) denote its upper-tail probability. For an observed value \(y\ge c\), the conditional distribution function is

F_{\delta}(y\mid S_1) = P_{\delta}(Y\le y\mid Y\ge c) = \frac{\Phi(y-\delta)-\Phi(c-\delta)} {1-\Phi(c-\delta)} = 1-\frac{Q(y-\delta)}{Q(c-\delta)}.

The denominator is essential: it accounts for the fact that only values at or above the efficacy boundary could have produced this early stop. The conditional median estimate solves \(F_{\delta}(y\mid S_1)=0.5\). For an equal-tailed interval, solve \(F_{\delta}(y\mid S_1)=1-\alpha/2\) for the lower endpoint and \(F_{\delta}(y\mid S_1)=\alpha/2\) for the upper endpoint. In this upper-boundary normal model the conditional distribution shifts upward as \(\delta\) increases, so its distribution function at a fixed \(y\) decreases. The equations can therefore be solved numerically as one-dimensional root-finding problems.

Worked Analysis of a Hypothetical Early Stop

Worked Example: Estimate and Conditional Interval After Efficacy

Consider a hypothetical trial estimating a treatment mean difference \(\theta\), measured in endpoint units. At the first look, \(n_1=25\) observations contribute to the estimate, and the known standard deviation is \(\sigma=10\) endpoint units. The prespecified upper efficacy boundary is \(c_1=2.00\). Suppose the trial stops at this look with \(\hat{\theta}_1=4.80\) endpoint units. Check that the stopping rule was met, then calculate the ordinary estimate, the conditional median-unbiased estimate, and a conditional 95% interval.

Check the model and stopping event. The standard error is \(SE=\sigma/\sqrt{n_1}=10/\sqrt{25}=2.00\) endpoint units. Thus \(Z_1=4.80/2.00=2.40\), which is at least \(2.00\), as required by the stated early-stop rule. The standardization gives \(y=2.40\) and \(\delta=\theta/2.00\); the standardized boundary is \(c=2.00\).

Ordinary estimate. The ordinary estimate is \(\hat{\theta}_1=4.80\) endpoint units. It is the estimate observed at the stop; it does not adjust for the rule that selected this look.

Conditional median-unbiased estimate. Here \(y-c=0.40\), so the median equation is \(Q(c-\delta+0.40)/Q(c-\delta)=0.5\). Numerical solution gives \(c-\delta\) approximately \(1.05\), hence \(\tilde{\delta}\approx2.00-1.05=0.95\). Returning to the endpoint scale gives \(\tilde{\theta}=\tilde{\delta}/\sqrt{I_1}=0.95(2.00)=1.90\) endpoint units. The adjusted estimate is much smaller than \(4.80\) because it is calibrated against the distribution conditional on having crossed the boundary.

Conditional 95% interval. Solve the same conditional distribution equation at probabilities \(0.975\) and \(0.025\). Numerical roots are approximately \(\delta_L=-6.91\) for \(F_\delta(2.40\mid S_1)=0.975\), and \(\delta_U=4.14\) for \(F_\delta(2.40\mid S_1)=0.025\). Since \(\theta=\delta(2.00)\), the interval is approximately \([-13.8,\ 8.28]\) endpoint units. Its breadth is not an arithmetic error: conditioning on a rare selection event can leave substantial uncertainty about the underlying mean.

Why the Conditional Interval Has Its Stated Coverage

The coverage claim follows by inverting the conditional distribution, rather than by treating the stopped observation as if no selection had occurred. Under a fixed true value \(\theta\), and conditional on \(S_1\), the random variable \(F_{\theta}(\hat{\theta}_1\mid S_1)\) is uniform on \([0,1]\), provided the conditional distribution is continuous. This is the probability integral transform applied to the sampling distribution restricted to the stopping event.

For the observed estimate \(y\), define \(\delta_L(y)\) by \(F_{\delta_L}(y\mid S_1)=0.975\) and \(\delta_U(y)\) by \(F_{\delta_U}(y\mid S_1)=0.025\). Monotonicity in \(\delta\) means the true parameter lies between these roots exactly when its conditional distribution value lies between the two probabilities:

\delta_L(y)\le\delta\le\delta_U(y) \quad\Longleftrightarrow\quad 0.025\le F_{\delta}(y\mid S_1)\le0.975.

The probability of the right-hand event, conditional on \(S_1\), is \(0.975-0.025=0.95\). Rescaling from \(\delta\) to \(\theta\) preserves the ordering because \(I_1>0\), so the endpoint-scale interval has 95% coverage conditional on stopping at the first look. This derivation assumes the stated normal model and fixed boundary; it does not establish coverage under a misspecified model or a different stopping rule.

Worked Example: A Stop Just Beyond the Boundary

Worked Example: How Close to the Boundary Affects the Adjustment

In a second hypothetical normal-endpoint analysis, suppose the standardized boundary is again \(c=2.00\), but the observed first-look statistic is only \(y=2.20\). The ordinary standardized estimate is \(2.20\). For the conditional median estimate, \(y-c=0.20\), and the equation becomes \(Q(a+0.20)/Q(a)=0.5\), where \(a=c-\delta\).

Numerical solution gives \(a\approx3.10\), since \(Q(3.30)/Q(3.10)\approx0.000483/0.000968\approx0.499\). Therefore \(\tilde{\delta}=2.00-3.10=-1.10\). If the standard error is \(1.50\) endpoint units, the observed estimate is \(2.20(1.50)=3.30\) endpoint units, while the conditional median-unbiased estimate is approximately \(-1.10(1.50)=-1.65\) endpoint units.

This result is counterintuitive but follows from the stated definition. Given that the statistic had to reach at least 2.00, an observed value of 2.20 is only slightly above the selection threshold. The conditional median estimate asks which parameter would make this observed value the median of the truncated sampling distribution; it need not have the same sign as the observed estimate. It is a property of this estimator and conditioning choice, not evidence that the treatment effect is negative.

Comparing the Main Analyses

A conventional fixed-sample calculation and a stopping-aware calculation answer different questions. For the first example, a conventional 95% normal interval is \(4.80\pm1.96(2.00)=[0.88,8.72]\) endpoint units. The corresponding one-sided normal tail probability under \(\theta=0\) is \(Q(2.40)\approx0.0082\). These calculations describe the usual normal model without conditioning on the event that this particular look crossed its boundary. The stopping-aware interval instead uses the conditional distribution given that event, and it is much wider.

The conditional interval is not the only possible stopping-aware analysis. An interval based on the stopped sampling law across all possible stopping stages uses the full prespecified design and does not condition only on the observed stage. Likelihood-based methods also use the sampling plan, but define uncertainty through a likelihood construction rather than the conditional equal-tailed probabilities used here. Which method is appropriate depends on the inferential target and the planned reporting approach; their coverage properties should not be assumed interchangeable.

Analysis	What it uses	Interpretation or limitation
Ordinary estimate and fixed-sample interval	The observed normal estimate and its standard error, without conditioning on the stop	Simple reference calculation; the interval is not made stopping-aware by its usual fixed-sample formula
Conditional median-unbiased estimate and interval	The normal sampling law conditional on crossing the first-look boundary	The estimate has a conditional median property; the interval has its stated coverage conditional on this early stop
Unconditional stopped-law interval	The sampling distribution across the stopping stages in the prespecified design	Targets coverage across possible stopping outcomes, rather than only among trials stopping at the first look

Diagnostics, Pitfalls, and Limitations

Before calculating any adjusted estimate or interval, verify that the recorded statistic really satisfies the boundary and that the analysis uses the boundary specified for that look. In the first example, \(Z_1=2.40\ge2.00\). If a recorded “early stop” instead had \(Z_1=1.90\) under this rule, it would not be an outcome of the stated design; applying the conditional formula anyway would analyze the wrong event. Also verify the standard error and the direction of the boundary. A lower-boundary rule requires a different conditional distribution.

Common mistake: treating the conventional interval as stopping-aware. A standard-error calculation such as \(\hat{\theta}_1\pm1.96SE\) does not condition on the efficacy event. The correction is to identify the sampling distribution associated with the intended coverage statement and use its corresponding interval construction.
Common mistake: calling the adjusted estimate a corrected mean. The estimate here is conditionally median-unbiased, not necessarily conditionally mean-unbiased and not a universal correction. State the property used to define it.
Common mistake: interpreting conditional coverage as unconditional coverage. A 95% interval here covers with probability 0.95 among trials that stop at the first look, under the model. That statement alone does not give 95% coverage across trials that might stop at later looks.
Diagnostic: the adjusted estimate and interval can look surprising. A point estimate below the observed value, or a wide interval containing zero, is not by itself a numerical failure. Check the root equations, the boundary, and the conditioning event before interpreting the result.

These calculations rely on a known variance and a normal endpoint estimate, plus a fixed, correctly implemented upper efficacy boundary. They do not automatically apply when the variance is estimated, the endpoint is binary, or the stopping rule differs from the one specified. In practice, report the observed estimate and its units, the adjustment and its defining property, the interval method, and whether the coverage statement is conditional or unconditional.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions below.

If the observed first-look estimate does not cross the prespecified boundary, can the early-stop conditional distribution in this tutorial be used for that observation?
For \(Y\sim N(\delta,1)\) with an upper stop at \(Y\ge c\), write the conditional distribution function at an observed \(y\ge c\).
What is the property that defines the conditional median-unbiased estimate?
Does a conditional 95% interval necessarily have 95% coverage across all trials in a multi-look design?
In the first worked example, why can the stopping-aware interval be much wider than the conventional fixed-sample interval?

Solutions. (1) No. The formula conditions on stopping at that look, so the observed value must satisfy the stated stopping event. (2) \(F_{\delta}(y\mid S_1)=[\Phi(y-\delta)-\Phi(c-\delta)]/[1-\Phi(c-\delta)]\). (3) It solves \(P_{\theta}(\hat{\theta}_1\le x\mid S_1)=0.5\). (4) No. The 95% statement established here is conditional on stopping at the first look; design-wide coverage requires an analysis of the full stopped sampling law. (5) The conditional interval accounts for selection through crossing the efficacy boundary, whereas the conventional interval does not.