Why an Interim Efficacy Stop Changes Inference
In a trial with a binary endpoint, an observed treatment-minus-control difference is easy to calculate. Its interpretation is less straightforward when the trial stopped because that difference crossed an efficacy boundary. The same boundary that makes early stopping possible also selects unusually high interim statistics. Consequently, an estimate and interval calculated as if the look had been fixed in advance can understate the effect of stopping.
This tutorial works through that issue for a two-look design in which the trial crosses its efficacy boundary at the first look. The calculations use a canonical normal approximation for the standardized binary-endpoint statistic. They illustrate the difference between an ordinary estimate and interval, which use the observed statistic without accounting for the stopping event, and stopping-aware alternatives that account for the crossing.
Model and Assumptions for the Worked Analysis
The binary data at a look comprise independent Bernoulli outcomes in the two randomized arms. For \(n_T\) treatment participants and \(n_C\) control participants, the observed risk difference is \(\hat{\delta}=\hat{p}_T-\hat{p}_C\), where each \(\hat{p}\) is the arm's observed event proportion. The example uses a null-based standard error fixed by the design. That choice gives a prespecified information scale and supports the canonical approximation; it is not the same as using a standard error recomputed from the observed event rates.
The approximation assumes independent observations between participants and arms, a prespecified first-look sample size and boundary, and a sufficiently accurate normal approximation for the binary statistic. The variance and information used for standardization are treated as known design quantities. Real analyses may need to account for nuisance-parameter estimation, discreteness, or the exact joint distribution across looks. Results below are therefore calculations under the stated model, not exact finite-sample binomial guarantees.
For clarity, focus on the first look. Because there is no earlier look to have failed, the event that the trial stops at this look for efficacy is simply \(Z_1\ge c_1\). Write \(\mu=\delta\sqrt{I_1}\) and \(Q(x)=1-\Phi(x)\), where \(\Phi\) is the standard normal cumulative distribution function. Under the canonical model, the conditional density of the observed statistic, given efficacy stopping, is
The denominator matters: it is the probability of crossing the boundary under the candidate effect \(\mu\). Ignoring it treats the observed \(z\) as though it came from an unselected normal distribution.
Worked Example: Binary Data Crossing the First-Look Boundary
Worked Example: Binary Data Crossing the First-Look Boundary
Suppose a hypothetical randomized trial plans its first analysis after 100 participants per arm. At that look, 49 of 100 treatment participants and 30 of 100 control participants have the event. The design uses a null event probability of 0.40 in both arms to set the standard error and has first-look efficacy boundary \(c_1=2.00\). Calculate the observed effect and statistic, then compare the ordinary estimate and interval with stopping-aware results under the canonical approximation.
Observed effect and standardization. The observed proportions are \(0.49\) and \(0.30\), so \(\hat{\delta}=0.49-0.30=0.190\), or a 19.0 percentage-point higher event proportion in the treatment arm. The design standard error is
Thus \(I_1=1/\sigma_0^2=208.3\), \(\sqrt{I_1}=14.43\), and \(Z_1=\hat{\delta}/\sigma_0=0.190/0.06928=2.742\). Since \(2.742\ge2.00\), the stipulated design stops for efficacy at this look. The calculations meet the model's design-variance assumption by using the prespecified null-based standard error; their accuracy still depends on the normal approximation for these binary data.
Ordinary fixed-look calculation. Treating the look as fixed gives the estimate \(0.190\). A normal 95% interval using the same design standard error is
Under a fixed-look interpretation, this interval estimates a treatment-minus-control risk difference between about 5.4 and 32.6 percentage points. It does not account for the fact that the statistic was observed after crossing the efficacy boundary.
Conditional median-unbiased estimate. Given a crossing, the conditional probability that \(Z_1\) is at least the observed value is \(Q(z-\mu)/Q(c_1-\mu)\). The conditional median-unbiased estimate sets that probability to one half. Here the equation is
Numerically solving gives \(\mu\approx2.17\), so the estimate on the risk-difference scale is \(\hat{\delta}_{\mathrm{med}}=\mu/\sqrt{I_1}\approx2.17/14.43=0.150\). It is still positive, but smaller than the ordinary estimate of 0.190. This is a median-unbiased estimate under the conditional model; it is not the conditional maximum-likelihood estimate.
Equal-tailed conditional 95% interval. For a candidate \(\mu\), the conditional cumulative distribution function at the observed statistic is
Inverting this distribution gives lower and upper endpoints by solving \(F_\mu(z\mid Z_1\ge c_1)=0.975\) and \(F_\mu(z\mid Z_1\ge c_1)=0.025\), respectively. For \(z=2.742\) and \(c_1=2.00\), numerical solutions are approximately \(\mu=-2.42\) and \(\mu=4.64\). Dividing by \(\sqrt{I_1}=14.43\) gives the conditional interval \((-0.168,\ 0.322)\) on the risk-difference scale. Under the model, it is an equal-tailed 95% interval conditional on stopping at the first look for efficacy. It includes zero, unlike the ordinary interval; the two intervals answer different inferential questions.
Why the Conditional Adjustment Has This Form
The conditional median equation follows directly from the truncated sampling distribution. Conditional on stopping, the chance of seeing a statistic at least as large as \(z\) is the unconditional upper-tail probability at \(z\), divided by the probability of reaching the stopping event at \(c_1\). A median estimate is the parameter value for which the observed statistic is the conditional median:
The division is valid because \(z\ge c_1\) for an observed efficacy stop, so \(\{Z_1\ge z\}\) is contained in \(\{Z_1\ge c_1\}\). This also explains why the conditional estimate depends on how far the statistic overshoots the boundary. There is no universal amount to subtract from every observed effect.
The interval inversion uses the same conditional distribution. At a fixed candidate \(\mu\), \(F_\mu(z\mid Z_1\ge c_1)\) is the conditional probability of observing a value no larger than \(z\). The equal-tailed construction chooses parameter endpoints at which the observed \(z\) lies at the 97.5th or 2.5th conditional percentile. Inverting these two tail tests yields conditional coverage of 95% under the specified model. It does not assert a 95% probability that a fixed parameter lies in the realized interval.
Worked Example: A Larger Overshoot Produces a Different Adjustment
Consider another hypothetical first look with the same 100 participants per arm, the same design standard error \(0.06928\), and the same boundary \(c_1=2.00\). Suppose 60 treatment participants and 35 control participants have the event. Check the crossing and calculate the conditional median-unbiased estimate.
The observed risk difference is \(0.60-0.35=0.250\). Standardizing gives \(Z_1=0.250/0.06928=3.608\), which exceeds 2.00. The observation is farther above the boundary than in the previous example. The conditional median equation is
Numerically, the solution is approximately \(\mu=3.53\). The corresponding risk-difference estimate is \(3.53/14.43=0.245\), compared with the ordinary estimate of \(0.250\). The adjustment is smaller here than in the first example because this statistic is farther above the threshold. The calculation again assumes that the specified design standard error and canonical normal model adequately represent the binary statistic.
How the Alternatives Differ
Stopping-aware methods are not interchangeable. The conditional median-unbiased estimate targets the median of the statistic's distribution among trials that cross at this look. A conditional likelihood instead uses the truncated density, including its normalizing probability \(Q(c_1-\mu)\), and generally produces a different point estimate. A confidence interval obtained by inverting the conditional distribution has a conditional coverage interpretation. An unconditional stopped-law interval instead calibrates coverage over the possible stopping outcomes and stages, rather than conditioning on this particular stage.
| Approach | What it accounts for | Interpretation or caution |
|---|---|---|
| Ordinary fixed-look estimate and interval | Uses the observed statistic and design standard error, but not the crossing event | Can overstate the effect after efficacy stopping; fixed-look coverage need not hold for the stopped analysis |
| Conditional median-unbiased estimate | Conditions on crossing the first-look boundary | Centers the conditional distribution by its median; does not automatically provide an interval |
| Conditional interval | Inverts the conditional distribution given the first-look efficacy stop | Targets conditional coverage under the assumed model |
| Unconditional stopped-law interval | Accounts for the stopping rule across its possible outcomes | Targets coverage averaged over stopping outcomes, not conditional coverage at this stage |
Worked Example: Checking the Null Crossing Probability
Under the canonical model and the same first-look boundary \(c_1=2.00\), calculate the probability of stopping for efficacy if the true risk difference is zero. This checks the null crossing probability, not the probability that the null hypothesis is true.
When \(\delta=0\), \(\mu=\delta\sqrt{I_1}=0\). Therefore
Under this one-look crossing calculation, the probability of an efficacy stop at the first look is about 2.28% when the true risk difference is zero. This is a model-based probability under the null. It does not by itself give the trial's overall Type I error for a two-look design, since that also depends on the final-look rule and the joint behavior across looks.
Diagnostics, Pitfalls, and Limits
Before reporting a stopping-aware calculation, check the recorded event counts, arm sizes, treatment-minus-control direction, prespecified boundary, and standard error used to construct \(Z_1\). Confirm that the observed statistic actually satisfies \(Z_1\ge c_1\). Also check that the interval endpoints come from the intended conditional distribution and are reported on the original risk-difference scale, with units such as percentage points.
- Using an ordinary interval as if stopping did not happen. The calculation \(0.190\pm1.96(0.06928)\) is arithmetically valid as a fixed-look interval, but its usual fixed-look interpretation does not account for the efficacy crossing.
- Confusing conditional and unconditional coverage. Conditioning on a first-look crossing answers a different question from averaging coverage over all possible stopping stages.
- Calling the median estimate a conditional maximum-likelihood estimate. The median estimate solves a conditional tail-probability equation. The conditional likelihood includes a truncation normalizer and has a different estimating equation.
- Ignoring model limitations. The normal approximation can be inaccurate with sparse events or very small arm sizes. Data-dependent variance estimation, a different efficacy statistic, additional stopping rules, or repeated looks require an analysis that reflects those details.
The numerical comparison is useful because it shows how stopping can affect both the point estimate and the interval, not because the conditional method is always the uniquely correct choice. The estimand, design, conditioning target, and model must be specified. For an actual trial, the analysis should match the prespecified monitoring rule and the binary-endpoint method used to generate the interim statistic.
Check Your Understanding
Try each question, then compare your reasoning with the solutions below.
- In the first worked example, what were the treatment-minus-control risk difference and the first-look standardized statistic?
- Under the canonical model, write the probability of crossing the boundary at the first look when the standardized mean is \(\mu\).
- Why does the conditional median equation divide one upper-tail probability by another?
- Does the conditional 95% interval in the first example imply a 95% probability that the fixed risk difference lies inside the realized interval?
- Why is the first-look null crossing probability alone insufficient to give the overall Type I error for a two-look design?
Solutions. (1) The risk difference was \(0.49-0.30=0.190\), and the statistic was \(0.190/0.06928=2.742\). (2) It is \(P_\mu(Z_1\ge c_1)=Q(c_1-\mu)\). (3) Given that the statistic crossed \(c_1\), the probability it also reaches \(z\ge c_1\) is the probability of reaching \(z\) divided by the probability of reaching \(c_1\). (4) No. The interval procedure has 95% conditional coverage under the stated model; it does not assign probability to a fixed parameter after observing the data. (5) Overall Type I error also depends on the final-look decision rule and the joint distribution of the statistics across looks.