Reporting Stopped Estimates and Their Uncertainty

Why Reporting After Stopping Needs Extra Care

When a group-sequential trial stops early, the observed treatment-effect estimate and its uncertainty are part of a result selected by a monitoring rule. Readers need to know not only the estimate and interval, but also when and why observation stopped and how the interval accounts for that stopping. A bare statement such as “the estimated benefit was 5 units (95% CI 1 to 9)” can conceal whether the estimate came from an early efficacy stop, a futility stop, or the planned final analysis.

The earlier material on conditional and unconditional confidence intervals, boundary-based limits, and likelihood-based and bootstrap intervals develops ways to quantify uncertainty after stopping. This tutorial concentrates on reporting those results clearly. It does not prescribe one estimator or interval for every trial; the choice depends on the design and analysis, and is a separate decision.

Definition: A stopped estimate is an estimate of the treatment effect calculated from the data available at the trial’s observed stopping point. A stopped-trial confidence interval is an interval whose construction and stated coverage interpretation take account of the design’s monitoring and stopping rule. A report should identify the interval’s calibration, including whether coverage is conditional on the observed stopping stage or evaluated across possible stopping stages.

What a Complete Report Identifies

Let \(J\) denote the observed stopping stage, \(K\) the maximum planned number of stages, and \(I_J\) the information available at the stop. Let \(\hat\theta_J\) be the estimate at that stage, for a treatment effect \(\theta\) defined on a stated scale. These labels are useful for describing a result; the report should also give their concrete meaning in the trial.

A complete account usually identifies the following:

The estimand and scale: what treatment contrast is estimated, in which population, and in what units. State the direction of benefit, so readers can tell what a positive or negative estimate means.
The observed stop: stage \(J\) of \(K\), the information level or sample size at that stage, and the prespecified reason for stopping, such as crossing an efficacy boundary, meeting a futility rule, or reaching the planned maximum.
The estimate: the numerical value of \(\hat\theta_J\), its units, and the estimation method. Say which data were included and identify adjustments relevant to the analysis.
The uncertainty measure: the interval’s level, endpoints, units, and calibration. State whether it is conditional on the observed stopping stage or calibrated across stopping stages, and name the method used to construct it.
Limitations: relevant assumptions, sensitivity to the stopping rule or analysis choices, and any simulation or approximation used in calibration. Avoid presenting the interval as if it came from an ordinary fixed-sample analysis unless that is what was actually done.

The stop description is not background detail. For example, “stopped at stage 2 of 4 after the efficacy boundary was crossed” gives readers information they cannot recover from the estimate alone. Information fraction can help describe how early the stop occurred, but it is not automatically the same as the fraction of participants enrolled. If the design uses an information measure, report that measure and its reference value explicitly.

Why the Interval’s Calibration Must Be Named

In a fixed-sample analysis, an interval procedure is often justified by its repeated-sampling behavior at a prespecified sample size. In a group-sequential design, the data available for estimation depend on the stopping rule. A statistic that crosses a boundary has been selected for doing so; this selection can affect the distribution of the estimate at the observed stop.

A short normal-model argument shows the issue. Suppose, for illustration, a standardized statistic at one monitoring look is \(Z\sim N(\delta,1)\), where \(\delta\) is a standardized effect. If the trial stops for efficacy when \(Z\geq c\), then, given that efficacy stopping occurred, the statistic has a truncated distribution:

P(Z\leq z\mid Z\geq c)=0 \quad \text{for } z<c, \qquad P(Z\leq z\mid Z\geq c)= \frac{\Phi(z-\delta)-\Phi(c-\delta)} {1-\Phi(c-\delta)} \quad \text{for } z\geq c.

Here \(\Phi\) is the standard normal cumulative distribution function. The conditional distribution is not the original \(N(\delta,1)\) distribution: values below the stopping threshold are excluded. Thus a fixed-sample interval’s nominal coverage does not, by itself, establish its coverage after the design has selected a stopping stage. The actual calibration depends on the complete stopping rule and the interval procedure. It is not safe to assume that every unadjusted interval is too narrow by the same amount, or that every adjusted interval must be wider.

A report should name the relevant calibration rather than merely calling an interval “95%.” For an unconditional interval, the intended coverage is assessed over repeated trials that may stop at different stages under the design. For a conditional interval, the coverage statement is tied to the observed stopping stage, under the specified conditional procedure and assumptions. Neither label means there is a 95% probability that the fixed, unknown treatment effect lies in the particular calculated interval.

Worked Reporting Examples

Worked Example: Early Efficacy Stop on a Continuous Outcome

Hypothetical trial. A two-arm trial compares a new treatment with control on change in a symptom score, measured in points; a lower score is better. The prespecified design has four monitoring stages. At stage 2, 160 participants have contributed outcome data, the information fraction is 0.50, and the efficacy boundary is crossed. The estimated mean difference, defined as treatment minus control, is \(\hat\theta_2=-4.8\) points, with \(SE=1.9\) points. A prespecified sequentially calibrated, unconditional 95% interval procedure returns \([-9.2,-0.4]\) points.

The estimated effect is expressed in the declared direction: a negative treatment-minus-control difference favors treatment because lower symptom scores are better. Its standardized statistic is

Z_2=\frac{\hat\theta_2}{SE} =\frac{-4.8\ \text{points}}{1.9\ \text{points}} =-2.526.

The standard error and standardized statistic describe the estimate at the observed stage; the interval endpoints here are the stated output of the prespecified sequential calibration, not endpoints calculated from a fixed-sample critical value. The interval lies below zero, with estimated differences from 0.4 to 9.2 points favoring treatment. A suitable report is: “The trial stopped for efficacy at stage 2 of 4 (information fraction 0.50; 160 participants with outcome data). The estimated treatment-minus-control difference in symptom-score change was \(-4.8\) points (SE 1.9); lower scores favor treatment. The sequentially calibrated unconditional 95% CI was \([-9.2,-0.4]\) points.” The result should also identify the analysis population and any design details needed to reproduce the calibration.

Worked Example: Reporting an Unequal Information Increment

Hypothetical trial. A trial plans three looks at target information levels of 50, 100, and 150 units. It stops at look 2, with 92 units accrued, after crossing its efficacy boundary. The observed information fraction relative to the planned maximum is \(92/150\), not a fraction inferred from the look number:

\frac{I_J}{I_{\max}}=\frac{92}{150}=0.6133.

The treatment-effect estimate is 2.6 units, where positive values favor treatment; its standard error is 1.1 units. The standardized statistic is \(2.6/1.1=2.364\). A prespecified conditional 90% confidence procedure, conditional on stopping at look 2, gives \([0.5,4.9]\) units. The report should say “look 2 of 3, information 92 of 150 units (information fraction 0.613), efficacy stop,” rather than “two-thirds of the information was accrued.” The latter incorrectly treats the looks as equally spaced in information. It should further identify the interval as conditional and give the method and assumptions that produced it. Its endpoints support a positive effect under that procedure, but the conditional calibration should not be described as unconditional coverage over all possible stopping stages.

Worked Example: Futility Stop and a Near-Zero Estimate

Hypothetical trial. A binary endpoint is analyzed as a risk difference, treatment risk minus control risk; negative values favor treatment. The trial stops for futility at stage 3 of 4. In the analysis population, 48 of 200 treatment participants and 52 of 198 control participants have the event. The observed risk difference is

\hat p_T-\hat p_C =\frac{48}{200}-\frac{52}{198} =0.2400-0.2626 =-0.02263.

Thus the estimated difference is \(-0.02263\), or about \(-2.26\) percentage points, favoring treatment. Suppose the trial’s prespecified stopping-rule-aware procedure gives an unconditional 95% interval of \([-0.109,0.063]\), in risk-difference units. Because this interval includes zero and also includes effects in either direction, the estimate alone does not establish benefit. A report might state: “The trial stopped for futility at stage 3 of 4 under the prespecified futility rule. The estimated risk difference was \(-2.26\) percentage points (treatment minus control); the sequentially calibrated unconditional 95% CI was \([-10.9,6.3]\) percentage points.” It should name the analysis population, interval procedure, and relevant assumptions. A futility stop is not proof that the treatment effect is exactly zero; it describes the rule and information available when the trial stopped.

Diagnostics, Comparisons, and Limitations

Before finalizing a report, check that the reported stage, information, reason for stopping, estimate, and interval all correspond to the same analysis and the same version of the design. A useful audit is to compare the observed stopping record with the prespecified boundary or futility rule and to verify the interval using the stated calibration procedure. If calibration relies on simulation, report enough about the simulation and its precision for readers to assess numerical stability. If the design, endpoint definition, analysis population, or stopping rule changed, disclose the change and explain which version was used.

The main reporting alternatives are not interchangeable labels. Conditional and unconditional intervals target different coverage statements. Boundary-based limits may be especially relevant when a boundary is crossed; likelihood-based or bootstrap procedures make different construction choices and rely on their own assumptions. The report should state the actual method and calibration, not blend their interpretations. If a conventional fixed-sample interval is included as a comparison, label it explicitly as such and do not substitute it for a stopping-rule-calibrated interval.

No interval removes all uncertainty about the model, estimand, or design. A confidence interval describes uncertainty under its specified procedure and assumptions; it does not account automatically for every unplanned analysis decision, measurement problem, missing-data mechanism, or change in the target population. Nor does an interval’s width alone communicate the stopping history. Readers need both the numerical uncertainty and the context in which the estimate was selected.

Common pitfall: Reporting “estimate (95% CI)” from a fixed-sample formula and treating the nominal level as sufficient because the formula is familiar. The stopping rule changes which data are observed and can change the estimate’s sampling behavior. Correct the report by identifying the observed stop and using an interval with a stated calibration appropriate to the sequential design; if a fixed-sample interval is also shown, label it clearly and explain its limitation.

A concise report can still be complete: name the treatment contrast and units; give the estimate and uncertainty interval; state the stop stage, information, and reason; and identify the estimation and interval methods with their calibration. This makes it possible to interpret the numerical result without mistaking it for an ordinary fixed-sample analysis.

Check Your Understanding

Answer each question using the reporting principles above. Solutions follow the questions.

A trial stops for efficacy at stage 2 of 5. Name three stopping details that should accompany the estimate.
Why is “95% CI” alone insufficient to describe uncertainty after a group-sequential stop?
A treatment-minus-control estimate is \(-3\) points, and lower outcomes favor treatment. State the direction of the estimated effect in context.
A design plans information levels 40, 80, and 160 units, and stops at 80 units. What is the information fraction relative to the maximum, and why should it not be inferred from the look number?

Solutions. (1) Report the stage and total planned stages, the information or sample size at stopping, and the reason for stopping, such as crossing the efficacy boundary; also identify the rule as prespecified if that is the case. (2) The label does not say whether coverage was calibrated conditionally on the observed stage or unconditionally across possible stopping stages, nor how the stopping rule was incorporated. (3) The estimate favors treatment by 3 points because the treatment group’s outcome is lower on the stated treatment-minus-control scale. (4) The fraction is \(80/160=0.500\). Look number alone is not an information fraction when planned increments are unequal; the stage-2 information is half, not two-thirds, of the planned maximum.