Confidence Limits at an Efficacy Boundary

Why an Exact Boundary Hit Needs Careful Interpretation

An efficacy boundary is a decision threshold, not a typical value selected independently of the data. When a trial stops because its statistic reaches that threshold, the reported estimate is selected by the stopping rule. A fixed-sample interval can therefore have different coverage after stopping than it would have had at a prespecified sample size.

At an exact boundary, a second issue appears: the observed value is at the edge of the set that triggers stopping. A distribution conditioned on stopping at that look is truncated at the boundary. It is important to use that distribution when describing the selected data, but conditioning on the boundary value itself can discard the information needed to estimate the effect. This tutorial separates those ideas and constructs confidence limits by inverting sequential tests.

Canonical Model and Boundary-Conditioned Distribution

Definition: In the canonical normal model, at look $k$, the standardized statistic $Z_k$ has distribution $N(\theta\sqrt{I_k},1)$, where $\theta$ is the effect parameter and $I_k$ is cumulative information. For $j\leq k$, $$ \operatorname{Cov}(Z_j,Z_k)=\sqrt{\frac{I_j}{I_k}}. $$ The information-scaled cumulative statistics $S_k=\sqrt{I_k}Z_k$ have independent increments: $S_k-S_{k-1}\sim N(\theta(I_k-I_{k-1}),I_k-I_{k-1})$. The standardized statistics $Z_k$ do not generally have independent increments.

Assume information increases across looks, the normal model is correctly specified, and the efficacy boundaries and analysis times are fixed in advance. For the examples below, the standard error is treated as known through the information values. If information or variance is estimated, the stated normal model and exact coverage argument need additional justification.

Consider two looks, with an upper efficacy boundary $c_1$ at the first look and $c_2$ at the second. Stopping at the second look means $Z_1<c_1$ and $Z_2\geq c_2$; for a two-sided design, also require $Z_1>-c_1$. If the observed second-look statistic is exactly $Z_2=z$, the conditional distribution of the first-look statistic follows from the joint normal model:

Z_1\mid Z_2=z\sim N\left(\rho z,1-\rho^2\right), \qquad \rho=\sqrt{\frac{I_1}{I_2}}.

To see why the mean has no $\theta$, the untruncated conditional mean is $$ \theta\sqrt{I_1}+\rho\left(z-\theta\sqrt{I_2}\right)=\rho z, $$ because $\rho\sqrt{I_2}=\sqrt{I_1}$. Conditioning further on not having crossed a first-look boundary truncates this normal distribution to the no-crossing region. This is the boundary-conditioned bridge distribution: it describes the earlier path given the later value and the fact that the trial survived to the later look.

That bridge is useful for diagnosing selection and calculating probabilities conditional on the boundary value. But it contains no $\theta$ once $Z_2=z$ is fixed. Therefore it cannot, by itself, supply a confidence interval for $\theta$. A procedure that uses the full observed score must retain the sampling information in the density of $Z_2$, rather than conditioning it away.

Sequential-Test Inversion at a Boundary

Suppose a sequence of two-sided boundaries $c_j>0$ has been chosen so that, under the canonical model, the probability of crossing any boundary when the tested effect is the true value is at most $\alpha$. For a candidate value $\theta_0$, the centered statistic at look $j$ is $Z_j-\theta_0\sqrt{I_j}$. The sequential test rejects that candidate if, at any observed look, $\left|Z_j-\theta_0\sqrt{I_j}\right|>c_j$.

Definition: The inverted sequential confidence set at the observed stopping look $k$ is the set of candidate effects not rejected at any observed look: $$ C_k= \bigcap_{j=1}^{k} \left[ \frac{Z_j-c_j}{\sqrt{I_j}}, \frac{Z_j+c_j}{\sqrt{I_j}} \right]. $$ If the planned boundaries control the probability of crossing at level $\alpha$, this set has unconditional coverage at least $1-\alpha$. The limits are the largest lower endpoint and the smallest upper endpoint among the observed looks.

At an exact upper-boundary observation $Z_k=c_k$, the constraint from look $k$ alone is $$ 0\leq\theta_0\leq \frac{2c_k}{\sqrt{I_k}}. $$ Earlier observed looks may narrow this range, so the final limits must use the intersection rather than only the boundary look.

The coverage justification is direct. If the true value $\theta$ is excluded from $C_k$, then at some observed look $j$, $\left|Z_j-\theta\sqrt{I_j}\right|\geq c_j$. That event is contained in the event that the true centered process crosses a planned boundary at some look. The probability of the latter event is at most $\alpha$ by the design’s calibration. Thus $P_\theta(\theta\in C_k)\geq1-\alpha$. This is an unconditional statement over repeated trials; it does not assert that coverage conditional on stopping at a particular look is exactly $1-\alpha$.

Worked Example: One-Look Boundary and Exact Limits

In a hypothetical one-look normal design, let $I_1=100$, let the two-sided boundary be $c_1=1.96$, and suppose the observed statistic is exactly $Z_1=1.96$. The standard error of the effect estimate is $1/\sqrt{I_1}=0.1$ effect units. The one-look procedure is the familiar two-sided normal test, with nominal level approximately 0.05.

The inverted interval is $$ \left[ \frac{Z_1-c_1}{\sqrt{I_1}}, \frac{Z_1+c_1}{\sqrt{I_1}} \right] = \left[ \frac{1.96-1.96}{10}, \frac{1.96+1.96}{10} \right] =[0,0.392]. $$

The limits are 0 and 0.392 effect units. The lower limit equals zero because the observed statistic is exactly at the positive efficacy boundary. Under the stated normal model, this interval has approximately 95% coverage over repeated one-look trials; it does not mean there is a 95% probability that this particular interval contains the fixed effect.

Worked Example: Two-Look Limits at the Second-Look Boundary

Consider a hypothetical two-look design with $I_1=50$, $I_2=100$, $c_1=2.50$, and $c_2=2.00$. Assume the boundary sequence has been calibrated for a prespecified overall crossing probability $\alpha$. The observed path is $Z_1=1.20$, followed by $Z_2=2.00$, so the trial stops at the second-look upper boundary. Neither boundary was crossed at the first look.

At look 1, the candidate-effect interval is $$ \left[ \frac{1.20-2.50}{\sqrt{50}}, \frac{1.20+2.50}{\sqrt{50}} \right] =[-0.1838,0.5233]. $$

At look 2, it is $$ \left[ \frac{2.00-2.00}{\sqrt{100}}, \frac{2.00+2.00}{\sqrt{100}} \right] =[0,0.4000]. $$

Intersecting the two constraints gives $C_2=[0,0.4000]$ effect units. The boundary look sets the lower limit at zero, while the upper limit is 0.4000. The calculation uses both observed looks; reporting only the look-2 constraint would happen to give the same result here, but need not do so in another path.

The bridge distribution also quantifies the selection at the boundary. Here $\rho=\sqrt{50/100}=0.7071$, so before imposing survival at look 1, $Z_1\mid Z_2=2.00\sim N(1.4142,0.5000)$. Its standard deviation is $0.7071$. Conditional on the exact second-look value, the probability of having stayed inside the first-look two-sided boundaries is $$ P(-2.50<Z_1<2.50\mid Z_2=2.00) = \Phi\left(\frac{2.50-1.4142}{0.7071}\right) - \Phi\left(\frac{-2.50-1.4142}{0.7071}\right) \approx 0.9377. $$

Thus the earlier statistic is drawn from a truncated bridge distribution among trials that survive to look 2. The conditional probability describes the selected path; the confidence limits above instead retain the score information and are justified by unconditional sequential-test inversion.

Why Conditioning on the Boundary Alone Is Not Enough

If one conditions on the exact value $Z_k=c_k$, the endpoint itself is fixed by that conditioning. In the canonical model, the conditional distribution of earlier standardized statistics given $Z_k=c_k$ is a normal bridge whose mean and covariance do not depend on $\theta$. Additional conditioning on no earlier crossing truncates that bridge but still does not restore dependence on $\theta$. Consequently, inference based only on this fully conditioned distribution cannot distinguish effect values.

A different conditional analysis may condition on the stopping stage rather than on the exact statistic. Such an analysis uses a truncated distribution for the statistic at that stage and can have conditional coverage properties different from unconditional coverage. The introductory material on conditional confidence intervals distinguishes those coverage targets. At an exact efficacy boundary, however, the observed statistic lies at the edge of the truncation region, so tail probabilities and limits can become degenerate or highly sensitive to the chosen conditioning event. State the conditioning event whenever reporting a conditional interval.

Worked Example: Boundary Stop at the First Look

Suppose a hypothetical design has first-look information $I_1=36$, symmetric efficacy boundaries $\pm2.30$, and a later planned look that is not reached if the first boundary is crossed. The observed first-look statistic is $Z_1=2.30$, exactly at the upper boundary. Only the first-look data are available for the interval.

The confidence constraint from that observed look is $$ \left[ \frac{2.30-2.30}{\sqrt{36}}, \frac{2.30+2.30}{\sqrt{36}} \right] = \left[0,\frac{4.60}{6}\right] =[0,0.7667]. $$

The limits are 0 and 0.7667 effect units. The later look cannot be treated as if it had occurred or contribute information to these limits. As with the two-look example, the coverage level is determined by the calibration of the planned sequential test, not by the fact that the observed statistic equals the boundary.

Diagnostics, Alternatives, and Limitations

Before reporting a boundary interval, check that the assumed information values and boundaries match the design, that all observed looks are included, and that the stopping rule was followed. Verify that the standardized statistics have the covariance implied by the canonical model. In particular, do not treat $Z_1,Z_2,\ldots$ as independent: it is the information-scaled cumulative statistics that have independent increments.

Approach	What it conditions on	Main implication at an exact boundary
Sequential-test inversion	Uses all observed looks and the planned crossing rule	Provides limits with the design’s stated unconditional coverage control
Stage-conditional interval	Conditions on the stopping stage	Uses a truncated sampling law; conditional coverage differs from unconditional coverage
Boundary-value-only conditioning	Conditions on the exact terminal statistic and, possibly, prior survival	The bridge distribution loses dependence on the effect parameter and cannot alone give effect limits
Fixed-sample interval	Uses a fixed-look reference distribution, ignoring the stopping rule	May not have the advertised coverage after sequential stopping

The inversion method depends on a correctly specified model and properly calibrated boundaries. It is not automatically exact with an estimated standard error, discrete data, data-dependent information, or an unplanned stopping rule. In those settings, the relevant joint sampling law and error control must be addressed rather than assuming the canonical normal calculation still applies. Also, the confidence set may be conservative, especially when it is formed by intersecting constraints from several looks.

A Practical Calculation Sequence

Record the observed path.
List each observed $Z_j$, its information $I_j$, and the boundary sequence. Confirm which boundary caused stopping.

Check the model and design calibration.
Confirm the canonical normal assumptions and identify the overall crossing probability controlled by the planned boundaries.

Invert at every observed look.
For each $j$, calculate $[(Z_j-c_j)/\sqrt{I_j},(Z_j+c_j)/\sqrt{I_j}]$.

Intersect and interpret.
Take the largest lower endpoint and smallest upper endpoint. Report the effect units and whether the coverage statement is unconditional or conditional.

Key takeaway: An exact efficacy-boundary hit makes the boundary-look lower limit equal to zero under the usual centered sequential-test inversion. The earlier path can further constrain the interval. The boundary-conditioned bridge describes selection among paths, but conditioning on the exact endpoint alone removes the effect information needed to construct confidence limits.

Check Your Understanding

Try each question, then check the brief solutions.

In the canonical model, which statistics have independent increments: the standardized $Z_k$ or the information-scaled $S_k=\sqrt{I_k}Z_k$?
At a second-look upper boundary $Z_2=c_2$, what is the lower endpoint imposed by look 2 when inverting a symmetric sequential test?
For $I_1=25$, $I_2=100$, and $Z_2=1.80$, calculate the mean and variance of $Z_1\mid Z_2=1.80$.
Why can the bridge distribution conditional on the exact terminal statistic not, by itself, estimate $\theta$?
Does an interval with unconditional 95% coverage necessarily have 95% coverage conditional on stopping at the efficacy boundary?

Solutions. (1) The information-scaled cumulative statistics have independent increments; standardized cumulative statistics generally do not. (2) The lower endpoint is $(c_2-c_2)/\sqrt{I_2}=0$. (3) Here $\rho=\sqrt{25/100}=0.5$, so the conditional mean is $\rho Z_2=0.9000$ and the variance is $1-\rho^2=0.7500$. (4) Given the exact terminal statistic, the canonical bridge distribution of earlier statistics is independent of $\theta$; the terminal density must not be conditioned away if it is to inform effect estimation. (5) No. Unconditional coverage averages over stopping stages and need not equal coverage conditional on a particular stage or boundary event.