Asymmetric Intervals Under Skewed Stopping Distributions

Why Stopping Can Make an Interval Asymmetric

A confidence interval centered on a stopped estimate is often presented as that estimate plus or minus a margin of error. That symmetry is justified only when the relevant sampling distribution supports it. In a group-sequential trial, the estimate’s distribution depends on the stopping rule: some paths stop early after an unusually large or small interim result, while other paths continue to later analyses. Combining these paths can yield a skewed distribution, so equal-tailed confidence limits need not be equally far from the observed estimate.

The key is to use the distribution generated by the actual stopping rule. This tutorial considers a scalar treatment effect $\theta$, with larger values representing a larger effect. At analysis $k$, let $S_k$ be the cumulative score and $I_k$ its information. Under the canonical model, the increments of $S_k$ are independent, and $S_k\sim N(\theta I_k,I_k)$. Assume the analysis times, information levels, and efficacy and futility boundaries are specified in advance, and that the stopping stage $K$ is determined by applying those boundaries to the interim data. The stopped estimate is $T=S_K/I_K$.

These assumptions let us calculate the distribution of $T$ under each candidate value of $\theta$, accounting for which paths stop at each analysis. If the variance is estimated or the model departs from the canonical assumptions, the stated normal-increment model may not adequately represent the stopped sampling law; interval calculations must then account for the relevant model uncertainty.

Definition: The stopped sampling distribution is the distribution of $T=S_K/I_K$ when the stopping stage $K$ is selected by the trial’s stopping rule. Its CDF at candidate effect $\theta$ is $F_\theta(t)=P_\theta(T\le t)$. An asymmetric equal-tailed interval is obtained by inverting the lower- and upper-tail probabilities of this distribution; its endpoints are not required to be equidistant from the observed $T$.

Stopping can distort the distribution because $T$ is formed from data that also determine $K$. For example, an unusually high interim score may lead to early efficacy stopping and a comparatively high estimate. Other paths with moderate scores may continue and produce estimates with a different spread. The proportions and shapes of these stage-specific contributions vary with $\theta$ and the boundaries. Thus the stopped law can have a long tail on one side, or different tail behavior at different values of $\theta$.

Inverting the Stopped-Law CDF

For a two-sided confidence level $1-\alpha$, use equal tail probabilities $\alpha/2$. For an observed stopped estimate $t_{\mathrm{obs}}$, retain candidate values $\theta$ whose CDF assigns the observation neither an unusually low nor an unusually high rank:

\mathcal{C}(t_{\mathrm{obs}}) = \left\{\theta: \frac{\alpha}{2} \le F_\theta(t_{\mathrm{obs}}) \le 1-\frac{\alpha}{2} \right\}.

If $F_\theta(t_{\mathrm{obs}})$ decreases continuously and strictly as $\theta$ increases, the interval endpoints solve

F_{\theta_L}(t_{\mathrm{obs}})=1-\frac{\alpha}{2}, \qquad F_{\theta_U}(t_{\mathrm{obs}})=\frac{\alpha}{2}.

The order of these equations matters: a candidate effect below the observation makes the observed estimate unusually high under that candidate, while a candidate effect above it makes the observation unusually low. If monotonicity or continuity fails, define the confidence set by the set expression, and examine whether it is connected rather than assuming that two roots always give one interval.

Justification: Suppose that, at the true $\theta$, the stopped estimate has a continuous distribution. Then the probability integral transform gives $F_\theta(T)\sim\mathrm{Uniform}(0,1)$. Consequently, $$ P_\theta\!\left( \frac{\alpha}{2}\le F_\theta(T)\le1-\frac{\alpha}{2} \right)=1-\alpha. $$ The set of candidate values retained by the inversion therefore has coverage $1-\alpha$ under the specified stopped sampling law. With discontinuities, the probability integral transform need not be uniform, and exact equal-tail coverage may require conservative tail handling.

An equivalent view uses error quantiles. Let $q_\gamma(\theta)$ be the $\gamma$-quantile of $T-\theta$ under the stopping rule when the true effect is $\theta$. The candidate $\theta$ is retained when

q_{\alpha/2}(\theta) \le t_{\mathrm{obs}}-\theta \le q_{1-\alpha/2}(\theta).

For a symmetric, fixed-shape error distribution, these quantiles are opposite in sign and the interval may be symmetric. Under skewness they are not. Moreover, in a sequential design the error distribution can depend on $\theta$, so the quantiles may change as the candidate value changes. In general, the limits require numerical inversion of $F_\theta(t_{\mathrm{obs}})$.

Worked Example: Asymmetric Limits From a Skewed Error Law

Suppose, for illustration, a hypothetical stopped estimator has the specified error law $T-\theta=E$, where $E=X-2$ and $X$ has an exponential distribution with rate $0.5$ per effect unit. This is a stylized model, not a claim that every group-sequential design has this law. It has mean-zero error and a long right tail. An observed estimate is $t_{\mathrm{obs}}=4.0$ effect units; construct a 95% equal-tailed interval under this model.

For $e\ge -2$, the error CDF is $P(E\le e)=1-\exp[-0.5(e+2)]$, and it is zero for $e<-2$. Its $\gamma$-quantile is $q_\gamma=-\ln(1-\gamma)/0.5-2$. At the two 95% tail probabilities,

q_{0.025} = \frac{-\ln(0.975)}{0.5}-2 = -1.949, \qquad q_{0.975} = \frac{-\ln(0.025)}{0.5}-2 = 5.378.

Because $T=\theta+E$, the equal-tailed interval is $[t_{\mathrm{obs}}-q_{0.975},\,t_{\mathrm{obs}}-q_{0.025}]$. Substitution gives $[4.0-5.378,\,4.0-(-1.949)]=[-1.378,\,5.949]$ effect units. The lower limit is 5.378 units below the estimate, while the upper limit is 1.949 units above it. This asymmetry follows from the long right tail of the error law; it is not a rounding artifact.

Computing the Stopped Distribution

For a group-sequential design, $F_\theta(t)$ must reflect the actual rule. One way to express it is as a sum over possible stopping stages:

F_\theta(t) = \sum_{k=1}^{K_{\max}} P_\theta\!\left(K=k,\ \frac{S_k}{I_k}\le t\right).

Each term is a probability over paths that reach stage $k$, satisfy the stopping condition there, and have a stage-$k$ estimate no greater than $t$. It can be evaluated by integrating the joint normal increment distribution over the corresponding stopping region, or by a numerical procedure that faithfully reproduces the design. Using the distribution of an estimate at a fixed sample size, without imposing the stopping rule, computes a different CDF.

Specify the design and model.
Record the information levels, boundaries, parameterization, and assumptions used to calculate the joint interim-data distribution.

Calculate the stopped-law CDF.
For candidate values of $\theta$, combine the probabilities across stopping stages under the actual rule.

Invert the tail equations.
For a continuous monotone CDF, find the candidate values where $F_\theta(t_{\mathrm{obs}})$ equals $1-\alpha/2$ and $\alpha/2$.

Check the confidence set.
Confirm that the endpoints are ordered, the CDF crosses the target levels as expected, and the reported set is not hiding a disconnected component.

Worked Example: Inverting a Two-Look Stopped-Law CDF

Consider a hypothetical two-look design with a specified stopping rule and a continuous stopped estimate measured in effect units. Numerical integration of the design’s joint normal increments gives the following CDF values at the observed estimate $t_{\mathrm{obs}}=1.80$: $F_{0.62}(1.80)=0.9750$ and $F_{2.47}(1.80)=0.0250$. The CDF is decreasing in the candidate effect over the relevant range. Find the 95% equal-tailed interval.

For $\alpha=0.05$, the two target probabilities are $1-\alpha/2=0.975$ and $\alpha/2=0.025$. The lower endpoint is the candidate effect for which the observation has CDF rank 0.975, so $\theta_L=0.62$. The upper endpoint is the candidate effect for which its rank is 0.025, so $\theta_U=2.47$. The interval is therefore $[0.62,2.47]$ effect units.

The distances from the estimate are $1.80-0.62=1.18$ units below and $2.47-1.80=0.67$ units above. The interval is asymmetric because the specified stopped-law CDF gives different candidate effects at the two tail ranks. These numerical CDF evaluations are hypothetical outputs of the stated integration; in an analysis, their accuracy should be checked by numerical tolerances or an independent calculation.

Conditional and Unconditional Intervals

The unconditional CDF sums over all possible stopping stages, as in the displayed sum. It supports coverage across repeated trials under the specified design, allowing the stopping stage to vary. A conditional interval instead uses the distribution given the observed stage, such as $P_\theta(T\le t\mid K=k)$. These are different sampling distributions and answer different coverage questions. An interval calibrated conditional on an early stop is not automatically an unconditional interval, and vice versa.

Worked Example: The Stopping Stage Changes the Limits

Suppose a hypothetical design’s unconditional stopped-law CDF, evaluated at an observed estimate of $0.90$ effect units, crosses 0.975 and 0.025 at candidate effects 0.10 and 1.55. For trials that stop specifically at the first look, suppose the conditional CDF at the same estimate crosses those levels at $-0.15$ and $1.30$. Compare the resulting 95% intervals.

Unconditional inversion gives $[0.10,1.55]$ effect units: its endpoint equations are $F_{0.10}(0.90)=0.975$ and $F_{1.55}(0.90)=0.025$. Conditional inversion for first-look stopping gives $[-0.15,1.30]$ effect units, using the CDF conditional on $K=1$. Their different endpoints are expected because the conditional law excludes paths reaching later looks. Each interval’s coverage claim must be stated for the distribution used to construct it.

Diagnostics, Alternatives, and Limitations

A computed asymmetric interval is only as reliable as its stopped-law calculation. Check that the numerical procedure applies every efficacy and futility boundary at the correct look, uses the correct information levels, and sums over mutually exclusive stopping stages. Evaluate the CDF at the proposed endpoints and verify the target tail probabilities. Also inspect the CDF over a wider parameter range: a nonmonotone CDF can produce multiple retained regions, while inaccurate integration can produce unstable or out-of-order roots.

Do not center a symmetric Wald interval automatically. A fixed-sample standard error and normal critical value ignore the distributional change induced by stopping. Such an interval can have incorrect coverage after stopping.
Do not confuse conditional and unconditional calibration. Conditioning on the observed stage changes the sampling law; use it only when that is the intended coverage statement.
Check model adequacy. The canonical normal-increment model may be unsuitable with important variance estimation, departures from the assumed data model, or a stopping rule not represented in the calculation.
Report disconnected or poorly determined sets honestly. If inversion does not yield one stable connected interval, do not replace the result silently with two convenient roots.

The main alternative approaches answer related but distinct questions. A symmetric Wald interval is simpler, but relies on an adequate approximately symmetric sampling distribution. Equal-tailed inversion of the stopped law directly targets the two tail probabilities under the specified stopping design. A stage-conditional interval instead targets coverage conditional on the observed stopping stage. Likelihood-based intervals use a different construction and require their own calibration; they should not be treated as interchangeable with stopped-law CDF inversion.

Stopped-law intervals can require substantial numerical integration, and their validity depends on a correct model of both the data and the stopping rule. Equal-tailed inversion also does not make every possible notion of interval optimal: it allocates specified error probability to each tail, but another allocation could be chosen for a different inferential goal. The important point is to report the calibration target and the design assumptions alongside the limits.

Key takeaway: When stopping makes the estimator’s sampling distribution skewed, equal-tailed limits are found by inverting the stopped-law CDF. Their unequal distances from the estimate reflect unequal tail quantiles, not a failure of the interval calculation.

Check Your Understanding

Answer each question, then compare with the solutions below.

For a 90% equal-tailed interval, what two CDF probabilities define the endpoint equations?
A stopped estimate has a long right-tailed error distribution. Should its equal-tailed interval necessarily be symmetric around the estimate? Explain.
Why can the unconditional interval differ from an interval conditional on stopping at the first look?
If $F_{\theta}(t_{\mathrm{obs}})$ is not monotone in $\theta$, what should replace the assumption that two endpoint roots define the confidence interval?

Solutions. (1) With $\alpha=0.10$, the lower endpoint solves $F_{\theta_L}(t_{\mathrm{obs}})=0.95$, and the upper endpoint solves $F_{\theta_U}(t_{\mathrm{obs}})=0.05$. (2) No. The tail quantiles need not be equal in magnitude, so inversion can give unequal distances on the two sides. (3) The unconditional law averages over all stopping stages; the conditional law restricts to paths stopping at the first look, changing the distribution being inverted. (4) Use the full set $\{\theta:0.05\le F_\theta(t_{\mathrm{obs}})\le0.95\}$ for the chosen confidence level, and examine whether it has multiple components rather than assuming it is one interval.