Deriving Stagewise-Ordering Confidence Limits

Why the Ordering Matters

After an interim analysis, the final data do not all arise in the same way: some trial paths stop early, while others continue and produce a second-stage result. A confidence limit obtained by test inversion must reflect that complete sampling procedure, including the stopping rule. A stagewise ordering does this by ranking paths according to the stage at which evidence accumulated and then by the evidence within that stage.

Here the construction is deliberately specific: a two-stage normal trial with a fixed early-efficacy boundary and a prespecified ordering. Its formulas do not automatically apply to other stopping rules or other orderings. The earlier material on test-inversion confidence intervals provides the general idea of retaining parameter values whose tests do not reject; the focus here is deriving the test probability for this stagewise ordering.

Model and design: Let \(X_1\) and \(X_2\) be independent stage-specific estimates of a treatment effect \(\theta\), with \(X_1\sim N(\theta,\sigma^2)\) and \(X_2\sim N(\theta,\sigma^2)\). The known standard deviation \(\sigma\) is the same at both stages. The trial stops for efficacy if the observed first-stage estimate satisfies \(X_1\ge b\), where \(b\) is fixed by the design and does not depend on the candidate value of \(\theta\). Otherwise, it continues and observes \(X_2\). For a continued trial, use the equally weighted combined statistic \(Z=(X_1+X_2-2\theta)/(\sqrt{2}\sigma)\).

The assumptions are that the stage estimates have the stated normal distributions, are independent, and have known variances; the stopping boundary and analysis ordering were specified before observing the data. The example calculations below also assume no additional adaptations, such as changing the dose or selecting among multiple treatments. Those changes would require their own sampling model and ordering.

Define the Stagewise Ordering and Its p-Value

For a one-sided test of \(H_0:\theta=\theta_0\) against \(H_a:\theta>\theta_0\), define the stagewise ordering as follows. Every early-efficacy-stop path ranks as more positive evidence than every continued path. Among early-stop paths, larger \(X_1\) ranks as more positive evidence. Among continued paths, larger combined statistic \(Z\), evaluated at \(\theta_0\), ranks as more positive evidence. Thus the p-value is the probability, under the candidate null value and the fixed design, of observing a path ranked at least as positive as the one observed.

Definition: If the observed trial stops early at \(X_1=x_1\ge b\), its stagewise-ordered positive-tail p-value at \(\theta_0\) is \(p^+(\theta_0)=P_{\theta_0}(X_1\ge x_1)\). If the trial continues, with observations \(x_1<b\) and \(x_2\), its p-value is \(p^+(\theta_0)=P_{\theta_0}(X_1\ge b)+P_{\theta_0}(X_1<b,\ Z\ge z_{\mathrm{obs}}(\theta_0))\), where \(z_{\mathrm{obs}}(\theta_0)=(x_1+x_2-2\theta_0)/(\sqrt{2}\sigma)\).

The first term for a continued trial is essential: under the ordering, all early-stop paths are at least as positive as any continued path. But the event defining an early stop is always \(X_1\ge b\), the same observable-data rule for every \(\theta_0\). Under a candidate value, it can be expressed in standardized form as \(U\ge a(\theta_0)\), where \(U=(X_1-\theta_0)/\sigma\) and \(a(\theta_0)=(b-\theta_0)/\sigma\). This standardization changes with the candidate; the design boundary does not.

Deriving the Continuation Probability

Under \(H_0:\theta=\theta_0\), define \(U=(X_1-\theta_0)/\sigma\) and \(V=(X_2-\theta_0)/\sigma\). Independence and the model imply that \(U\) and \(V\) are independent standard normal random variables. For an observed continued trial, the event \(Z\ge z_{\mathrm{obs}}(\theta_0)\) is equivalent to \(U+V\ge q(\theta_0)\), where \(q(\theta_0)=(x_1+x_2-2\theta_0)/\sigma\). Conditional on \(U=u\), this event requires \(V\ge q(\theta_0)-u\). Consequently:

p^+(\theta_0) = 1-\Phi\!\left(\frac{b-\theta_0}{\sigma}\right) + \int_{-\infty}^{(b-\theta_0)/\sigma} \phi(u)\left[1-\Phi\!\left(\frac{x_1+x_2-2\theta_0}{\sigma}-u\right)\right]\,du .

Here \(\phi\) and \(\Phi\) are the standard normal density and cumulative distribution function. The upper integration limit is the continuation condition \(X_1<b\), written under the candidate null. The argument of the inner tail probability comes from the sum \(U+V\), not from the standardized combined statistic alone: multiplying \(Z\ge z_{\mathrm{obs}}\) by \(\sqrt{2}\) gives \(U+V\ge\sqrt{2}z_{\mathrm{obs}}=q(\theta_0)\).

Why the Inversion Gives a Confidence Limit

For a fixed \(\theta_0\), the p-value is the upper-tail probability of the observed path’s rank in the specified ordering. Under the model and design, the rank has a continuous distribution: early-stop paths occupy the most-positive ranks, and continued paths occupy the rest, ordered by their combined statistic. In the early-stop branch, the tail probability ranges from zero up to the probability of stopping; in the continuation branch, it ranges from that stopping probability up to one. Thus the test has size \(\alpha\) under the candidate null, subject to the stated model and ordering.

Inverting the positive-tail test retains candidate values for which \(p^+(\theta_0)>\alpha\). The lower confidence limit is the boundary at which the p-value reaches \(\alpha\), when the retained set has a single boundary. If the p-value is nonmonotone, inversion means finding the entire retained set rather than assuming a unique root.

Record the observed branch.
Determine whether the fixed rule \(x_1\ge b\) stopped the trial or \(x_1<b\) led to continuation.

Write the p-value for that branch.
For continuation, include the early-stop probability and the restricted combined-statistic tail.

Invert in the candidate effect.
Solve \(p^+(\theta_0)=\alpha\) numerically for a boundary, checking the retained set and not changing the observed-data stopping rule.

Worked Example: Evaluating a Continued-Trial p-Value

Suppose, hypothetically, that \(\sigma=1\), \(b=1.96\), and the observed results are \(x_1=1.50\) and \(x_2=2.50\). Since \(1.50<1.96\), the trial continued. Evaluate the positive-tail p-value at \(\theta_0=0\).

The early-stop probability is \(P_0(X_1\ge1.96)=1-\Phi(1.96)\approx0.0250\). For the continuation event, \(a(0)=1.96\) and \(q(0)=1.50+2.50=4\). Therefore the required restricted probability is:

\int_{-\infty}^{1.96}\phi(u)[1-\Phi(4-u)]\,du .

Numerical integration gives approximately \(0.000862\). Hence \(p^+(0)\approx0.0250+0.000862=0.0259\). This is a one-sided p-value for the stated stagewise ordering at the candidate effect of zero. The integral is restricted to the continuation branch; it is not the unrestricted probability of a large combined statistic.

Worked Example: Inverting to Obtain a Lower Limit

Use the same hypothetical design and observations, and find the approximate 95% one-sided lower confidence limit. The endpoint solves \(p^+(\theta_0)=0.05\).

For each candidate \(\theta_0\), keep \(b=1.96\) fixed and calculate \(a(\theta_0)=1.96-\theta_0\) and \(q(\theta_0)=4-2\theta_0\). For example, at \(\theta_0=0.288\), \(a=1.672\) and \(q=3.424\). The early-stop probability is \(1-\Phi(1.672)\), approximately \(0.0473\). Numerical evaluation of the restricted integral with upper limit \(1.672\) and inner tail argument \(3.424-u\) gives approximately \(0.0027\). Thus \(p^+(0.288)\approx0.0500\).

Numerically solving the same equation gives a lower limit of approximately \(0.288\) treatment-effect units. In this example, the p-value is below \(0.05\) at smaller candidate values and rises through \(0.05\) near this root; the inverted one-sided confidence set retains values above the boundary. A 95% one-sided confidence procedure means that, under repeated trials using this fixed design and ordering, its lower limit is at or below the true effect in 95% of trials. It does not mean that the true effect has a 95% probability of lying above this observed limit.

Worked Example: A Trial That Stops Early

In a separate hypothetical trial, let \(\sigma=1\), \(b=1.96\), and suppose the observed first-stage estimate is \(x_1=2.40\). The trial stops because \(2.40\ge1.96\). Find the 95% one-sided lower confidence limit under the early-stop branch.

For a candidate \(\theta_0\), the p-value is \(1-\Phi((2.40-\theta_0)/1)\). Setting it equal to \(0.05\) gives \((2.40-\theta_0)=\Phi^{-1}(0.95)\approx1.645\), so \(\theta_0\approx2.40-1.645=0.755\) treatment-effect units. The observed early-stop branch uses only \(x_1\); there is no \(x_2\) to include. The estimate lies above the design threshold, while the confidence limit reflects the uncertainty in the first-stage estimate rather than treating the threshold crossing as certainty about the effect.

Diagnostics, Alternatives, and Limitations

A reliable calculation starts with the path and the scale. For a continued trial, verify \(x_1<b\), use the same fixed \(b\) at every candidate value, and check that the combined statistic uses the stated standardization. The integral should be no larger than the unrestricted probability of its combined-statistic event, because it additionally imposes \(X_1<b\). These checks can expose a missing scale factor or a mistakenly unrestricted tail.

For example, with the data in the first worked example and \(\theta_0=0\), the combined-statistic event is \((U+V)/\sqrt{2}\ge 4/\sqrt{2}\), equivalently \(U+V\ge4\). Its unrestricted probability is \(1-\Phi(4/\sqrt{2})\approx0.00234\), so the restricted continuation probability must be no greater than \(0.00234\). Using \(4/\sqrt{2}-u\) as the threshold for \(V\) would be an error: the condition on \(V\) is \(V\ge4-u\).

The main alternatives answer different inferential questions or use different orderings. A naive Wald interval applies a normal approximation to an estimate and its standard error but does not, by itself, account for the early-stop selection. A likelihood-based interval uses a likelihood construction and a chosen calibration rather than this stagewise rank. A conditional interval conditions on an adaptation event, changing the probability calculation and its interpretation. The stagewise procedure here instead uses the unconditional probability of paths under the fixed two-stage design and the ordering defined above.

Its limits are important. The result depends on known, equal stage variances, normality, independence, the fixed boundary, and the declared ordering. With unequal variances the combined statistic and integral must be rederived. With estimated variance, non-normal outcomes, additional adaptations, or treatment selection, these formulas are not automatically valid. Finally, test inversion can produce a confidence set with multiple components if the p-value is nonmonotone; numerical root-finding should therefore inspect the p-value over the candidate range rather than report a root as a limit without checking the retained set.

Common Mistake

Wrong reasoning: “As I vary \(\theta_0\), I will define stopping by whether the candidate-standardized statistic exceeds a fixed critical value.” That makes the stopping rule depend on the value being tested, so different points in the inversion describe different trial procedures.

Correction: State the design using observable data, such as \(X_1\ge b\), once and keep that event fixed throughout. Under each candidate null, express the same event as \((X_1-\theta_0)/\sigma\ge(b-\theta_0)/\sigma\). Also retain the correct \(\sqrt{2}\) conversion between the combined statistic and the sum \(U+V\).

Key takeaway: A stagewise-ordered confidence limit comes from inverting a p-value for one fixed sampling design. For a continued path, that p-value includes both the probability of an earlier efficacy stop and the combined-statistic tail restricted to paths that actually continue.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

Under the stated design, what event defines an early efficacy stop as the candidate \(\theta_0\) changes?
For a continued path with equal stage standard deviations, why does the inner normal-tail argument use \(q-u\), where \(q=(x_1+x_2-2\theta_0)/\sigma\)?
In the first worked example, what upper bound must the restricted continuation probability satisfy at \(\theta_0=0\), and why?
For an observed early stop at \(x_1=2.40\), \(\sigma=1\), what equation gives the 95% one-sided lower limit?
What should be done if numerical evaluation shows that \(p^+(\theta_0)\) is not monotone?

Solutions. (1) The event remains \(X_1\ge b\); only its standardized representation, \((X_1-\theta_0)/\sigma\ge(b-\theta_0)/\sigma\), changes. (2) The combined-statistic event is \(U+V\ge q\); conditional on \(U=u\), it requires \(V\ge q-u\). (3) It must be no greater than \(P(U+V\ge4)=1-\Phi(4/\sqrt{2})\approx0.00234\), because the continuation event adds the restriction \(U<1.96\). (4) Solve \(1-\Phi(2.40-\theta_0)=0.05\), yielding \(\theta_0\approx0.755\). (5) Invert the test by finding the full set of candidate values with \(p^+(\theta_0)>\alpha\); do not assume a unique endpoint from a single root.