Calculating Error For A Two-Stage Normal Test

Why Condition on the First Stage?

In a two-stage clinical trial, the final test combines evidence from both stages. Before stage one, its type I error is an unconditional probability under the null hypothesis. After observing the first-stage statistic, however, the relevant quantity for the planned test is the probability that the still-unobserved second stage will lead to rejection, conditional on the first-stage result. That probability is the conditional error for the fixed test at the observed first-stage value.

This tutorial derives that probability for a one-sided normal test with a fixed final boundary. The calculation is useful because it makes explicit how much rejection probability remains after an interim result. It does not, by itself, establish that a modified design preserves error control; that requires a separate result.

Model and Assumptions

Let $Z_1$ and $Z_2$ be the standardized statistics from stages one and two. Under the simple null hypothesis, assume they are independent standard normal random variables. This is appropriate, for example, when stagewise normal score increments are independent, the stage information is fixed, and the normal model (with known variance, or an exact normal score construction) applies. We use a one-sided upper-tail test: large positive values support the alternative.

Let $t$ be the fraction of total information available at the first stage, with $0<t<1$. With independent increments, the final standardized statistic is the information-weighted combination

Z_{\mathrm{final}}=\sqrt{t}\,Z_1+\sqrt{1-t}\,Z_2.

The squared weights sum to one, so under the null $Z_{\mathrm{final}}\sim N(0,1)$. The planned test rejects when $Z_{\mathrm{final}}\ge c$, where $c$ is a fixed upper critical boundary. For a one-sided nominal level $\alpha$, $c$ is chosen as the standard normal upper quantile satisfying $1-\Phi(c)=\alpha$; $\Phi$ denotes the standard normal cumulative distribution function and $\phi$ its density.

Definition: For an observed first-stage value $Z_1=z$, the conditional error of this fixed test is the null probability of rejection at the end of stage two, conditional on that observed value: $$ \mathrm{CE}(z)=P_0\!\left(Z_{\mathrm{final}}\ge c\mid Z_1=z\right). $$

Here $P_0$ denotes probability under the specified null model. The conditional error is a function of the observed statistic $z$, not a single probability that applies equally at every interim result. Stagewise statistics are dimensionless standardized quantities, so the conditional error is also dimensionless.

Deriving the Conditional Tail

Once $Z_1=z$ is fixed, the first contribution to the final statistic is fixed as well. Independence means that conditioning on $Z_1=z$ does not change the null distribution of $Z_2$: it remains $N(0,1)$. Consequently, the conditional final statistic has mean $\sqrt{t}\,z$ and variance $1-t$:

Z_{\mathrm{final}}\mid Z_1=z =\sqrt{t}\,z+\sqrt{1-t}\,Z_2 \sim N\!\left(\sqrt{t}\,z,\,1-t\right).

The event of rejection can be rearranged to a threshold on $Z_2$. Since $\sqrt{1-t}>0$, the inequality direction is unchanged:

\begin{aligned} Z_{\mathrm{final}}\ge c &\iff \sqrt{t}\,z+\sqrt{1-t}\,Z_2\ge c\\ &\iff Z_2\ge \frac{c-\sqrt{t}\,z}{\sqrt{1-t}}. \end{aligned}

Integrating the conditional density of $Z_2$ above this threshold gives the conditional error:

\begin{aligned} \mathrm{CE}(z) &=\int_{(c-\sqrt{t}\,z)/\sqrt{1-t}}^\infty \phi(u)\,du\\ &=1-\Phi\!\left(\frac{c-\sqrt{t}\,z}{\sqrt{1-t}}\right). \end{aligned}

This expression has a direct interpretation. If $z$ is larger, the first-stage contribution $\sqrt{t}\,z$ is larger, the threshold required of $Z_2$ is lower, and the conditional error increases. If the first-stage value is small or negative, more second-stage evidence is needed to cross the same final boundary, so the conditional error decreases.

Worked Example: A Moderate First-Stage Result

Suppose a hypothetical trial has $t=0.50$, a fixed one-sided final boundary $c=1.96$, and an observed first-stage statistic $z=0.40$. The boundary is approximately the usual upper critical value for a one-sided level of 0.025. The assumed independent normal increments and fixed information fraction meet the model conditions above.

First calculate the standardized threshold that the second-stage statistic must exceed:

\frac{c-\sqrt{t}\,z}{\sqrt{1-t}} = \frac{1.96-\sqrt{0.50}(0.40)}{\sqrt{0.50}} = \frac{1.677157\ldots}{0.707107\ldots} \approx 2.3719.

Therefore,

\mathrm{CE}(0.40)=1-\Phi(2.3719)\approx 0.008846.

Under the simple null model and this observed first-stage result, the probability that the fixed final test rejects is about 0.8846%. This is a conditional probability for the remaining test, not the original unconditional type I error.

Worked Example: A Result Near Zero

Consider a second hypothetical trial with $t=0.25$, $c=1.645$, and observed $z=0.50$. The critical value is rounded and corresponds approximately to a one-sided 0.05 test. With independent standard normal stage statistics and a fixed information fraction, the formula applies.

The first-stage weight is $\sqrt{0.25}=0.5$, so the first-stage contribution is $0.5(0.50)=0.25$. The remaining standard deviation is $\sqrt{0.75}$:

\frac{1.645-\sqrt{0.25}(0.50)}{\sqrt{0.75}} = \frac{1.395}{0.866025\ldots} \approx 1.6108.

Thus,

\mathrm{CE}(0.50)=1-\Phi(1.6108)\approx 0.0536.

The value is slightly above 0.05 even though the observed first-stage statistic is positive. This is not a contradiction: a positive interim result makes final rejection more likely than it would be with no first-stage contribution, and the particular result and rounded boundary determine the conditional probability.

Worked Example: A Strong First-Stage Result

In a third hypothetical trial, let $t=0.25$, $c=1.645$, and suppose the first-stage statistic is $z=3.29$. Assume again independent standard normal stage statistics and the same fixed final test. The first-stage contribution is $\sqrt{0.25}(3.29)=1.645$, exactly the stated boundary.

\frac{1.645-\sqrt{0.25}(3.29)}{\sqrt{0.75}} = \frac{1.645-1.645}{\sqrt{0.75}}=0.

The final test rejects precisely when $Z_2\ge 0$, so under the null

\mathrm{CE}(3.29)=1-\Phi(0)=0.5.

A conditional error of 0.5 does not mean the test has an unconditional type I error of 0.5. It means that, conditional on this unusually large first-stage value, half of the null distribution for the independent second-stage statistic lies on the rejection side of the fixed boundary.

Checks, Diagnostics, and Edge Cases

The formula can be checked against the original unconditional test. Under the null, the weighted sum $\sqrt{t}Z_1+\sqrt{1-t}Z_2$ is standard normal, so its unconditional rejection probability is $1-\Phi(c)$. Equivalently, integrating the conditional rejection probability over the first-stage density must recover that probability:

\int_{-\infty}^{\infty} \left[ 1-\Phi\!\left(\frac{c-\sqrt{t}\,z}{\sqrt{1-t}}\right) \right]\phi(z)\,dz = 1-\Phi(c).

This identity is a useful numerical diagnostic. If numerical integration of the conditional error curve does not reproduce the planned unconditional level, check the weights, the sign of the tail, and whether the model used for the calculation matches the planned test.

Several qualitative checks are also immediate. For $0<t<1$, the conditional error increases with $z$, because

\frac{d}{dz}\mathrm{CE}(z) = \phi\!\left(\frac{c-\sqrt{t}\,z}{\sqrt{1-t}}\right) \frac{\sqrt{t}}{\sqrt{1-t}}>0.

As $z$ tends to negative infinity, the conditional error tends to zero; as $z$ tends to positive infinity, it tends to one. At $t=0$, there is no first-stage information in the final statistic, and the conditional error is the constant $1-\Phi(c)$. At $t=1$, there is no second-stage information and the displayed formula has a zero denominator. In that degenerate case the final statistic is already determined: the conditional rejection probability is 1 if $z\ge c$, and 0 otherwise.

A useful comparison is between the conditional calculation and other ways to assess the planned test:

Approach	Quantity calculated	Strength and limitation
Unconditional normal test	$1-\Phi(c)$	Gives the planned type I error before observing stage one, but does not show how that error probability depends on the interim result.
Conditional normal integration	$\mathrm{CE}(z)$ at the observed $z$	Gives the remaining rejection probability under the stated model; depends on the correct conditional distribution and fixed boundary.
Simulation under the null	An empirical conditional rejection frequency	Can check calculations or handle more complicated distributions, but introduces Monte Carlo error and must condition or bin on the interim statistic appropriately.

Common Mistakes and Limitations

Using the unconditional level as the conditional error. The statement “the test is level 0.05, so the conditional error is 0.05 at every interim result” is wrong. The unconditional level averages over all possible first-stage results; $\mathrm{CE}(z)$ varies with the observed $z$.
Dropping the remaining-stage standard deviation. The threshold is $(c-\sqrt{t}z)/\sqrt{1-t}$, not simply $c-\sqrt{t}z$. The division standardizes the second-stage statistic.
Using the wrong tail or reversing the inequality. For an upper-tail test, rejection requires $Z_2$ to exceed the threshold, giving $1-\Phi(\cdot)$. A lower-tail test requires a corresponding change in direction.
Assuming independence without justification. If stagewise statistics are dependent, or if information and data reuse alter the conditional distribution, the conditional law used here may be wrong. The normal tail formula then does not apply as written.

The calculation is exact under the stated normal independent-increment model, but it is not automatically exact for every clinical endpoint. Estimated variances, small-sample distributions, missing outcomes, nuisance parameters, changing information, and non-normal statistics can require a different conditional distribution or an appropriate approximation. Likewise, this tutorial calculates the conditional error of a fixed boundary. It does not claim that any subsequent adaptation is valid merely because this quantity has been calculated.

Key takeaway: For a one-sided two-stage normal test with independent standard normal increments and information fraction $t$, the fixed-boundary conditional error after observing $Z_1=z$ is $1-\Phi\!\left((c-\sqrt{t}\,z)/\sqrt{1-t}\right)$. The observed first-stage result shifts the conditional mean; the remaining information determines the conditional standard deviation.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

For $t=0.36$, $c=1.645$, and $z=0$, write the second-stage threshold and explain why the conditional error is below the unconditional upper-tail probability at $c$.
For $t=0.25$, $c=1.645$, and $z=3.29$, calculate the conditional error.
Name two assumptions needed for the displayed conditional-error formula.
Why does the formula not apply directly at $t=1$, and what is the conditional rejection probability then?

Solutions. (1) The threshold is $(1.645-\sqrt{0.36}(0))/\sqrt{0.64}=1.645/0.8=2.05625$. Since $2.05625>1.645$, the second-stage statistic must exceed a more stringent standardized value than the original boundary, so $1-\Phi(2.05625)<1-\Phi(1.645)$. (2) The threshold is $(1.645-0.5(3.29))/\sqrt{0.75}=0$, giving $1-\Phi(0)=0.5$. (3) Required assumptions include independent stagewise statistics, each standard normal under the null, a fixed information fraction, and the specified fixed upper rejection boundary. (4) At $t=1$, the denominator $\sqrt{1-t}$ is zero because no second-stage variation remains. The final statistic equals $Z_1$, so conditional rejection probability is 1 when $z\ge c$ and 0 when $z<c$.

Approach	Quantity calculated	Strength and limitation
Unconditional normal test	\(1-\Phi(c)\)	Gives the planned type I error before observing stage one, but does not show how that error probability depends on the interim result.
Conditional normal integration	\(\mathrm{CE}(z)\) at the observed \(z\)	Gives the remaining rejection probability under the stated model; depends on the correct conditional distribution and fixed boundary.
Simulation under the null	An empirical conditional rejection frequency	Can check calculations or handle more complicated distributions, but introduces Monte Carlo error and must condition or bin on the interim statistic appropriately.