One-Sided And Two-Sided Error Functions

Why Test Direction Matters

A conditional error function describes how much of a test's type I error remains, conditional on the interim information observed so far. Its value depends not only on the interim statistic, but also on the original test's rejection region. An upper-sided test rejects for a sufficiently large statistic; a lower-sided test rejects for a sufficiently small one; a two-sided test rejects in either tail. Consequently, the same interim result can leave very different conditional error for these three tests.

This distinction matters in adaptive trials because the conditional error of the original design is the relevant reference when judging a changed continuation procedure. A statistic that moves away from the original direction of evidence may sharply reduce the chance of an upper-tail rejection while increasing the chance of a lower-tail rejection. For a two-sided test, both chances contribute.

Definition: Let $R$ be the rejection event of a specified test under its null model, and let $Y$ denote the interim information on which adaptation may depend. The conditional error at interim information $Y=y$ is $$ A(y)=P_{H_0}(R\mid Y=y). $$ It is the conditional probability, under the null, that the original test would reject given the interim information. It is not the probability that the null hypothesis is true.

Conditional Distribution Under the Canonical Model

For the calculations below, assume a canonical joint-normal model under the null. Let $Z_I$ be the interim standardized statistic and $Z_T$ the statistic at the planned final analysis. Let $t$, with $0<t<1$, be the fraction of total information available at interim. Assume $Z_I\sim N(0,1)$, $Z_T\sim N(0,1)$, and their correlation is $\sqrt{t}$. Equivalently, under this model the final statistic can be represented as $Z_T=\sqrt{t}Z_I+\sqrt{1-t}Z_F$, where $Z_F\sim N(0,1)$ is independent of $Z_I$.

The representation makes the conditional distribution immediate: after observing $Z_I=z$, only the independent future increment remains random. Thus the conditional mean is $\sqrt{t}z$, and the conditional variance is $1-t$. This is a null-model calculation; if nuisance parameters affect the conditional distribution, the resulting error control must account for them rather than treating this pivotal model as automatic.

Z_T\mid Z_I=z\sim N\!\left(\sqrt{t}\,z,\;1-t\right), \qquad SE_{\mathrm{conditional}}=\sqrt{1-t}.

Here the second argument of $N(\mu,\sigma^2)$ is the variance. Write $\Phi$ for the standard normal cumulative distribution function. To calculate a conditional tail probability, standardize each relevant final boundary using the conditional mean and standard deviation, not the unconditional standard normal distribution.

One-Sided Conditional Error Functions

For a level-$\alpha$ upper-sided test based on $Z_T$, the rejection event is $Z_T\ge c_U$, where $c_U=\Phi^{-1}(1-\alpha)$. Its conditional error function is the probability of crossing that upper boundary given $Z_I=z$. For a lower-sided test, the rejection event is $Z_T\le c_L$, where $c_L=\Phi^{-1}(\alpha)$; its conditional error uses the lower tail instead.

A_U(z)=1-\Phi\!\left( \frac{c_U-\sqrt{t}\,z}{\sqrt{1-t}} \right), \qquad A_L(z)=\Phi\!\left( \frac{c_L-\sqrt{t}\,z}{\sqrt{1-t}} \right).

These formulas show the directional effect directly. Increasing $z$ raises the conditional mean of $Z_T$, which generally raises the chance of crossing an upper boundary and lowers the chance of crossing a lower boundary. A negative interim statistic reverses that pattern. The conditional error is a probability under the specified null model, not a one-sided p-value calculated from the interim statistic.

Worked Example: Opposite Directions at the Same Interim Result

Suppose $t=0.50$ and the interim result is $Z_I=-0.80$. Consider level $0.05$ upper-sided and lower-sided tests. Their critical values are approximately $1.645$ and $-1.645$, respectively. Under the conditional model, the mean is $\sqrt{0.50}(-0.80)=-0.5657$, and the standard deviation is $\sqrt{0.50}=0.7071$.

For the upper-sided test, $$ A_U(-0.80)=1-\Phi\!\left(\frac{1.645-(-0.5657)}{0.7071}\right) =1-\Phi(3.1264)\approx 0.0008849. $$ Thus, given this interim result, the null probability that the final statistic reaches the upper rejection boundary is about $0.0008849$.

For the lower-sided test, $$ A_L(-0.80)=\Phi\!\left(\frac{-1.645-(-0.5657)}{0.7071}\right) =\Phi(-1.5264)\approx 0.0635. $$ The interim result is aligned with the lower-tail direction, so the conditional probability of lower-tail rejection is about $0.0635$, greater than $0.05$. The two values are not contradictory: they refer to different rejection events.

Two-Sided Conditional Error

For a symmetric level-$\alpha$ two-sided test, the rejection event is $|Z_T|\ge c$, where $c=\Phi^{-1}(1-\alpha/2)$. It is the union of a lower-tail and an upper-tail event. Those events are disjoint, so their conditional probabilities add. A two-sided conditional error is therefore not obtained by choosing whichever one-sided tail appears more relevant after seeing the interim statistic.

A_2(z) = \Phi\!\left(\frac{-c-\sqrt{t}\,z}{\sqrt{1-t}}\right) + 1-\Phi\!\left(\frac{c-\sqrt{t}\,z}{\sqrt{1-t}}\right).

A two-sided test need not split its type I error equally between tails in every application. If the prespecified rejection region instead has boundaries $c_L$ and $c_U$, with $c_L<c_U$, its conditional error is the sum of the conditional probabilities below $c_L$ and above $c_U$. The boundaries must be those of the test whose error function is being evaluated.

Worked Example: Adding the Two Tails

Continue with $t=0.50$ and $Z_I=-0.80$, but now consider a symmetric two-sided test at level $0.05$. Its critical value is $c=1.960$. The conditional mean remains $-0.5657$ and the conditional standard deviation remains $0.7071$.

The lower-tail contribution is $$ \Phi\!\left(\frac{-1.960-(-0.5657)}{0.7071}\right) =\Phi(-1.972)\approx 0.0243. $$ The upper-tail contribution is $$ 1-\Phi\!\left(\frac{1.960-(-0.5657)}{0.7071}\right) =1-\Phi(3.572)\approx 0.000177. $$ Adding the disjoint tail probabilities gives $$ A_2(-0.80)\approx 0.0243+0.000177=0.0245. $$ The lower tail dominates because the conditional mean is negative. The upper-tail probability is small but is still part of the two-sided rejection event.

A Short Derivation and the Error-Budget Interpretation

The normal formulas follow from conditioning the independent-increments representation. Given $Z_I=z$, the first term $\sqrt{t}Z_I$ is fixed at $\sqrt{t}z$, while the future increment remains $N(0,1)$ and independent. Scaling that increment by $\sqrt{1-t}$ gives the conditional variance $1-t$, hence the stated normal distribution. Substituting the upper, lower, or two-sided rejection event into its conditional distribution yields the corresponding tail formula.

The conditional error also has an unconditional consistency property. If the original test has null rejection probability $\alpha$, then averaging its conditional rejection probability over the interim distribution recovers that probability:

E_{H_0}\{A(Z_I)\} = E_{H_0}\!\left[P_{H_0}(R\mid Z_I)\right] = P_{H_0}(R) = \alpha.

This identity is the law of total probability. It explains why a conditional error can be below or above $\alpha$ at a particular interim result even though the original test has type I error $\alpha$ overall. In conditional-error methods, a proposed continuation may use the original conditional error as an available null rejection probability, subject to the applicable assumptions and calibration. The conditional error is not a fresh $\alpha$ allocation that can be spent independently at every possible interim value.

Worked Example: Directional Error When the Interim Statistic Is Negative

Suppose $t=0.40$, $Z_I=-0.80$, and the original test is an upper-sided level $0.05$ test, with $c_U=1.645$. The conditional mean and standard deviation are $\sqrt{0.40}(-0.80)=-0.5060$ and $\sqrt{0.60}=0.7746$. Therefore, $$ A_U(-0.80) = 1-\Phi\!\left(\frac{1.645-(-0.5060)}{0.7746}\right) = 1-\Phi(2.777) \approx 0.002744. $$ The small conditional error follows from the direction of the test: the negative interim value shifts the conditional mean away from the upper rejection region. This does not imply that the original test's unconditional type I error is below $0.05$; other interim results have larger conditional errors, and their probabilities are included in the unconditional average.

Assumptions, Diagnostics, and Common Pitfalls

The displayed formulas require the canonical joint-normal null model with correlation $\sqrt{t}$, the stated information fraction, and a final statistic whose distribution and rejection boundary are correctly specified. The interim statistic must be the conditioning information used in the calculation. If the design uses a different joint distribution, dependence structure, or test statistic, the conditional distribution must be derived or computed for that setting; inserting a familiar normal formula without checking its assumptions can misstate the error function. With nuisance parameters, a pointwise conditional error under one null value alone may not ensure control across the composite null.

Useful checks include verifying that the conditional standard deviation is positive for $0<t<1$, that each tail is between zero and one, and that the two-sided probability includes both tails. As $z$ increases, the upper-tail conditional error should not decrease under this model, while the lower-tail conditional error should not increase. At $t$ near one, little future variation remains, so the conditional probability becomes increasingly determined by whether the interim result is already on the relevant side of the final boundary.

Using the wrong tail. A negative interim result does not make a lower-tail probability relevant to an originally upper-sided test. The rejection event defines which tail to calculate.
Dropping a tail from a two-sided test. A two-sided rejection event is the union of both tails; calculate both and add them.
Using the unconditional distribution after conditioning. Standardizing a final boundary with mean zero and standard deviation one ignores the observed interim result. Use conditional mean $\sqrt{t}z$ and standard deviation $\sqrt{1-t}$.
Changing direction after observing data. Selecting whichever direction gives the more favorable result changes the rejection procedure. The conditional error must be defined for the original test's prespecified rejection event, not retrospectively relabelled as one-sided.

Conditional error is related to, but distinct from, conditional power: the former is evaluated under the null and concerns rejection probability under the null, whereas conditional power concerns rejection probability under an alternative. It is also distinct from a p-value, which summarizes evidence under a null distribution rather than providing a conditional error budget for a continuation.

The formulas here are exact for the stated canonical model. In discrete tests, other joint distributions, or designs with nuisance parameters, a conditional error function may require a different calculation and may not vary smoothly with the interim result. The essential discipline is unchanged: specify the null model, the information conditioned upon, and the exact rejection event before calculating.

Check Your Understanding

Try each question, then compare your reasoning with the solutions.

Under the canonical model, what is the conditional distribution of $Z_T$ given $Z_I=z$?
At $t=0.50$ and $Z_I=-0.80$, why can an upper-sided conditional error be smaller than $0.05$ while a lower-sided conditional error is larger than $0.05$?
For a symmetric two-sided test with critical value $c$, state the two conditional tail probabilities that must be added.
Does a conditional error below $\alpha$ at one interim value imply that the original test has unconditional type I error below $\alpha$? Explain.

Solutions. (1) $Z_T\mid Z_I=z\sim N(\sqrt{t}z,1-t)$. (2) With a negative $z$, the conditional mean shifts down: this makes reaching an upper boundary less likely and a lower boundary more likely. For the level-$0.05$ example, the probabilities are approximately $0.0008849$ and $0.0635$, respectively. (3) Add $\Phi((-c-\sqrt{t}z)/\sqrt{1-t})$ and $1-\Phi((c-\sqrt{t}z)/\sqrt{1-t})$, the lower- and upper-tail probabilities. (4) No. The unconditional type I error is the average of the conditional errors over the null distribution of the interim statistic; some interim values can have conditional error below $\alpha$ and others above it.