Conditional Error Across Multiple Looks

Why Error Budgets Depend on Interim History

In a multi-look trial, the probability of eventual rejection under the null is determined by the entire sequence of interim results, not by the last result alone. After observing an interim history, some future paths may be much more likely to cross a rejection boundary than others. The conditional error describes that remaining risk on the particular history observed.

This path-specific quantity is useful because it separates two questions: how much type I error the original design had overall, and how much of that error remains available conditional on the trial’s data so far. The introductory material on conditional error functions defines the general idea; here the emphasis is its recursive calculation across several looks.

Definition: Consider a test with $K$ planned looks and a fixed null hypothesis $H_0$. Let $A$ be the event that the original test rejects $H_0$, and let $\mathcal{F}_k$ denote the information available through look $k$. On a history where the test has not rejected by look $k$, the remaining conditional error is $$ e_k=\;P_{H_0}(A\text{ occurs at a later look}\mid\mathcal{F}_k,\ \text{no rejection through look }k). $$ It is a function of the observed history, not generally a single constant. After the final look, $e_K=0$ on a surviving history.

The probability is evaluated under the null model used to calibrate the original test. The definition presumes that the planned looks, rejection rule, and conditional distributions are specified. For a test of exact size $\alpha$, the unconditional probability $P_{H_0}(A)$ is $\alpha$; for a conservative test it is at most $\alpha$. The conditional error at an interim history can be larger or smaller than $\alpha$, without changing that unconditional calibration.

The Backward Recursion

Let $R_k$ denote rejection at look $k$, and suppose the trial continues only when $R_k$ has not occurred. For a surviving history through look $k-1$, define the pre-look budget $b_{k-1}$ as the conditional probability of rejection at look $k$ or any later look. Partitioning those possibilities according to whether rejection occurs at look $k$ gives the recursion:

b_{k-1} = P_{H_0}(R_k\mid\mathcal{F}_{k-1},\text{survival through }k-1) + E_{H_0}\!\left[ \mathbf{1}_{R_k^c}e_k \mid\mathcal{F}_{k-1},\text{survival through }k-1 \right].

Here $e_k$ is evaluated using the history after look $k$, and $\mathbf{1}_{R_k^c}$ is 1 when look $k$ does not reject and 0 otherwise. The first term is the conditional probability of using error at the next look. The second is the expected remaining error among paths that survive that look. This is a backward calculation: begin with $e_K=0$ after the final decision, then calculate the conditional error at each earlier look using the null distribution of the next result.

Key result: The recursion follows from the conditional law of total probability. Conditional on the history through look $k-1$, eventual rejection either happens at look $k$, or look $k$ does not reject and a later rejection occurs. These events are disjoint and exhaustive for eventual rejection.

Applying the law of total expectation repeatedly also gives an overall calibration check. Starting before the first look, the expected probability of eventual rejection is the sum of the probability of rejection at each look, with each contribution weighted by the probability of reaching that look without earlier rejection. Equivalently, averaging the conditional probabilities over histories must recover the original unconditional rejection probability. This is why an interim conditional error is not itself an additional allowance to spend on top of the original $\alpha$.

Canonical Normal Model for Repeated Looks

A common setting is a one-sided group sequential test with standardized statistics $Z_1,\ldots,Z_K$, information fractions $0<t_1<\cdots<t_K=1$, and prespecified upper boundaries $c_1,\ldots,c_K$. Under the simple null, the canonical model has mean-zero, unit-variance statistics with $ \operatorname{Corr}(Z_i,Z_j)=\sqrt{t_i/t_j} $ for $i<j$. Rejection at look $j$ occurs when $Z_j\ge c_j$, provided no earlier look rejected.

This model assumes the information times and boundaries were specified, the canonical joint normal approximation is adequate under $H_0$, and the stated information fractions correctly describe the covariance of the standardized statistics. Its conditional transition law is:

Z_{k+1}\mid Z_1,\ldots,Z_k \sim N\!\left( \sqrt{\frac{t_k}{t_{k+1}}}\,Z_k,\, 1-\frac{t_k}{t_{k+1}} \right).

The transition depends only on $Z_k$ in this canonical model. That Markov property makes a one-dimensional backward recursion possible. In a model where the next statistic depends on more of the history, the recursion remains valid, but the full relevant history must be retained.

Recursion for canonical boundaries: On a surviving history at look $k$, the remaining conditional error is the probability of crossing at least one future boundary. At the final look it is $e_K=0$ after survival. For $k<K$, it is $$ e_k(z_k) = \int_{-\infty}^{c_{k+1}} e_{k+1}(z_{k+1})\,f(z_{k+1}\mid z_k)\,dz_{k+1} + \int_{c_{k+1}}^{\infty} f(z_{k+1}\mid z_k)\,dz_{k+1}, $$ where $f(\,\cdot\mid z_k)$ is the normal transition density above. The first integral accounts for survival at the next look followed by a later rejection; the second accounts for rejection at the next look itself.

Worked Example: Conditional Error With One Look Remaining

Suppose a hypothetical three-look design has $t_2=0.65$, $t_3=1$, and final boundary $c_3=2.00$. A trial reaches look 2 without rejecting and observes $Z_2=1.20$. Calculate its remaining conditional error.

Only the final look remains, so the conditional error is the conditional probability that $Z_3$ crosses 2.00. Under the canonical model, the conditional mean and standard deviation are $$ \mu_{3\mid 2}=\sqrt{0.65}(1.20)=0.9675,\qquad \sigma_{3\mid 2}=\sqrt{1-0.65}=0.5916. $$

Standardizing the boundary gives $$ \frac{2.00-0.9675}{0.5916}=1.745, \qquad e_2=P(Z_3\ge 2.00\mid Z_2=1.20)=1-\Phi(1.745)\approx 0.0405. $$

Thus, conditional on this surviving history, the original test has about a 4.05% probability of rejecting at the final look under $H_0$. For comparison, if $Z_2=1.80$, the conditional mean is $\sqrt{0.65}(1.80)=1.451$, and the standardized boundary is $(2.00-1.451)/0.5916=0.928$. The remaining conditional error is then about $1-\Phi(0.928)=0.177$. Both histories belong to the same design, but their remaining error probabilities differ substantially.

A Multi-Look Budget Update in a Finite Example

The recursion does not depend on normal statistics. The following hypothetical finite-state null model makes the update exact and visible. At look 1, either history $A$ or $B$ occurs, with probabilities 0.40 and 0.60; neither history rejects. At look 2, conditional on $A$, rejection occurs with probability 0.10, and otherwise the probability of rejection at look 3 is 0.20. Conditional on $B$, the corresponding probabilities are 0.05 and 0.10.

Worked Example: Updating the Budget on Each History

Calculate the remaining error after each look-1 history and verify the total null rejection probability.

After history $A$, rejection can occur at look 2, or the trial can survive look 2 and reject at look 3. Therefore $$ e_1(A)=0.10+(1-0.10)(0.20)=0.10+0.18=0.28. $$ After history $B$, $$ e_1(B)=0.05+(1-0.05)(0.10)=0.05+0.095=0.145. $$

Averaging these conditional errors over the look-1 histories gives the unconditional rejection probability: $$ P_{H_0}(A)=0.40(0.28)+0.60(0.145)=0.112+0.087=0.199. $$

The same total can be checked by separating rejection at look 2 from later rejection. The look-2 contribution is $0.40(0.10)+0.60(0.05)=0.070$. The look-3 contribution is $0.40(0.90)(0.20)+0.60(0.95)(0.10)=0.072+0.057=0.129$. Their sum is $0.070+0.129=0.199$. The example’s overall size is therefore 0.199; it is not being presented as a 0.05-level design.

How to Calculate and Check the Recursion

In a continuous model, the recursion is evaluated by integrating over the next-look distribution. In practice, this may use numerical quadrature, a multivariate normal probability calculation, or simulation under the calibrated null model. The method of calculation does not change what the conditional error means: it remains a conditional probability on the observed, surviving history.

Specify the original test.
Record the null model, planned looks, rejection rule, and any stopping rule that is part of the original design.

Start at the final look.
On a history that has survived the final decision, set the remaining error to zero. A rejection at the final look is accounted for in the preceding recursion step.

Integrate backward.
At each earlier surviving history, add the conditional probability of rejection at the next look to the expected later error over next-look outcomes that survive.

Check unconditional calibration.
Average path-specific probabilities under the null, including the probabilities of reaching each history. The result must match the original test’s size, up to numerical error.

Worked Example: Auditing a Budget Decomposition

In the finite example, the probability of reaching look 2 is 1. At that look the rejection contribution is 0.070, and the probability-weighted remaining contribution is 0.129. The recursion therefore gives $$ b_0=0.070+0.129=0.199. $$ The history-specific form gives the same answer: $$ b_0=0.40e_1(A)+0.60e_1(B) =0.40(0.28)+0.60(0.145)=0.199. $$

This equality is a practical audit. If the two calculations disagree beyond numerical tolerance, likely causes include omitting the probability of surviving a look, using the wrong conditional transition distribution, or averaging conditional errors without weighting by the probabilities of their histories.

Interpretation, Alternatives, and Failure Modes

A conditional error is not an unconditional increment of $\alpha$. An alpha-spending function describes marginal error allocated by a given look across all possible histories. Conditional error instead answers: given this particular history and survival so far, what is the probability of a later rejection under the original null model? The two summaries can be connected through averaging, but they are not interchangeable.

Combination tests provide another way to calibrate evidence across stages: they specify a combined statistic and its rejection rule. Conditional error focuses instead on the remaining rejection probability of an already specified test conditional on interim information. These approaches answer related but distinct questions; the earlier material on combination tests develops the former.

Several checks help identify a faulty calculation:

Check survival conditioning. The remaining budget is defined on histories where no rejection has occurred. Including paths that already rejected changes the quantity being calculated.
Check the transition law. Repeated-look statistics are usually dependent. Treating their p-values or Z-statistics as independent can produce incorrect conditional probabilities.
Check the conditioning information. A recursion using only $Z_k$ is justified in the canonical model by its Markov property. In a different model, discarding relevant earlier history can invalidate the calculation.
Check numerical integration. Conditional errors must lie between 0 and 1, and averaging them over the null distribution of surviving histories, together with rejection contributions, must reproduce the original size.

A common mistake is to say, “The trial has spent 0.03 of its 0.05 level, so 0.02 remains after every interim history.” That treats a marginal allocation as a fixed pathwise budget. The correction is to calculate the conditional probability of future rejection for the observed history. It can be above or below 0.02, while the design’s overall error is still controlled through the probabilities of the histories that lead to those conditional values.

The calculation depends on a correctly specified null distribution and on sufficiently accurate conditional probabilities. A misspecified covariance model, incorrect information times, or poor numerical integration can undermine the claimed error preservation. In a simple-null calculation, the result is tied to that null model; the treatment of composite null hypotheses and nuisance parameters requires additional care.

Key takeaway: Across multiple looks, calculate conditional error backward from the final decision. At each step, combine the chance of rejection at the next look with the expected remaining error on paths that survive. Averaging over histories recovers the original test’s unconditional error rate.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

After the final look, what is the remaining conditional error on a history that did not reject?
A surviving history has a 0.08 conditional probability of rejection at the next look. If it survives that look, its expected later conditional error is 0.12. What is its pre-look error budget?
In the canonical model, what are the conditional mean and variance of $Z_{k+1}$ given $Z_k=z_k$?
Why should conditional errors be averaged with the probabilities of the corresponding histories?
What is wrong with assuming that the remaining conditional error equals $\alpha$ minus the marginal error spent so far?

Solutions. (1) It is 0 because no later look remains. (2) The budget is $0.08+(1-0.08)(0.12)=0.08+0.1104=0.1904$. (3) The conditional mean is $\sqrt{t_k/t_{k+1}}\,z_k$, and the conditional variance is $1-t_k/t_{k+1}$. (4) Histories with different probabilities contribute differently to the unconditional rejection probability; probability weighting is required by the law of total probability. (5) Marginal error spending does not specify the future rejection probability conditional on a particular history, so the remaining conditional error must be calculated from that history and the null transition law.