Two Ways to Describe Type I Error Over Time
A sequential trial may examine efficacy at several planned analysis times. Under the null hypothesis \(H_0\), its type I error is the probability of rejecting \(H_0\) at any of those times. There are two useful ways to describe how that error is distributed across the trial. An alpha-spending function describes cumulative, unconditional rejection probability by analysis time. A conditional error function describes the probability of future rejection for a particular interim history, conditional on the data already observed.
These descriptions answer different questions. A spending increment is a probability averaged over possible trial histories under \(H_0\). A conditional error is specific to one history. When a design changes after an interim analysis, the history-specific quantity can be used to check whether the proposed future rejection rule spends no more error than the original design allowed at that history.
The phrase “conditional on the interim information” matters. For a statistical model with interim information \(\mathcal{F}_j\), the conditional error is a function of that information, such as \(e_j(\mathcal{F}_j)=P_{H_0}(\text{a future rejection under the original plan}\mid\mathcal{F}_j)\), evaluated on histories eligible to continue. It is not the same as the chance of eventual rejection under an alternative hypothesis; that is a conditional-power question.
The definitions presume a specified null distribution, planned analysis times, and a specified rejection rule, including how earlier rejection or stopping is handled. If the trial has already crossed an efficacy boundary, there is no continuation history to which a future conditional error budget needs to be applied. When comparing an original and a proposed design, their earlier decisions and the null distribution of the interim data must be the same for the comparison below.
How Conditional Error and Spending Are Related
At a given look, the increment in alpha spending is the probability of first rejecting at that look. It is not the conditional error at a particular interim observation. The connection is an averaging identity: conditional probabilities, weighted by the null distribution of the histories that reach the relevant analysis, give the corresponding unconditional probability.
For example, suppose a trial has no rejection before interim \(j\), and future rejection can occur at one or more later looks. Let \(e_j(\mathcal{F}_j)\) be the original design’s conditional probability of any future rejection. On histories that have not already rejected, the probability of any eventual rejection is the probability of earlier rejection plus the probability of reaching the interim and then rejecting later. The latter term is:
Here \(I(\cdot)\) is 1 when its condition holds and 0 otherwise. The indicator is essential: histories with an earlier rejection are not continuation histories. If cumulative spending by look \(j\) is \(A(t_j)\) and final spending is \(\alpha\), then, under the original design, the displayed expectation equals \(\alpha-A(t_j)\). This is the remaining unconditional type I error probability, averaged over histories that survive to the interim; it is not the conditional error for every such history.
Worked Example: Conditional Error in a Two-Look Normal Design
Worked Example: Conditional Error in a Two-Look Normal Design
Consider a hypothetical two-look, one-sided trial. Under \(H_0\), the independent stage increments satisfy \(Y_1,Y_2\sim N(0,1)\). The first-look statistic is \(Z_1=Y_1\), and the final statistic is \(Z_2=(Y_1+Y_2)/\sqrt{2}\), so each statistic has a standard normal null distribution. The original plan rejects at look 1 if \(Z_1>2.40\); otherwise it rejects at the final look if \(Z_2>1.96\). Assume the increments are independent under \(H_0\), there is no other stopping rule, and the stated boundaries fully determine rejection.
Suppose the observed interim value is \(Y_1=1.20\). It does not cross 2.40, so continuation is possible. Given this observation, final rejection occurs exactly when
Because \(Y_2\sim N(0,1)\) under \(H_0\), the conditional error is \(1-\Phi(1.5719)\approx 0.0580\), or about 5.80%. This is the original plan’s future type I error probability at this particular history. At a different surviving history, \(Y_1=0\), the corresponding calculation is \(P(Y_2>\sqrt{2}(1.96))=1-\Phi(2.7719)\approx0.0028\). The two histories have different conditional errors even though they belong to the same analysis time.
The calculation conditions on the observed interim statistic and uses the null distribution of the unobserved increment. It does not claim that the observed value has a 5.80% probability of being significant, nor does it calculate power under an alternative.
Worked Example: Calculating Spending Increments
Worked Example: Calculating Spending Increments
Suppose a hypothetical trial uses a cumulative spending function \(A(t)=\alpha t^2\), with total one-sided \(\alpha=0.05\), at information fractions \(t_1=0.50\) and \(t_2=1.00\). The values of the function are:
Thus the first-look cumulative type I error target is 0.0125 (1.25%). The second-look increment is \(A(t_2)-A(t_1)=0.0500-0.0125=0.0375\) (3.75%). These are unconditional probabilities under \(H_0\): the first is the probability of rejection by the first look, and the second is the probability of first rejection at the second look, when the boundaries have been calibrated to meet these targets.
The spending function alone does not give a conditional error for a patient-independent interim statistic value such as \(Z_1=1.20\). To obtain that conditional probability, one also needs the test-statistic model and the actual boundaries. In a group-sequential design, those boundaries are commonly chosen so the cumulative null rejection probability at each planned look matches the spending target.
Worked Example: Averaging History-Specific Error
Worked Example: Averaging History-Specific Error
A simplified hypothetical design has two possible interim histories among trials that reach the interim: history \(h_1\), with null probability 0.60, and history \(h_2\), with null probability 0.40. Under the original continuation rule, the conditional probabilities of eventual future rejection are 0.02 at \(h_1\) and 0.08 at \(h_2\). These discrete probabilities are chosen to make the averaging explicit; they are not a normal approximation.
The conditional probability of future rejection, given that a trial reaches the interim, is obtained by weighting each conditional probability by its history probability:
The average is 4.40%, even though neither history has a conditional error of 4.40%. If a proposed rule instead has future rejection probabilities 0.01 and 0.07 at the two histories, its averaged future error is \(0.60(0.01)+0.40(0.07)=0.034\), or 3.40%. This lower average does not show that the proposed rule stays within the original conditional budget at every history: it does so here, because 0.01 is below 0.02 and 0.07 is below 0.08. In another proposal, a low probability at one history could offset an excess at another in the average. That is why the conditional comparison is more informative when preserving the original design’s history-specific error protection is the goal.
Checking a Changed Continuation Rule
Suppose the proposed design has the same earlier rejection decisions as the original design, and let \(C\) denote its total rejection event, including any earlier rejection handled identically. Let \(R\) denote the original design’s total rejection event, including any earlier rejection handled identically. A sufficient history-by-history condition for not increasing the null rejection probability is that, at every history with no earlier rejection, the proposed conditional probability of future rejection is no greater than the original conditional error. Averaging this inequality over histories that reach the interim preserves the comparison for future rejection probability. The common probability of earlier rejection must then be included in both designs.
More explicitly, write \(E_C(\mathcal{F}_j)\) for the proposed future rejection probability under \(H_0\), conditional on the interim information. If \(E_C(\mathcal{F}_j)\leq e_j(\mathcal{F}_j)\) on every eligible history, then:
The last equality uses the definition of the original conditional error and the fact that earlier rejection is handled identically. This comparison requires the same null distribution for the interim information and correct conditional probabilities for the proposed continuation. If the proposed design changes the interim data distribution under \(H_0\), the expectation cannot simply be taken using the original distribution.
Record its null model, planned analyses, boundaries, and earlier stopping or rejection rules.
For each eligible interim history, calculate the probability under the reference null model of any later rejection under the original plan.
Calculate its conditional future rejection probability using the proposed rule and its null distribution, at the same histories.
Compare the two conditional probabilities on the histories covered by the guarantee, then include the probability of common earlier rejections when assessing total type I error.
Assumptions, Diagnostics, and Alternatives
The conditional-error comparison is only as reliable as its probability model. The null distribution of interim and future statistics must be correctly specified, including their dependence. If information at the interim is incomplete, the conditional error must be calculated given the information actually available for the adaptation, not an unobserved quantity. If the continuation rule includes multiple later looks, its conditional probability must account for the probability of rejection at any of those looks, not just at the final analysis.
Useful diagnostics include checking that the original conditional error is between 0 and 1 on all relevant histories, confirming that it is computed only for histories eligible to continue, and independently verifying that its null-weighted expectation agrees with the original remaining unconditional error. Numerical integration or simulation can be used for complex distributions, but numerical error, tail accuracy, and the treatment of stopping histories should be checked. A discrepancy between the integrated conditional error and the known remaining alpha is a reason to audit the model or computation.
Alpha spending and conditional error are complementary, not competing names for the same method. Spending functions conveniently specify cumulative unconditional targets across planned analysis times and are useful for calibrating sequential boundaries. Conditional error is useful for evaluating a proposed continuation at the observed data history. A fixed-boundary design is another way to specify a rejection rule, but its boundaries alone do not express how much future rejection probability is available at every possible interim history. None of these summaries removes the need to state the null model and the decision rules.
Common Mistake and Limitations
- Mistake: comparing a proposed future rejection probability only with the original design’s total rejection probability. This mixes a conditional quantity with an unconditional one. Compare future conditional rejection with the original conditional error at the same history.
- Mistake: averaging proposed continuation rejection and forgetting earlier rejection. The average then bounds only the future component. Include the common probability of earlier rejection in both designs, as in the displayed total-error comparison.
- Mistake: treating a spending increment as the error budget at each interim value. An increment is an unconditional probability mass at an analysis time, not a pointwise allowance for every history.
- Limitation: a history-by-history conditional comparison can be conservative, and estimating conditional probabilities may be computationally demanding. A spending target can be simpler to communicate, but by itself it does not establish a pointwise conditional-error guarantee for a changed continuation rule.
The central practical question is therefore precise: is the goal to allocate an unconditional probability of rejection across analysis times, or to preserve a bound on future rejection probability conditional on the interim data? The answer determines which quantity must be calculated and checked.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A trial has cumulative spending \(A(t_1)=0.01\) and \(A(t_2)=0.04\). What is the alpha-spending increment at the second look, and what does it mean?
- In the two-look normal example, calculate the conditional error at \(Y_1=0\) using the final boundary 1.96.
- Why must an argument comparing conditional future rejection probabilities also account for earlier rejection?
- At two equally likely surviving histories, original conditional errors are 0.02 and 0.06. A proposal has conditional future rejection probabilities 0.01 and 0.08. Does its lower average establish that it is within the original budget at each history?
Solutions. (1) The increment is \(0.04-0.01=0.03\), or 3 percentage points. It is the unconditional probability of first rejection at the second look under \(H_0\), for boundaries calibrated to those cumulative targets. (2) The final rejection condition is \(Y_2>\sqrt{2}(1.96)-0=2.7719\), so the conditional error is \(1-\Phi(2.7719)\approx0.0028\), or 0.28%. (3) A comparison of future rejection probabilities bounds only the future component; total rejection probability also includes any earlier rejection. If earlier decisions are common, add their common probability to both total probabilities. (4) No. The proposal’s average is \(0.5(0.01)+0.5(0.08)=0.045\), which is higher than the original average \(0.5(0.02)+0.5(0.06)=0.040\). Regardless of averages, 0.08 exceeds the original history-specific budget of 0.06 at the second history, so the proposal is not within the original budget at every history.