Why the Conditioning Information Matters
A conditional error function describes the probability, under the null hypothesis, that a specified test will reject given the information available at an interim analysis. The phrase “given the information” is essential. A trial history may include treatment assignments, individual outcomes, timing, and data-quality information, while a summary statistic may retain only one number. Conditioning on these two information sets need not produce the same conditional probability.
The choice should reflect the trial’s actual interim information and the original test’s null distribution. Conditioning on too little information can average over histories that have different continuation error probabilities. Conditioning on information not available at the interim analysis, by contrast, does not describe a usable interim conditional error.
The rejection event \(R\) includes the entire prespecified testing procedure, including its planned continuation and final decision rule. The conditional error is therefore a random variable: it is a function of the interim history, even though its value is determined by the null model and the original test. For a composite null, the definition is applied under each \(\theta\in\Theta_0\); any required uniform error control must hold over the null parameter values specified for the procedure.
The interim sigma-field should include information genuinely available when the adaptation decision is made, such as the observed treatment assignments and outcomes, the data cutoff, and any interim summaries used by the design. It should not include future outcomes. If the adaptation rule also uses operational information, such as a prespecified site-level accrual measure, that information belongs in the conditioning set if it is available and relevant to the rule.
Full History and Summary Statistics
A summary statistic \(T\) is a function of the full interim data. Consequently, \(\sigma(T)\subseteq\mathcal F_I\): knowing the full history always determines the summary, but knowing the summary need not reconstruct the history. Conditioning on \(T\) averages over all full histories compatible with the observed value of \(T\).
This averaging has an exact relationship to the full-history conditional error. Under the same null model and for the same original test, if \(\mathcal G_I=\sigma(T)\subseteq\mathcal F_I\), then
This identity is a consequence of conditional expectation, not a claim that the two conditional errors are equal for each history. The summary-based value is the average full-history conditional error among histories with that summary. Equality holds when the full-history conditional error is already a function of the summary, apart from null-probability cases.
State which fixed test is being evaluated and the null distribution under which its rejection probability is defined.
Include all observed variables the decision process can use, but exclude future data.
If using a summary, state exactly what it retains and what history it discards.
Ask whether the discarded information can change the null distribution of the remaining test or its probability of rejection.
Worked Examples
Worked Example: Averaging Over Histories With the Same Summary
Consider a hypothetical trial under \(H_0\) with two binary interim observations \(X_1,X_2\), each independently equal to 0 or 1 with probability \(1/2\). Let the interim history be \(H=(X_1,X_2)\), and let the summary be \(T=X_1+X_2\). Suppose the original test uses an independent continuation random variable and, under \(H_0\), its conditional rejection probabilities for the four histories are specified as follows: \(0.20\) for \((0,0)\), \(0.10\) for \((1,0)\), \(0.70\) for \((0,1)\), and \(0.20\) for \((1,1)\). These probabilities are part of this hypothetical test’s defined null model.
Given the full history, the conditional error is the listed probability. For example, \(CE_{H_0}(\sigma(H))=0.10\) after observing \((1,0)\), and it is \(0.70\) after observing \((0,1)\).
If only \(T=1\) is retained, the compatible histories are \((1,0)\) and \((0,1)\). Each has conditional probability \(1/2\), since the four histories are equally likely under \(H_0\). Thus
For \(T=0\), only \((0,0)\) is possible, so the conditional error is \(0.20\). For \(T=2\), only \((1,1)\) is possible, also giving \(0.20\). The two histories with \(T=1\) therefore have different full-history conditional errors, although the summary-based conditional error is \(0.40\) for both. The summary is not sufficient for this conditional error function.
Worked Example: A Summary That Omits Site Information
Suppose a hypothetical two-site trial has an interim efficacy summary \(T=0\). Under the stated null model for the original test, the conditional probability of eventual rejection given the full interim history is \(0.10\) at Site A and \(0.50\) at Site B. Assume that, among histories with \(T=0\), the site is A with probability \(0.75\) and B with probability \(0.25\). These conditional probabilities and weights are stipulated by this example’s null model.
If the interim information is the full history, including site, the conditional error is \(0.10\) for a Site A history and \(0.50\) for a Site B history. If the analysis conditions only on \(T\), it averages across the two site histories:
The value \(0.200\) is correct for conditioning only on \(T=0\), but it is not the full-history conditional error at either site. If site information affects the null distribution of the future test statistic, treating the summary-based value as the conditional error for each site would misstate the history-specific error. The site distinction may be omitted only if it does not alter the conditional error, or if the intended calculation explicitly averages over sites as above.
Worked Example: Conditioning on the Information Used by an Adaptation Rule
In a hypothetical trial, an interim rule uses both a binary efficacy summary \(T\) and an operational indicator \(D\), where \(D=1\) means that a prespecified accrual condition has been met. At the observed efficacy summary \(T=1\), suppose the original test’s null conditional rejection probabilities are \(0.15\) when \(D=0\) and \(0.45\) when \(D=1\). Suppose that, conditional on \(T=1\), the null probability of \(D=1\) is \(0.40\).
Because the adaptation rule uses \(D\), the full interim information includes it. The full-history conditional error is \(0.15\) or \(0.45\), according to the observed value of \(D\). A calculation conditioning only on \(T=1\) instead gives
The value \(0.27\) is an average over the two operational states, not the conditional error for either observed state. Since the rule uses \(D\), conditioning on \(T\) alone does not describe the information on which the actual adaptation decision is based. Including \(D\) is necessary for a history-specific calculation unless the conditional error is the same in both states.
Justification and Practical Choice
The averaging identity follows directly from the defining property of conditional expectation. Let \(I_R\) be the indicator of rejection, equal to 1 if the original test rejects and 0 otherwise. Since \(CE_{\theta}(\mathcal F_I)=E_{\theta}(I_R\mid\mathcal F_I)\), and \(\mathcal G_I\subseteq\mathcal F_I\), iterated conditioning gives
This derivation also clarifies what a summary-based calculation means: it averages the original test’s rejection indicator over the null distribution of full histories given the summary. It does not replace that distribution with an arbitrary or convenient one.
A useful practical choice is to condition on the full interim history when the continuation error may depend on details beyond the proposed summary, particularly details used by the adaptation rule. A lower-dimensional summary can be appropriate when the conditional error is demonstrably a function of that summary, or when the target is explicitly the error averaged over histories given that summary. Full-history conditioning is not automatically preferable in every calculation: it can be computationally difficult, and a highly detailed representation requires a correctly specified conditional null model.
The relevant comparison is not “large information set is conservative, small information set is liberal.” Neither ordering holds in general for a particular observed history. The full-history value may be above or below the summary average; the summary value averages across histories. Any comparison must preserve the same original test and null distribution.
Diagnostics, Alternatives, and Limitations
Before using a conditional error function, check that the proposed conditioning set matches the information actually observed at the interim analysis. Check also that the null model gives a valid conditional distribution for the remaining data, including any dependence induced by the trial design. If a statistic is used as a summary, verify whether histories sharing that statistic can have different conditional rejection probabilities.
- Omitted adaptation inputs: If the adaptation rule uses an observed variable absent from the conditioning set, the calculated value may average over decisions or states that are already known.
- Unjustified sufficiency: Equal summary values do not imply equal conditional errors. The summary must retain the information relevant to the original test’s remaining rejection probability.
- Future information: Conditioning on outcomes not yet observed describes a different probability calculation and cannot define an interim-available error function.
- Wrong averaging weights: A summary-based conditional error uses the null conditional probabilities of the full histories given that summary, not equal weights unless those histories are equally likely.
- Model mismatch: If the assumed conditional distribution ignores dependence, randomization, or other design features, the resulting conditional error may not represent the stated test.
The main alternative to full-history conditioning is conditioning on a coarser sigma-field generated by a prespecified statistic or set of summaries. This can simplify computation and communication, but it is valid for the intended purpose only when the averaging implicit in that coarser field is understood. An intermediate information set is also possible: it may retain treatment group, site, or timing while discarding individual-level detail. The right choice is the smallest information set that answers the intended conditional question without averaging away distinctions that matter.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- If \(\mathcal G_I=\sigma(T)\subseteq\mathcal F_I\), how is \(CE_{\theta}(\mathcal G_I)\) related to \(CE_{\theta}(\mathcal F_I)\)?
- Two histories have the same interim summary but full-history conditional errors of \(0.08\) and \(0.32\). If they are equally likely given the summary, what is the summary-based conditional error?
- A trial’s adaptation rule uses site and an efficacy statistic, but the proposed conditioning set contains only the efficacy statistic. What should be checked?
- Is the conditional error based on a summary always greater than, less than, or equal to the full-history conditional error for an observed history? Explain.
Solutions. (1) The summary-based conditional error is the conditional expectation of the full-history value given \(T\): \(CE_{\theta}(\mathcal G_I)=E_{\theta}[CE_{\theta}(\mathcal F_I)\mid\mathcal G_I]\). (2) It is \((0.08+0.32)/2=0.20\), using the stipulated equal conditional weights. (3) Check whether site changes the original test’s conditional rejection probability under the null. Because the adaptation rule uses site, include it for a history-specific calculation unless the conditional error is invariant to site. (4) None of those orderings is guaranteed. The summary value averages full-history values over compatible histories; an individual value can be above or below that average, or equal to it.