Why the Conditioning Set Matters
An adaptation can use more than an interim treatment-effect estimate. Investigators might use baseline risk group, site, a prognostic score, or a randomization variable to decide how a trial proceeds. The conditional-error argument must account for the information that actually drives that decision. Otherwise, the calculation may average over differences that the adaptation has already used, and the claimed protection of type I error may not apply to the procedure that was implemented.
The key question is not simply whether a variable was measured before or after randomization. It is whether the null distribution used to calculate the original test's conditional error is correct given the information, and whether the conditioning set includes the information used for adaptation. Baseline variables are often convenient because their observed values can be treated as fixed in a conditional analysis, but their relationship to future outcomes still matters for the calculation.
The notation \(E_0\) and \(P_0\) refers to expectation and probability under the null model. The conditional-error requirement is applied at each realized value of the conditioning information, not just after averaging over it. This is the central safeguard when an adaptation is selected using interim information.
Ancillary Does Not Mean Irrelevant
In its strict statistical meaning, a statistic is ancillary for a parameter if its probability distribution does not depend on that parameter. In adaptive-trial discussions, “ancillary information” is also used more broadly for information—often baseline data or design information—that is not itself the treatment-effect evidence being tested but can be used to describe the interim state of the trial. These meanings should not be conflated. A baseline covariate can be strongly associated with outcomes and still be appropriate to condition on; conversely, calling a variable ancillary does not excuse using an incorrect conditional null distribution.
For example, a baseline prognostic group may predict the distribution of future outcomes under \(H_0\). That does not make it evidence of a treatment effect. It does mean the conditional-error calculation should use the appropriate distribution within that group. If a covariate is treated as fixed after it is observed, the analysis conditions on its realized value. If its distribution is modeled as random, the analyst must still verify that the conditional test is valid for the relevant null and any nuisance parameters.
A useful operational test is: Could the adaptation rule choose a different action if this variable had taken a different value, while the other conditioning information stayed the same? If yes, include that variable in \(\mathcal{F}\), or explicitly average over its randomization in a way that proves the same error bound. Include the realized randomization variable when the action depends on its realized value. Information not used by the adaptation need not be included merely because it was collected, although adding it is acceptable only if the conditional model remains valid and the conditional-error statement is established.
Worked Example: Conditional Error With a Baseline Risk Group
Consider a hypothetical two-stage one-sided normal test. Under \(H_0\), the stagewise statistics \(Z_1\) and \(Z_2\) are independent standard normal variables. The original test rejects when \(T=(Z_1+Z_2)/\sqrt{2}>1.96\). A baseline risk group \(G\), recorded before randomization, may be used in the adaptation. Assume the stated null model gives \(Z_2\sim N(0,1)\) conditional on \(Z_1\) and \(G\). At interim, \(Z_1=0.4\) and \(G\) is observed.
The assumptions needed here are explicit: the combination statistic has the stated form, \(Z_2\) is standard normal under \(H_0\) conditional on the observed information, and the stagewise statistics are independent under the null. If risk group changes the variance or mean of the future statistic, this calculation must be modified rather than used unchanged.
Given \(Z_1=0.4\), the original rejection event can be written as a threshold on \(Z_2\):
Thus the conditional error is \(A_0(0.4,G)=P_0(Z_2>2.3719\mid Z_1=0.4,G)=1-\Phi(2.3719)\approx0.00884\). Under the assumptions, this value is the same for either risk group. An adapted rule that uses \(G\) may use a different rejection event in each group, but the conditional probability of rejection must be no greater than 0.00884 for this realized interim information if it is to satisfy conditional-error preservation. The group label does not create extra type I error allowance.
Choosing and Using the Conditioning Set
The conditioning set should be stated before asserting preservation. It normally contains the observed interim information needed to calculate the original conditional error and all variables that determine the adaptation. Depending on the design, those variables can include stagewise statistics, baseline covariates, treatment assignments, accrued sample composition, and a randomization variable used to select an action. This is not a rule to condition on every recorded datum; it is a requirement to match the information set in the error calculation to the information set used by the adaptation.
The main result follows directly from conditional expectation. Suppose the original test has type I error at most \(\alpha\), and the adapted test satisfies the conditional-error inequality for every relevant null parameter value. Taking expectations on both sides gives the unconditional guarantee.
Define \(\mathcal{F}\) to include the observed quantities used by the adaptation, and calculate \(A_0=E_0(\phi_0\mid\mathcal{F})\) using their conditional null distribution.
For each realized value of \(\mathcal{F}\), verify that \(E_0(\phi_1\mid\mathcal{F})\leq A_0\). The check must use the actual adapted rejection rule and its correct conditional distribution.
Taking expectations yields \(E_0(\phi_1)\leq E_0(A_0)=E_0(\phi_0)\leq\alpha\).
This justification is short but important. Conditional error is not a separate source of type I error; it is the original test's rejection probability allocated at each observed interim state. Preserving or reducing that probability state by state ensures that averaging over all possible states cannot increase the original test's size.
Worked Example: A Covariate Changes the Future Variance
Suppose the hypothetical trial in the preceding example has a raw future-stage statistic \(Y_2\) whose null distribution depends on baseline risk group: \(Y_2\mid G=\text{low}\sim N(0,1)\) and \(Y_2\mid G=\text{high}\sim N(0,4)\). Assume \(Y_2\) is independent of \(Z_1\) conditional on \(G\). The original test uses the standardized statistic \(Z_2=Y_2/\sigma_G\), where \(\sigma_{\text{low}}=1\) and \(\sigma_{\text{high}}=2\), so that its stated conditional null distribution is standard normal.
At \(Z_1=0.4\), the original test still requires \(Z_2>2.3719\). In the high-risk group, this is equivalent to \(Y_2>2(2.3719)=4.7438\). Therefore, \(P_0(Y_2>4.7438\mid G=\text{high})=P_0(Z>2.3719)\approx0.00884\), where \(Z\sim N(0,1)\). The conditional error is calculated correctly when the group-specific scale is used.
If an analyst incorrectly treated the high-risk \(Y_2\) as standard normal and used the raw cutoff 2.3719, the calculated rejection probability would be \(P_0(Z>2.3719/2)=P_0(Z>1.18595)\), approximately 0.118. That is far above the actual conditional error of about 0.00884. The mistake is not that the covariate was included; it is that its effect on the conditional null distribution was ignored.
Adaptation, Randomization, and Composite Nulls
If an adaptation is randomized, the realized randomization can be treated as part of \(\mathcal{F}\) when the selected action depends on it. Alternatively, one can average over the randomization explicitly, but the calculation must reflect the implemented randomization probabilities. A statement about error conditional on the realized action is stronger than a statement averaged over a random action; either approach requires a correct accounting of that randomness.
For a composite null, the null distribution may depend on nuisance parameters. The conditional-error inequality must then be justified for every parameter value covered by the type I error claim, or by a valid conservative construction that supplies the required bound. A calculation under one convenient nuisance value alone does not establish uniform control. Conditioning on covariates can help describe the relevant distribution, but does not remove the need to address nuisance parameters.
Diagnostics should therefore check both the information set and the model. Confirm that every adaptation input is recorded and represented in \(\mathcal{F}\); check whether the conditional mean, variance, dependence structure, or randomization probabilities change across covariate values; and verify conditional error over plausible covariate patterns, not only at an average pattern. Simulation under the null can be useful as a check of the implemented procedure, especially when conditional distributions are complex, but it does not replace a correct definition of the null model or conditioning set.
Worked Example: Why a Marginal Error Comparison Is Not Enough
Let \(G\) be a baseline group with probability \(1/2\) for each of two values under \(H_0\), and let \(Z\sim N(0,1)\) be independent of \(G\). The original test rejects when \(Z>1.645\), giving approximately 0.05 type I error in each group. Consider an adapted rule that uses a lower cutoff \(Z>0.645\) for \(G=1\) and a higher cutoff \(Z>2.645\) for \(G=0\).
The conditional rejection probabilities are approximately \(1-\Phi(0.645)=0.259\) for \(G=1\), and \(1-\Phi(2.645)=0.00408\) for \(G=0\). The unconditional rejection probability is therefore \((0.259+0.00408)/2\approx0.132\), substantially above 0.05. More fundamentally, at \(G=1\) the adapted conditional rejection probability exceeds the original conditional error of 0.05. Averaging or appealing to the fact that one group has a stricter cutoff cannot justify preservation; the error calculation must address the information actually used to select the cutoff.
Common Pitfalls and Limits
- Calling a variable ancillary and stopping there. The label does not establish that the conditional null model is correct. State the relevant distribution given the observed information.
- Conditioning on a coarser set than the adaptation uses. If the adaptation responds to \(G\), an argument stated only for an average over \(G\) does not establish the statewise conditional-error inequality.
- Ignoring covariate-dependent variability. A common cutoff on a raw statistic may have different conditional error across groups. Use the appropriate group-specific distribution or valid standardization.
- Assuming baseline means statistically irrelevant. Baseline status does not imply independence from outcomes. It may predict outcomes under \(H_0\), so the conditional model may need to reflect it.
- Checking only one null parameter value. With nuisance parameters, a pointwise calculation may not support a uniform type I error claim.
There are limits to this framework. If the conditional distribution is poorly modeled, if adaptation inputs are incompletely recorded, or if the conditional-error bound cannot be established across the relevant null, the preservation claim is not secured. Some settings permit a simpler marginal argument, but it must be demonstrated for the actual randomized procedure rather than inferred from the fact that a variable is baseline or ancillary. The practical advantage of a carefully chosen conditioning set is transparency: it makes clear what the adaptation observed and what probability bound must be maintained.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In the two-stage normal example, calculate the conditional error at \(Z_1=0.4\) when the original rejection rule is \((Z_1+Z_2)/\sqrt{2}>1.96\).
- In a high-risk group, \(Y_2\mid G\sim N(0,4)\) and \(Z_2=Y_2/2\). What raw \(Y_2\) cutoff corresponds to the original rule at \(Z_1=0.4\), and why must the group-specific scale be used?
- Why does calling a baseline covariate ancillary not by itself establish conditional-error preservation?
- An adaptation uses a baseline group and a realized randomization coin to choose an action. What should the conditioning information include, or what must an alternative calculation do?
Solutions. (1) Rearranging gives \(Z_2>1.96\sqrt{2}-0.4=2.3719\). Thus the conditional error is \(1-\Phi(2.3719)\approx0.00884\). (2) The corresponding cutoff is \(Y_2>2(2.3719)=4.7438\). Since the conditional standard deviation is 2, using the group-specific scale gives the correct null tail probability, approximately 0.00884; treating \(Y_2\) as standard normal would miscalculate that probability. (3) Ancillarity describes parameter-dependence of a distribution; it does not supply the conditional null distribution needed for the calculation or prove that the adapted rejection probability is bounded state by state. (4) Include both the group and the realized coin in the conditioning information when they determine the action, or explicitly average over the coin using its actual randomization probabilities and prove the corresponding error bound.