Proof By Conditional Expectation

Why a Conditional Argument Controls Overall Error

An adaptive trial may use interim information to choose how to continue. That choice changes the rule applied to future data, so the original test’s overall Type I error cannot simply be assumed to carry over. A useful way to justify a modification is to compare the two tests at each possible interim history: under the null, the modified rule must have no greater conditional probability of rejection than the original rule had at that history.

The key step is then to average those conditional probabilities over the interim histories that could occur. The law of iterated expectation says that this average is the test’s overall rejection probability. This tutorial develops that argument and makes explicit which conditional probability is being compared.

A rejection indicator equals 1 when the test rejects and 0 otherwise. The conditional expectation notation also accommodates randomized tests: if \(\phi\) is a conditional rejection probability between 0 and 1, the same argument applies. In either case, the conditioning must use the information actually available when the adaptation is made. The conditional distribution of future data is taken under the null parameter value being considered.

The Conditional-Expectation Argument

Write \(\phi_0\) for the original test’s rejection indicator and \(\phi_1\) for the modified test’s rejection indicator. For a fixed null parameter value \(\theta\), define the original conditional error by \(A_\theta=E_\theta(\phi_0\mid\mathcal{F})\). The modified test’s conditional rejection probability at the adaptation time is \(E_\theta(\phi_1\mid\mathcal{F})\). A sufficient condition for preserving the original test’s error bound is that the latter does not exceed the former, almost surely under that null parameter value.

The almost-sure qualification is important: the conditional inequality must hold except possibly on interim histories having probability zero under the null parameter value in question. For a composite null, the argument is applied separately at each \(\theta\in H_0\). The conditional distribution used to compute each expectation is the one under that same \(\theta\).

The first and third equalities are applications of the law of iterated expectation: averaging a conditional expectation over the conditioning information recovers the original expectation. The inequality in the middle follows from the assumed conditional comparison. This proves the result without requiring the modified test to have the same form as the original test.

Applying the Argument to a Two-Stage Normal Test

A normal model makes the conditional rejection probability especially clear. Suppose a planned test uses independent stage statistics \(Z_1\) and \(Z_2\), each standard normal under a fixed null hypothesis, and rejects when the weighted combination exceeds a critical value. Let the information fraction be \(t=0.5\), and let the rejection rule be \(\sqrt{t}Z_1+\sqrt{1-t}Z_2>1.96\). Under the null, the combined statistic is standard normal, so the original test’s rejection probability is \(P(Z>1.96)\), approximately 0.0250. The assumptions used here are independence of the two stage statistics under this null and their standard normal distributions.

At the adaptation time, \(Z_1=z_1\) is observed. Conditional on that value, only \(Z_2\) remains random. Rearranging the original rejection rule gives its conditional error:

where \(\Phi\) is the standard normal cumulative distribution function. The second equality uses the stated independence: conditional on \(Z_1=z_1\), \(Z_2\) is still standard normal.

Checking the Condition and Recognizing Failure

For a proposed adaptation, the proof reduces to a specific diagnostic: for each relevant interim history, calculate or bound the modified test’s conditional probability of eventual rejection and compare it with the original test’s conditional error at that same history. A marginal calculation alone does not establish this pointwise condition. Nor does it suffice to compare two conditional probabilities evaluated under different null models or different conditioning information.

A common mistake is to say, “The original test has level \(\alpha\), so a modified test that uses the same interim data also has level \(\alpha\).” Sharing interim data does not imply that the modified test has a controlled conditional rejection probability. The correction is to write down both conditional rejection probabilities under the null, verify the inequality at the adaptation time, and only then average using iterated expectation.

This conditional proof differs from direct unconditional recalibration. An unconditional argument calculates the modified rule’s rejection probability after averaging over interim outcomes; it can be valid if that distribution and the post-adaptation rule are correctly accounted for. The conditional-error argument instead checks a bound at each interim history, then obtains the unconditional guarantee by averaging. Its assumptions are explicit, but calculating the conditional probabilities can be difficult when the future-data distribution is complex or when nuisance parameters affect it.