Why the Null Parameter Value Matters
A conditional-error calculation is tied to a probability model. If the null hypothesis contains more than one possible parameter value, the probability of rejection after a particular interim history can differ across those values. Controlling it at one convenient value does not, by itself, control it at the others. This matters whenever an adaptive change is justified by saying that it uses “no more than the available conditional error”: the comparison must be made under every parameter value allowed by the null.
The issue is distinct from choosing a conservative estimate of an unknown parameter. The question here is the logical scope of the guarantee. A composite null is a set of models, and Type I error control must hold under each model in that set. The earlier material on the conditional error bound establishes the role of the conditional rejection probability; here the key point is to index that probability by the null parameter and preserve the bound pointwise.
The probability law may change with \(\theta\), even when the test rules themselves are fixed and do not use the unknown parameter. Consequently, a rule that is operationally well-defined can have different conditional rejection probabilities at different null values. Conversely, a table of conditional probabilities indexed by \(\theta\) is not automatically an implementable rule: it may simply be an abstract description of the probabilities that must be compared.
Pointwise Preservation and Overall Type I Error
The Type I error probability at a particular null value is \(P_\theta(R)\). For a composite null, control at level \(\alpha\) means \[ \sup_{\theta\in\Theta_0}P_\theta(R)\le\alpha, \] or equivalently \(P_\theta(R)\le\alpha\) for every \(\theta\in\Theta_0\). A bound at a single value \(\theta_0\) says nothing by itself about values elsewhere in \(\Theta_0\).
The conditional comparison gives a direct route to this guarantee. If the original test has Type I error at most \(\alpha\) at every null value, and the revised test has no greater conditional rejection probability at every null value and almost every history under that value, then the revised test also has Type I error at most \(\alpha\) throughout the null.
The condition is parameter-by-parameter. “Almost surely” allows histories having probability zero under a particular \(\theta\) to be disregarded for that \(\theta\); it does not allow a history that occurs under one null value to be ignored just because it is rare under another. For a continuous history, conditional probabilities are defined up to probability-zero sets, so the inequality is required almost surely under each null model.
Why the Result Holds
Condition on the interim history, then average over its distribution under the parameter value being considered. The identity used is the law of total probability for a conditional rejection probability. Applying it separately at each \(\theta\) proves the claim.
The inequality between the expectations follows from the assumed almost-sure inequality under that same \(P_\theta\). This is why the condition must be checked under each null value: each expectation uses the history distribution and conditional future-data distribution belonging to that value. A bound averaged over parameter values, or verified only for one of them, does not establish the displayed inequality for the others.
Worked Example: A Single-Parameter Check Misses an Inflation
Consider an abstract conditional-error illustration with two null values, \(\theta_A\) and \(\theta_B\), and two interim histories, \(A\) and \(B\). Suppose the original and revised conditional rejection probabilities have the following values. Treat the entries as conditional probabilities for comparison, not as a proposed rule that uses the unknown \(\theta\) as a threshold.
| Null value | History | Original \(C_\theta(s)\) | Revised \(C^*_\theta(s)\) | Comparison |
|---|---|---|---|---|
| \(\theta_A\) | A | 0.41 | 0.10 | 0.10 ≤ 0.41 |
| \(\theta_A\) | B | 0.01 | 0.41 | 0.41 > 0.01 |
| \(\theta_B\) | A | 0.20 | 0.15 | 0.15 ≤ 0.20 |
| \(\theta_B\) | B | 0.30 | 0.25 | 0.25 ≤ 0.30 |
At history A under \(\theta_A\), the revised probability is no greater than the original one: \(0.10\le0.41\). At history B under \(\theta_A\), it exceeds the original one: \(0.41>0.01\). Thus, even a comparison at one null value fails to establish preservation over histories. At \(\theta_B\), both displayed comparisons pass, but that does not repair the failure at \((\theta_A,B)\). To infer preservation, every history with positive probability under each null value must satisfy the bound. The table is only an abstract illustration of the comparisons; it does not specify a parameter-dependent operational test.
An Operational Example: Fixed Rules, Different Null Values
The next example shows how parameter-indexed conditional probabilities can arise from rules that never use the unknown parameter. Let the interim history be \(S\), and let the continuation observation \(X\) be Bernoulli with success probability \(\theta\), independently of \(S\). The composite null is \(\Theta_0=[0,0.5]\). The original rule rejects when \(X=1\); a proposed revised rule rejects when \(X=0\). These are fixed, parameter-independent rules.
Worked Example: Checking a Bernoulli Continuation Rule
Because \(X\) is independent of \(S\), conditioning on any possible interim history does not change its Bernoulli distribution. Under the original rule,
Under the revised rule,
At the convenient boundary value \(\theta=0.5\), the comparison passes with equality: \(C^*_{0.5}(s)=0.5=C_{0.5}(s)\). But at the null value \(\theta=0.2\),
The original test has size \(\sup_{\theta\in[0,0.5]}\theta=0.5\). The revised test has size \(\sup_{\theta\in[0,0.5]}(1-\theta)=1\), attained at \(\theta=0\). Thus passing the conditional comparison at \(\theta=0.5\) does not protect the composite null. The example also clarifies that the conditional probability depends on \(\theta\) through the distribution of \(X\), not through a test threshold that requires knowing \(\theta\).
When the Bound Holds at Every Null Value
A useful check is to tabulate conditional probabilities across both null values and possible histories, then verify the inequality cell by cell. The following hypothetical values illustrate the averaging step when every comparison passes. They are stated as conditional probabilities supplied by the respective test models; the table alone is not an operational specification of either test.
Worked Example: Averaging Pointwise Conditional Bounds
Suppose the null contains \(\theta_1\) and \(\theta_2\). At \(\theta_1\), the interim history is A with probability 0.60 and B with probability 0.40. At \(\theta_2\), those probabilities are 0.20 and 0.80. The conditional probabilities are:
| Parameter | History | \(P_\theta(S=s)\) | Original \(C_\theta(s)\) | Revised \(C^*_\theta(s)\) |
|---|---|---|---|---|
| \(\theta_1\) | A | 0.60 | 0.10 | 0.08 |
| \(\theta_1\) | B | 0.40 | 0.30 | 0.20 |
| \(\theta_2\) | A | 0.20 | 0.25 | 0.20 |
| \(\theta_2\) | B | 0.80 | 0.05 | 0.04 |
Each revised conditional probability is no greater than its corresponding original probability. At \(\theta_1\), the original and revised unconditional rejection probabilities are, respectively,
At \(\theta_2\), they are
For this two-value null, the original worst-case error is \(\max(0.18,0.09)=0.18\), and the revised worst-case error is \(\max(0.128,0.072)=0.128\). The revised test therefore does not increase the Type I error at either null value. In practice, the same reasoning must cover the full null parameter set, not just a convenient finite selection from it.
Diagnostics, Alternatives, and Limits
A practical audit should first state the composite null precisely and identify the conditioning history. Then, for a fixed null value, calculate or bound both conditional rejection probabilities using that value’s probability model. Repeat the comparison throughout the null. If a proposed adaptation includes randomization, its randomization rule must be part of the defined test; the conditional probability may average over that randomization, but the rule itself must not require the unknown true parameter.
Common warning signs include a calculation that substitutes a single “least favorable” value without proving it is least favorable for the conditional comparison, an average over null values presented as a uniform guarantee, or a history omitted from the conditioning information even though the adaptation uses it. Another warning sign is comparing probabilities calculated under different parameter values or different conditioning events. Each side of \(C^*_\theta(s)\le C_\theta(s)\) must refer to the same \(\theta\) and the same observed history.
| Approach | What it establishes | Main caution |
|---|---|---|
| Pointwise conditional comparison | Preservation if the bound holds for every null value and almost every relevant history | Requires a valid conditional calculation throughout the null |
| Direct unconditional size calculation | Control if the revised rejection probability is bounded by \(\alpha\) at every null value | May require a difficult supremum calculation |
| Check at one null value | A result only for that value | Does not establish control of a composite null |
| Average over null values | An average under the chosen weighting distribution | Does not guarantee control at each null value |
Directly bounding \(\sup_{\theta\in\Theta_0}P_\theta(R^*)\) is a valid alternative to a conditional-error argument. The conditional approach can be useful because it compares the revised procedure to an already valid test history by history. Neither approach removes the need to state the model and null set. A bound that is correct under a misspecified conditional distribution need not provide the claimed error control.
A particularly common mistake is: “The revised conditional error is below the original one at the null boundary, so the adaptation preserves level.” This reasoning is incomplete unless the conditional inequality is known to hold at every null value. The correction is to establish the pointwise inequality for all \(\theta\in\Theta_0\), or to use another argument that directly proves the revised test’s unconditional error is at most \(\alpha\) for all such \(\theta\). If the comparison fails at even one relevant parameter-history pair, the conditional-preservation theorem cannot be invoked.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A null hypothesis permits \(\theta\in[0,1]\). A conditional bound is verified only at \(\theta=0.5\). What does this establish, and what does it leave unestablished?
- At a particular null value, a history has probability 0.25. The original conditional rejection probability is 0.12 and the revised one is 0.09. What is the revised contribution of this history to the unconditional rejection probability?
- An analyst averages conditional errors over two null values and finds the revised average is smaller. Why is that not sufficient for composite-null control?
- Under \(H_0:\theta\in[0,0.5]\), an original rule rejects when \(X=1\), where \(X\sim\text{Bernoulli}(\theta)\). At \(\theta=0.3\), find its conditional error given an independent interim history.
Solutions. 1. It establishes a comparison only at \(\theta=0.5\) (and for histories covered by that check); it does not establish the required inequality at other null values. 2. The contribution is \(0.25(0.09)=0.0225\). 3. An average can be small even if the revised error exceeds the allowable bound at one null value; control requires a guarantee at each value. 4. Independence means conditioning on the interim history does not change the distribution of \(X\), so \(P_{\theta=0.3}(X=1\mid S=s)=0.3\).