Intersection Hypotheses in Multiple Testing

Why Intersections Matter

In a multiple-testing problem, the truth of the elementary null hypotheses can vary with the unknown parameter. Strong familywise error control must therefore apply not only when every null is true, but also when only some are true. An intersection hypothesis gives a precise way to represent the simultaneous truth of a selected group of elementary nulls. It links a particular configuration of true nulls to the error event that matters in that configuration.

The central point is logical as well as statistical: an intersection requires all the nulls in its selected group to be true, but says nothing about nulls outside that group. Keeping this distinction clear is essential when reasoning about error control.

Definition: Let $\Theta$ be the parameter space, and let $H_1,\ldots,H_K$ be elementary null hypotheses, each represented as a subset of $\Theta$. For a nonempty index set $I\subseteq\{1,\ldots,K\}$, the intersection hypothesis is $$ H_I=\bigcap_{i\in I}H_i. $$ It is true at parameter value $\theta$ precisely when $\theta\in H_i$ for every $i\in I$. Its alternative is the complement $\Theta\setminus H_I$: at least one null indexed by $I$ is false.

An intersection may be written by listing its indices, such as $H_{\{1,3\}}$, or as $H_{1,3}$ when the meaning is clear. The index set identifies which elementary hypotheses are combined; it does not specify what happens to the other hypotheses. If $I$ contains every index, $H_I$ is the global intersection, requiring every elementary null to be true.

Constructing Intersections From Clinical Hypotheses

Suppose a trial compares two experimental treatments with a control on an outcome where larger values are better. Let $\delta_1$ and $\delta_2$ be the population mean differences (experimental treatment minus control), measured in outcome points. Consider the one-sided elementary nulls $H_1:\delta_1\leq 0$ and $H_2:\delta_2\leq 0$. These nulls assert that the respective treatment does not improve the population mean over control.

Worked Example: Combining Two Treatment Nulls

Construct the intersection for the two treatment comparisons, and state its alternative in words and symbols.

The selected index set is $I=\{1,2\}$. By definition, the intersection requires both elementary nulls to hold: $$ H_{\{1,2\}}=\{\theta:\delta_1\leq 0\text{ and }\delta_2\leq 0\}. $$ Here $\theta$ includes the parameters $\delta_1$ and $\delta_2$. The alternative is the complement of this set. The negation of “$\delta_1\leq0$ and $\delta_2\leq0$” is “$\delta_1>0$ or $\delta_2>0$,” so $$ H_{a,\{1,2\}}:\delta_1>0\text{ or }\delta_2>0. $$ Thus the intersection null says that neither experimental treatment improves the population mean over control. Its alternative says that at least one does. It does not require both treatments to improve the outcome.

In general, the alternative of an intersection is a union of elementary alternatives: at least one selected null fails. The word “or” is inclusive. For example, $\delta_1>0$ or $\delta_2>0$ includes the case where both differences are positive.

Intersections and True-Null Configurations

For each possible parameter value, define the true-null set as the indices of all elementary null hypotheses that contain that value. This set describes the actual configuration of true and false elementary nulls at that parameter value. The intersection indexed by this set is therefore a natural summary of the nulls that could be falsely rejected there.

Definition: At parameter value $\theta$, the true-null set is $$ T(\theta)=\{i\in\{1,\ldots,K\}:\theta\in H_i\}. $$ For any nonempty set $T$, the intersection $H_T$ requires all elementary nulls indexed by $T$ to be true. If $T=T(\theta)$, then $\theta\in H_T$; nulls outside $T$ are false at $\theta$ by the definition of the true-null set.

A common logical error is to treat $H_T$ as a statement that exactly the nulls in $T$ are true. It is not: membership in $H_T$ imposes no restrictions on nulls outside $T$. The exact configuration is specified by $T(\theta)$, not by the intersection alone.

Worked Example: Membership Is Not an Exact Configuration

Suppose there are three elementary hypotheses $H_i:\delta_i\leq0$, for $i=1,2,3$, and consider the parameter value $(\delta_1,\delta_2,\delta_3)=(0,-0.4,0.2)$, with differences measured in outcome points. Find its true-null set and check membership in two intersections.

For $H_1$, $0\leq0$, so $H_1$ is true. For $H_2$, $-0.4\leq0$, so $H_2$ is true. For $H_3$, $0.2\leq0$ is false. Therefore $$ T(\theta)=\{1,2\}. $$ The parameter value belongs to $H_{\{1,2\}}$, because both of its defining inequalities hold. It does not belong to $H_{\{1,3\}}$, because the inequality defining $H_3$ fails.

Now consider a different parameter value, $(\delta_1,\delta_2,\delta_3)=(-0.1,-0.2,-0.3)$. It belongs to $H_{\{1,3\}}$, since $\delta_1\leq0$ and $\delta_3\leq0$. Its true-null set is $\{1,2,3\}$, not $\{1,3\}$. This confirms that the intersection requires the indexed nulls to be true but does not require every other null to be false.

The Link to Strong Familywise Error Control

Let $R$ be the set of elementary hypotheses rejected by a multiple-testing procedure, and let $T=T(\theta)$ be the true-null set at the data-generating parameter value. A familywise error occurs when the procedure rejects at least one true null. Equivalently, at least one rejected index lies in $T$.

\{\text{at least one true null is rejected}\} = \bigcup_{i\in T}\{i\in R\}.

Strong familywise error control at level $\alpha$ means that this event has probability at most $\alpha$ for every parameter value $\theta$, including values for which some elementary nulls are true and others are false. If $T$ is nonempty, then $\theta\in H_T$. Consequently, a test of $H_T$ must be calibrated over parameter values where all nulls indexed by $T$ are true if it is to provide a valid bound for that configuration.

Definition: A test of an intersection hypothesis $H_I$ has level $\alpha$ if, when the data follow the trial’s specified model, $$ \sup_{\theta\in H_I}P_\theta(\text{reject }H_I)\leq\alpha. $$ This requirement bounds the probability of rejecting that intersection for every parameter value at which all its indexed elementary nulls are true.

This gives the key reason intersections are central: each possible nonempty true-null set $T$ points to a corresponding hypothesis $H_T$, and the familywise error event at that configuration concerns rejection among the indices in $T$. A strong-control strategy must account for every such configuration, not only the global intersection. Tests of the relevant intersections provide the error bounds a strategy can use.

The connection requires care. Merely having a level-$\alpha$ test for each intersection does not, by itself, guarantee strong familywise control for any arbitrary way of making elementary rejection decisions. The rule that connects intersection-test outcomes to elementary rejections must ensure that, under a true-null configuration $T$, a familywise error is bounded by a valid level-$\alpha$ event for $H_T$. The construction of particular local tests and the procedure that assembles them are separate questions.

A Short Justification of the Error-Control Connection

The logic can be stated directly. Fix a parameter value $\theta$, and suppose its true-null set $T=T(\theta)$ is nonempty. Then $\theta\in H_T$. Suppose a multiple-testing rule has the property that, at this configuration, rejecting any true elementary null entails rejection of $H_T$. The familywise error event is then a subset of the event of rejecting $H_T$. Therefore,

P_\theta(\text{at least one true null is rejected}) \leq P_\theta(\text{reject }H_T) \leq \sup_{\vartheta\in H_T}P_\vartheta(\text{reject }H_T) \leq\alpha.

The first inequality follows from the stated event containment. The second follows because the actual $\theta$ lies in $H_T$. The last follows from the level-$\alpha$ condition. Since the argument applies to every parameter value with a nonempty true-null set, it explains why accounting for all relevant intersections is the natural route to strong control. If $T$ is empty, no true elementary null can be rejected, so the familywise error event is empty.

Why Separate Elementary Tests Can Be Insufficient

Testing every elementary null at level $\alpha$ does not generally control the probability of making at least one false rejection across the family. The following calculation uses a deliberately simple model to show the issue. It is not a recommendation about how to analyze a clinical trial.

Worked Example: Three Independent Tests Under the Global Null

Suppose three elementary tests are each carried out at $\alpha=0.05$. Assume all three nulls are true and that, under this configuration, their p-values are independent and each is uniformly distributed on $[0,1]$. Calculate the probability of at least one false rejection when each null is rejected if its p-value is at most 0.05.

For each test, the probability of not rejecting its true null is $1-0.05=0.95$. Independence gives the probability of no false rejection in all three tests as $$ 0.95^3=0.857375. $$ Thus the familywise error probability is $$ 1-0.95^3=1-0.857375=0.142625\approx0.1426. $$ The resulting probability, about $14.26\%$, exceeds $5\%$. The calculation depends on the stated independence and uniformity assumptions; without independence, this numerical expression need not hold. The broader lesson does not depend on this example: separate level-$\alpha$ guarantees do not automatically guarantee a level-$\alpha$ familywise error rate.

The global intersection $H_{\{1,2,3\}}$ represents the configuration in which all three nulls are true. But strong control also concerns partial configurations, such as $T=\{1,3\}$, where those two nulls are true and the second is false. A strategy justified only at the global intersection has not, on that basis alone, established control for every possible configuration.

Assumptions, Diagnostics, and Limitations

The set-theoretic definitions require only a specified parameter space and well-defined elementary hypotheses. Probabilistic error claims require more: a data-generating model under which probabilities $P_\theta$ are defined, and a test whose level condition holds for every parameter value in the intersection null. In a clinical trial, the model and calibration must match the design and analysis being used. If design adaptation affects the distribution of a test statistic, a calibration justified only for a different design cannot automatically be used for the adapted design.

Several checks help prevent errors in constructing or interpreting intersections:

Check the quantifiers. An intersection requires every indexed null to hold: $\theta\in H_I$ means $\theta\in H_i$ for all $i\in I$.
Negate the statement carefully. The alternative to “all selected nulls are true” is “at least one selected null is false,” not “all selected nulls are false.”
Separate membership from configuration. $H_I$ says nothing about indices outside $I$. Use $T(\theta)$ to describe the full true-null set.
Check whether the intersection is possible. Some collections of nulls may have no common parameter value, making their intersection empty. A test’s level condition over an empty null set is not a useful operating guarantee for a realizable configuration.
Check for duplicates. Different index sets can define the same subset of parameter space. They remain distinct index collections, even if the resulting hypotheses coincide.

Intersections also clarify the difference between a global-null guarantee and strong control. A procedure may have an error bound when all nulls are true but still lack a guarantee when only a subset is true. Strong control asks for the bound at every parameter value, so all nonempty true-null sets must be considered. Conversely, intersection hypotheses do not prescribe one unique procedure: they identify the null statements that need to be addressed, while the test construction and the rule for making elementary rejections determine how those statements are used.

Common Pitfall

Common mistake: “$H_{\{1,3\}}$ is true exactly when $H_1$ and $H_3$ are true and $H_2$ is false.” This adds a condition that the intersection does not contain. The correct statement is that $H_{\{1,3\}}$ is true when both $H_1$ and $H_3$ are true, regardless of whether $H_2$ is true or false. To claim that exactly hypotheses 1 and 3 are true, specify the full true-null set as $T(\theta)=\{1,3\}$.

Key takeaway: For a nonempty true-null set $T$, the intersection $H_T$ expresses the simultaneous truth of all elementary nulls indexed by $T$. At a parameter value with true-null set $T$, the nulls outside $T$ are false by the definition of $T$. Strong familywise error control must account for every such possible configuration.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For $H_1:\delta_1\leq0$ and $H_2:\delta_2\leq0$, write the intersection null and its alternative.
With three hypotheses, a parameter value belongs to $H_1$ and $H_3$, but not $H_2$. What is its true-null set, and does it belong to $H_{\{1,3\}}$?
Can a parameter value belong to $H_{\{1,3\}}$ while all three elementary nulls are true? Explain.
In one sentence, explain why checking only the global intersection does not by itself establish strong familywise error control.
Under three independent true-null tests at level $0.02$, what is the probability of at least one false rejection? State the assumptions used.

Solutions. (1) The intersection is $H_{\{1,2\}}:\delta_1\leq0$ and $\delta_2\leq0$. Its alternative is $\delta_1>0$ or $\delta_2>0$, because at least one selected null must fail. (2) The true-null set is $T=\{1,3\}$. The parameter value belongs to $H_{\{1,3\}}$ because both indexed nulls hold; the status of $H_2$ does not affect that membership. (3) Yes. The intersection requires $H_1$ and $H_3$ to be true, and it places no restriction on $H_2$; all three can therefore be true. (4) Strong control must hold at parameter values where only a subset of the elementary nulls is true, not just where all nulls are true. (5) Under independence, with each p-value uniform under its true null and each test rejecting at 0.02, the probability is $1-(1-0.02)^3=1-0.98^3=1-0.941192=0.05881$, or about $5.881\%$.