Why Consonance Matters
A closed test can control the familywise error rate while still producing a decision that may seem surprising: it can reject an intersection null without rejecting any elementary null contained in that intersection. Consonance rules out this pattern. When the property holds, evidence against a combined null always leads to at least one elementary treatment-comparison rejection. That link makes it possible to organize decisions as a stepwise process.
Consider a family of \(m\) elementary null hypotheses, \(H_1,\ldots,H_m\), such as null hypotheses for treatment-arm comparisons with a common control. For every nonempty index set \(I\subseteq\{1,\ldots,m\}\), the corresponding intersection null is \(H_I=\bigcap_{i\in I}H_i\). A closed testing procedure specifies a level-\(\alpha\) local test for every such intersection and rejects an elementary null only when every intersection containing it is rejected. This closure principle is established in the earlier material on closed testing; here the focus is the additional property of consonance.
The definition concerns the closed-test decisions for elementary hypotheses, not simply whether one of the local tests of a singleton is significant. To reject \(H_i\), every local test for an intersection that contains \(i\) must reject. Thus, a small local p-value for \(H_i\) by itself is not sufficient.
Closed-Test Adjusted P-Values and a Consonance Check
Let \(p_I\) denote the p-value from the local test of \(H_I\). The closed-test adjusted p-value for elementary null \(H_i\) is the largest local p-value among the intersections that contain it:
This maximum is required because every such intersection must be rejected. At level \(\alpha\), the closed test rejects \(H_i\) exactly when \(p_i^{\mathrm{cl}}\leq\alpha\). Consequently, consonance has a direct p-value check: for each intersection \(H_I\), a rejection of that intersection must imply that at least one \(i\in I\) has \(p_i^{\mathrm{cl}}\leq\alpha\).
The all-level version follows by considering the level \(\alpha=p_I\). If \(H_I\) is rejected at that level, consonance requires an elementary rejection inside \(I\), so the smallest closed adjusted p-value in \(I\) cannot exceed \(p_I\). Conversely, if this inequality holds for every \(I\), any rejection at level \(\alpha\) has \(p_I\leq\alpha\), and therefore at least one elementary hypothesis in \(I\) has closed adjusted p-value no greater than \(\alpha\).
Worked Example: Checking Consonance for Three Comparisons
Suppose a hypothetical MAMS trial has three elementary null hypotheses, \(H_1,H_2,H_3\), and the local tests produce the following p-values. Use \(\alpha=0.05\).
| Intersection | Local p-value |
|---|---|
| \(H_1\) | 0.010 |
| \(H_2\) | 0.025 |
| \(H_3\) | 0.200 |
| \(H_{12}\) | 0.040 |
| \(H_{13}\) | 0.030 |
| \(H_{23}\) | 0.045 |
| \(H_{123}\) | 0.020 |
For \(H_1\), the intersections containing it are \(H_1,H_{12},H_{13},H_{123}\). Therefore,
Similarly, \(p_2^{\mathrm{cl}}=\max(0.025,0.040,0.045,0.020)=0.045\), and \(p_3^{\mathrm{cl}}=\max(0.200,0.030,0.045,0.020)=0.200\). The closed test therefore rejects \(H_1\) and \(H_2\), but not \(H_3\), at 0.05.
Every rejected intersection contains at least one of those rejected elementary hypotheses: \(H_{13}\) contains \(H_1\), \(H_{23}\) contains \(H_2\), \(H_{12}\) contains both, and \(H_{123}\) contains both. The procedure is consonant at 0.05 for these decisions.
How Consonance Supports a Stepwise Algorithm
Closure can require checking many intersections: a family of \(m\) elementary hypotheses has \(2^m-1\) nonempty intersections. Consonance supplies a useful logical ordering. Begin with the intersection of all hypotheses. If it is not rejected, no elementary hypothesis can be rejected, because each elementary hypothesis requires rejection of this global intersection. If it is rejected, consonance guarantees that at least one elementary hypothesis can be rejected under the closed rule.
After establishing an elementary rejection, remove that hypothesis from the set still under consideration and assess the intersection of the remaining hypotheses. Intersections that include an already rejected elementary hypothesis have already passed the necessary closed-test requirement for the other hypotheses. The remaining question is whether the intersections among the unrejected hypotheses support another rejection. Continue until the current intersection is not rejected or no hypotheses remain.
Start with \(H_{\{1,\ldots,m\}}\). If its local test does not reject at \(\alpha\), stop: no elementary hypothesis can pass closure.
If the current intersection is rejected, consonance guarantees at least one elementary null in the current set is rejected by the induced closed test. Determine it using the required local intersection tests.
Continue with the hypotheses not yet rejected. Stop when the intersection of the remaining set fails its local test; no hypothesis in that set can then be rejected.
This is a stepwise representation of the closed-test decisions, not a license to choose any treatment arm after seeing the data and declare it significant. At each step, the elementary rejection must still satisfy its closure conditions. Consonance guarantees that the stepwise search can proceed when the current intersection is rejected; it does not replace the local tests or their prespecified rules.
Worked Example: Running the Stepwise Decisions
Use the p-values in the preceding example. The all-hypotheses intersection \(H_{123}\) has local p-value 0.020, below 0.05, so the first step does not stop. The closed adjusted p-values show that \(H_1\) and \(H_2\) are rejected.
For the remaining set \(\{2,3\}\), the local test of \(H_{23}\) has p-value 0.045, so the algorithm continues. Within this set, the closed adjusted p-value for \(H_2\), considering its relevant remaining intersections, is \(\max(0.025,0.045)=0.045\); for \(H_3\) it is \(\max(0.200,0.045)=0.200\). Thus \(H_2\) is rejected and \(H_3\) is not. This agrees with the original full closure.
After removing \(H_2\), the remaining singleton \(H_3\) has local p-value 0.200, which exceeds 0.05. The procedure stops with \(H_1\) and \(H_2\) rejected and \(H_3\) unrejected. The numerical sequence shows the logic of the steps; a valid implementation must retain all necessary local tests, not treat the displayed sequence as a new unadjusted testing rule.
When Consonance Fails
A closed procedure can be valid without being consonant. For example, suppose the local p-values for three hypotheses are \(p_{123}=0.010\), \(p_{12}=0.080\), \(p_{13}=0.080\), \(p_{23}=0.080\), and \(p_1=p_2=p_3=0.100\). At \(\alpha=0.05\), the global intersection is rejected, but each elementary closed adjusted p-value is 0.100. No elementary hypothesis is rejected. This fails the definition of consonance: a rejected intersection has not led to a rejection inside it.
Such a result does not by itself show that the closed procedure lacks familywise error control. Consonance is a property of how its rejections relate to one another, not the condition that establishes the validity of its local tests or the closure principle. A procedure may control familywise error and still be nonconsonant.
Worked Example: Diagnosing a Nonconsonant Decision
In a hypothetical trial with three treatment comparisons, suppose the local p-value for the global intersection is 0.010, and each pairwise intersection and each singleton has local p-value 0.100. At level 0.05, \(H_{123}\) is rejected because \(0.010\leq0.05\).
For each elementary hypothesis, the closed adjusted p-value is the maximum over its singleton, pair, and global tests. For example, \(p_1^{\mathrm{cl}}=\max(0.100,0.100,0.100,0.010)=0.100\); the same calculation gives 0.100 for \(H_2\) and \(H_3\). None is rejected. The global intersection rejection therefore has no elementary rejection within it, so the procedure is nonconsonant at this level.
The correct conclusion is not that all three treatment comparisons are significant. The only rejected null in this calculation is the global intersection null. Consonance would be needed to translate that rejection into at least one elementary treatment-comparison rejection.
Assumptions, Diagnostics, and Alternatives
The consonance argument assumes a well-defined family of elementary nulls and a local test for every nonempty intersection. Each local test must be valid at its assigned level under the corresponding intersection null; its p-value must satisfy \(P(p_I\leq u)\leq u\) under that null for every \(0\leq u\leq1\). In a multi-stage trial, the local tests must also follow the specified stagewise analysis and stopping rules. Consonance cannot repair invalid local tests, unplanned analyses, or a change in the multiplicity family after results are observed.
To diagnose consonance at a particular \(\alpha\), enumerate the intersections whose local tests reject, calculate the closed decisions for their component elementary hypotheses, and verify that each rejected intersection contains at least one closed-test rejection. For an all-level claim, check \(\min_{i\in I}p_i^{\mathrm{cl}}\leq p_I\) for every \(I\). A failure points to a nonconsonant closed procedure, not automatically to an error in the p-value calculations.
Direct closure is the general alternative: evaluate every relevant intersection and apply the definition, whether or not the procedure is consonant. Stepwise methods such as Holm can be computationally convenient for particular local-test structures, but consonance alone does not make an arbitrary closed test identical to Holm or justify using Holm’s thresholds. Conversely, a nonconsonant closed test remains interpretable by its closure rule, though it does not have the same stepwise implication.
Common Mistake
- Wrong reasoning: “The global intersection was rejected, so at least one treatment comparison must be significant.” A global rejection alone does not imply this for every closed procedure.
- Correction: That implication requires consonance. Compute the closed decisions for the elementary hypotheses; only if at least one is rejected may an elementary treatment comparison be reported as rejected.
- Another error: Checking only the singleton local p-values. The closure condition uses every intersection containing the elementary hypothesis, so its closed adjusted p-value is the maximum over those local p-values.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- At level 0.05, \(H_{12}\) is rejected, while neither \(H_1\) nor \(H_2\) is rejected by closure. What does this say about consonance at that level?
- If \(p_{123}=0.03\), what must be true for the procedure to be consonant at level 0.05?
- For three hypotheses, \(p_1=0.02\), \(p_2=0.08\), \(p_3=0.10\), \(p_{12}=0.04\), \(p_{13}=0.06\), \(p_{23}=0.09\), and \(p_{123}=0.05\). Calculate \(p_1^{\mathrm{cl}}\) and \(p_2^{\mathrm{cl}}\), and state which of \(H_1,H_2\) is rejected at 0.05.
- Does consonance by itself establish familywise error control? Explain briefly.
Solutions. (1) It is nonconsonant at 0.05: the rejected intersection contains no closed-test elementary rejection. (2) At least one of \(H_1,H_2,H_3\) must have a closed adjusted p-value no greater than 0.05. (3) \(p_1^{\mathrm{cl}}=\max(0.02,0.04,0.06,0.05)=0.06\), so \(H_1\) is not rejected. \(p_2^{\mathrm{cl}}=\max(0.08,0.04,0.09,0.05)=0.09\), so \(H_2\) is not rejected either. Since \(H_{123}\) is rejected at 0.05 but no elementary null is rejected, this set of decisions is nonconsonant. (4) No. Consonance is a relationship among rejection decisions; familywise error control depends on the validity of the local tests and the closed testing procedure.