Why Consonance Matters
A sequential closed-testing procedure can reject a hypothesis about several arms without rejecting the hypothesis for any one arm. That outcome is mathematically possible, but it can be difficult to communicate: the trial has found evidence against a joint null, yet it has not established which individual arm hypothesis to reject under the procedure. Consonance is the property that rules out this gap. It matters whenever an intersection-level decision is intended to support a claim about at least one constituent arm.
The key distinction is between a local rejection and a closed rejection. A local test is the test assigned to one intersection hypothesis. Closed testing rejects an intersection only when its local test and the local tests for all larger intersections reject. Consonance concerns the decisions after this closure has been applied; a local rejection by itself does not generally establish that any arm hypothesis is rejected.
This definition is deliberately about closed rejection. If the local test for \(H_{\{1,2,3\}}\) crosses its boundary, that is not by itself a rejection of the intersection under closed testing: the local tests for every superset must also reject. For the full-family intersection, there are no larger sets, so its local rejection is its closed rejection. Even then, consonance asks whether at least one of its individual arm hypotheses is closed-rejected.
Sequential Validity and the Closure Logic
The local tests must be designed for the sequential setting. For each nonempty \(J\), the test of \(H_J\) must control its type I error over the entire prespecified monitoring process: under any parameter value satisfying \(H_J\), the probability that this local test rejects at any permitted look is at most the chosen level \(\alpha\). This is an “ever rejects” probability, not a separate guarantee at each look. The closure principle then gives strong familywise error rate control, whether or not the procedure is consonant.
To see why, suppose at least one true arm hypothesis is rejected by closure. Let \(T\) be the set of all true arm hypotheses at the parameter value in question. A rejected true arm \(H_i\), with \(i\in T\), requires the local test of every intersection containing \(i\) to reject. In particular, it requires rejection of \(H_T\). Since every hypothesis in \(T\) is true, \(H_T\) is true, so this event is contained in the local test’s type I error event. Thus the probability of rejecting any true arm is at most \(\alpha\). Consonance adds an interpretive guarantee; it is not needed for this error-control argument.
This gives a direct way to check the property from a table of local rejection flags. First find the closed-rejected intersections by checking all their supersets. Then, for each one, check whether at least one constituent arm has all of its own supersets locally rejected. To claim consonance throughout a sequential trial, repeat this check at each reported stage. A procedure can satisfy the condition at the final look but fail it at an interim look.
Worked Example: An Intersection Is Closed-Rejected but No Arm Is
Consider three arm hypotheses and a hypothetical set of local sequential test outcomes at the final analysis. A 1 means the local test rejected by that analysis; a 0 means it did not.
| Local hypothesis | Local rejection flag |
|---|---|
| \(H_{\{1\}}\) | 1 |
| \(H_{\{2\}}\) | 0 |
| \(H_{\{3\}}\) | 0 |
| \(H_{\{1,2\}}\) | 1 |
| \(H_{\{1,3\}}\) | 0 |
| \(H_{\{2,3\}}\) | 0 |
| \(H_{\{1,2,3\}}\) | 1 |
For the full intersection, closure gives \(C_{\{1,2,3\}}=L_{\{1,2,3\}}=1\), so it is closed-rejected. Arm 1 is not closed-rejected because \(C_{\{1\}}=1\times1\times0\times1=0\), where the factors are the local flags for \(\{1\},\{1,2\},\{1,3\},\{1,2,3\}\). Arm 2 is not closed-rejected because \(C_{\{2\}}=0\times1\times0\times1=0\); arm 3 is not closed-rejected because \(C_{\{3\}}=0\times0\times0\times1=0\).
The procedure is nonconsonant for this rejection pattern: a closed-rejected intersection has no closed-rejected constituent arm. The local flags alone do not tell us whether a particular statistical design can produce this pattern; that depends on the local tests. They do show exactly what a consonance check must rule out.
Conditions That Ensure Consonance
The exact condition can be stated in terms of local rejection flags: whenever all local tests for supersets of \(I\) reject, at least one constituent \(i\in I\) must have all local tests for supersets of \(\{i\}\) reject. This condition is often verified by examining the structure of the local tests, rather than by hoping that a particular data set avoids a problematic pattern.
One sufficient construction uses arm-level sequential tests with valid, prespecified whole-trial rejection decisions. Write \(A_i(t)=1\) if arm \(i\)'s sequential test has rejected by stage \(t\). Define the local test of \(H_J\) to reject exactly when every arm-level test in \(J\) has rejected: $$ L_J(t)=\prod_{j\in J}A_j(t). $$ This is a valid local test at level \(\alpha\) when each arm-level test has probability at most \(\alpha\) of ever rejecting under its own null. Under \(H_J\), all component nulls are true, and the event that every test in \(J\) rejects is contained in the event that any one specified true arm test rejects. Its probability is therefore at most \(\alpha\).
Under this construction, if \(H_I\) is closed-rejected, then in particular the local test for the full-family intersection rejects, because that intersection is a superset of \(I\). Its local rejection means \(\prod_{j=1}^m A_j(t)=1\), so every arm-level test has rejected. Consequently every local test rejects, and each arm hypothesis is closed-rejected. This establishes consonance. The construction is sufficient, not necessary, and may be conservative: it requires all arm-level tests to reject before any arm is rejected by closure.
Worked Example: Checking a Sufficient Construction
Suppose three arm-level sequential tests use boundaries calibrated in advance to control each test’s probability of ever rejecting its own null at \(\alpha=0.025\). At a particular look, their hypothetical standardized statistics are \(2.71\), \(2.34\), and \(2.63\), and their prespecified sequential boundaries at that look are \(2.50\), \(2.50\), and \(2.50\), respectively. Assume the tests reject when their statistic reaches or exceeds the applicable boundary.
The arm-level decisions are \(A_1=1\) because \(2.71\geq2.50\), \(A_2=0\) because \(2.34<2.50\), and \(A_3=1\) because \(2.63\geq2.50\). The local test of \(H_{\{1,3\}}\) rejects because \(L_{\{1,3\}}=A_1A_3=1\times1=1\). In contrast, \(L_{\{1,2\}}=1\times0=0\), \(L_{\{2,3\}}=0\times1=0\), and \(L_{\{1,2,3\}}=1\times0\times1=0\).
Despite the local rejection of \(H_{\{1,3\}}\), it is not closed-rejected: its superset \(H_{\{1,2,3\}}\) did not reject. No arm is closed-rejected either, because each arm’s closure includes the nonrejected full-family intersection. If arm 2 crosses its calibrated boundary at a later look, then all three \(A_i\) equal 1. Every local intersection test rejects, every arm is closed-rejected, and the construction’s consonance condition holds. The boundaries in this example are assumed to be valid sequential boundaries; applying an ordinary fixed-sample cutoff repeatedly would not establish the stated whole-trial error guarantee.
Other local-test systems can also be consonant, but their rejection structure must be checked rather than inferred from the word “closed.” A useful sufficient condition is that, whenever a closed intersection is rejected, the local rejection pattern supplies a constituent arm whose every containing intersection is locally rejected. Some designs impose this through a common ordered decision rule; others may require direct verification over possible rejection patterns. The exact condition above remains the definition to use when a simpler sufficient structure is unavailable.
Diagnostics, Comparisons, and Limitations
A practical diagnostic is to record, at each interim and final analysis, the local rejection flags for every intersection and calculate the closed flags from the product condition. Search for a row \(I\) with \(C_I(t)=1\) but \(C_{\{i\}}(t)=0\) for every \(i\in I\). Such a row is a concrete witness of nonconsonance. This audit checks the logical property of the decision system; it does not replace verification that each local sequential test is valid or that the closure controls familywise error.
Sequential timing adds an important qualification. If closure is evaluated at stage \(t\), every local test needed for \(C_I(t)=1\) must have rejected by that stage. A decision rule that reports an intersection as rejected while some required local tests have not yet rejected is not applying closed testing as defined here. Also, a terminal consonance check does not justify an interim statement that an arm has been rejected: interim decisions need the same closure and consonance checks at the interim stage.
| Decision property | What it establishes | What it does not establish by itself |
|---|---|---|
| Local rejection of \(H_I\) | The local sequential test for \(H_I\) has rejected. | Closed rejection of \(H_I\), or rejection of any arm hypothesis. |
| Closed rejection of \(H_I\) | Every local test for a superset of \(I\) has rejected. | Rejection of a constituent arm unless the procedure is consonant for this decision. |
| Consonant closed rejection | At least one constituent arm hypothesis is also closed-rejected. | Identification of a unique arm, or proof that the effect is clinically important. |
The main alternative to a consonant closure is not an invalid procedure. A nonconsonant closed procedure can still control the familywise error rate, and rejection of an intersection can remain a valid statement about that joint null. What it cannot do is promise that at least one arm will be rejected by closure whenever an intersection is rejected. Conversely, a consonant procedure provides that interpretive link but may impose stronger or more restrictive local decision rules. Consonance does not itself control error rates, guarantee useful power, or show which rejected arm is best.
Check Your Understanding
Use the definitions above. For each question, give a reason for your answer.
- With three arms, the local tests for \(H_{\{1,2,3\}}\) and \(H_{\{1,2\}}\) reject, but the test for \(H_{\{1,3\}}\) does not. Is \(H_{\{1,2,3\}}\) closed-rejected? Does this information establish that \(H_1\) is closed-rejected?
- State the event that must hold for \(H_{\{2\}}\) to be closed-rejected in a three-arm family.
- Under the construction \(L_J(t)=\prod_{j\in J}A_j(t)\), the arm-level decisions are \(A_1=1,A_2=1,A_3=0\). Which local intersections reject, and are any arm hypotheses closed-rejected?
- Why must a sequential local test’s error guarantee apply to the probability of rejection at any permitted look, rather than only to its error probability at one look?
- Can a nonconsonant closed procedure still control the familywise error rate? Explain briefly.
Solutions. (1) The full intersection is closed-rejected because it has no strict supersets and its own local test rejects. This does not establish rejection of \(H_1\): \(H_{\{1\}}\) is closed-rejected only if its singleton, pairwise, and full-family local tests all reject, and the \(H_{\{1,3\}}\) test did not. (2) The local tests for \(H_{\{2\}},H_{\{1,2\}},H_{\{2,3\}}\), and \(H_{\{1,2,3\}}\) must all reject. (3) The local tests for \(\{1\},\{2\},\{1,2\}\) reject; those involving arm 3 do not. No arm is closed-rejected because each arm’s closure includes the nonrejected full-family test. (4) Repeated monitoring creates multiple opportunities to cross a boundary. A guarantee at only one look does not bound the probability of crossing at any look, which is the relevant sequential type I error. (5) Yes. If a true arm is closed-rejected, closure requires rejection of the intersection of all true arm hypotheses; validity of that local test bounds the probability of any such false arm rejection by \(\alpha\). Consonance is an additional decision-interpretation property, not the basis of this error-control argument.