What Counts as Disjunctive Success?
In a multi-arm trial, a trial-level success criterion must say how the results across treatment arms combine. A disjunctive criterion asks whether at least one treatment comparison in a prespecified set meets its success rule. This is different from asking whether every comparison succeeds: one successful target comparison is sufficient.
The definition starts with the arm-specific decisions, not with a calculation of their joint probability. Let there be \(K\) treatment arms, indexed by \(j=1,\ldots,K\), and let \(S\) be the prespecified set of arms included in the trial-level claim. For each \(j\), let \(R_j\) denote the event that the procedure declares the comparison for arm \(j\) successful—for example, by rejecting its corresponding null hypothesis under the prespecified testing procedure.
For example, if success for arm \(j\) means rejecting its null hypothesis of no treatment benefit versus control, then \(R_j\) is that rejection event. The rejection rule might use multiplicity adjustment, as required by the prespecified testing plan. The event \(D(S)\) asks whether at least one such adjusted arm-specific rejection occurs. It does not require all arms in \(S\) to be significant.
The probability is evaluated under a specified configuration of treatment effects. To call it power in the usual sense, the configuration should include at least one treatment effect for which a rejection would be a correct detection. The chance of success may differ across configurations, even when the same arms and testing procedure are used.
Define the Event Before Evaluating Its Probability
A useful way to prevent ambiguity is to list the target arms and write down which arm-specific results count. If \(S=\{1,2,3\}\), for instance, then success is \(R_1\) or \(R_2\) or \(R_3\). Results in arms outside \(S\) do not satisfy this particular criterion, even if they are statistically significant. This is why the target set is part of the definition rather than a detail to decide after seeing the results.
There are two related ideas to keep separate when some target arms truly have no effect. The operational criterion “at least one target comparison is rejected” is the event \(\bigcup_{j\in S}R_j\). Under a mixed-effect configuration, this event can occur because a null arm is rejected by chance. That outcome meets the literal any-rejection criterion, but it is not evidence that a truly effective treatment was detected. If the scientific question specifically asks whether at least one truly effective arm was detected, define \(E(\theta)\subseteq S\) as the arms with effects meeting the prespecified target and use the event \(\bigcup_{j\in E(\theta)}R_j\). State which criterion is intended; they need not have the same probability.
A short indicator argument shows why the union is the right representation. Let \(I(R_j)\) equal 1 if \(R_j\) occurs and 0 otherwise. The indicator of any success is 1 exactly when at least one of the arm-specific indicators is 1. Therefore it is the maximum of those indicators, and its expectation is the probability of the event:
This argument establishes the event and its probability without imposing independence among comparisons. It does not provide a numerical value from the individual arm powers alone; evaluating the probability requires information about the joint behavior of the rejection events. That calculation is a separate question from defining what success means.
Worked Example: Two Treatments, Either One Can Succeed
Consider a hypothetical trial with two treatment arms, A and B, each compared with a shared control. The protocol defines success as a statistically significant benefit for at least one of the two arms, using the prespecified multiplicity procedure. Suppose the reported adjusted p-values are 0.018 for A and 0.041 for B, and the trial-wide decision rule rejects an arm-specific null when its adjusted p-value is at most 0.025.
Step 1: Identify the target set and rule. Both treatments are in the target set, so \(S=\{A,B\}\). Let \(R_A\) and \(R_B\) denote rejection under the specified adjusted procedure. The decision cutoff is 0.025.
Step 2: Apply the arm-specific rule. For A, \(0.018\leq 0.025\), so \(R_A\) occurs. For B, \(0.041>0.025\), so \(R_B\) does not occur.
Step 3: Evaluate the success event for these results. Here \(D(S)=R_A\cup R_B\). Since at least one event occurs—specifically, \(R_A\)—the trial meets the stated disjunctive criterion. It does not meet a criterion requiring both treatments to succeed.
This example classifies a set of trial results; it does not calculate power. Power is the probability of this same event over repeated trials under a specified treatment-effect configuration.
Arm Configurations and the Meaning of Success
The event is defined by the prespecified target set and decision rules. The treatment-effect configuration matters when interpreting what a successful event represents. In a configuration where every target treatment has the effect of interest, any rejection within \(S\) is a rejection for an arm that truly has that effect. In a mixed configuration, the event of any target rejection can include false rejections as well as correct detections.
Worked Example: Three Arms With One Effective Treatment
In a hypothetical three-arm trial, treatments A, B, and C are all in the prespecified target set \(S=\{A,B,C\}\). Suppose the specified effect configuration says A and C have no benefit over control, while B has a benefit meeting the target. In one hypothetical trial, the multiplicity-adjusted decisions are: A rejected, B not rejected, and C not rejected.
Any-rejection criterion. The event is \(D(S)=R_A\cup R_B\cup R_C\). The observed pattern has one rejection, so the any-rejection criterion is met.
True-positive detection criterion. Under the stated configuration, \(E(\theta)=\{B\}\). The event for detecting at least one truly effective treatment is \(R_B\). Since B was not rejected, this criterion is not met.
The difference is not a contradiction: the two criteria answer different questions. The first is a literal rule about any rejection in the tested family. The second asks specifically whether a treatment that truly meets the effect target was detected. If the protocol describes “disjunctive power” as the chance of at least one true detection, the second event must be used and named explicitly.
The choice of target set can also be narrower than the set of all treatment arms. A protocol might designate only certain comparisons as primary for the trial-level claim. A significant result on a non-target arm does not satisfy a success event defined only over the primary set. This remains true even if that result is scientifically interesting or is reported as a secondary finding.
Worked Example: A Prespecified Subset of Four Treatments
Suppose a hypothetical study has four treatment arms, A, B, C, and D. Before the trial, the protocol names A and C as the target set for a particular claim: \(S=\{A,C\}\). The criterion is at least one adjusted rejection in this set. Imagine the adjusted decisions are A not rejected, B rejected, C not rejected, and D rejected.
Step 1: Restrict attention to the stated set. The success event is \(D(S)=R_A\cup R_C\), not \(R_A\cup R_B\cup R_C\cup R_D\).
Step 2: Check each included arm. Neither A nor C is rejected, so both \(R_A\) and \(R_C\) are false for this result.
Step 3: Decide whether the criterion is met. Since zero of the two target arms is rejected, the stated disjunctive criterion is not met. The rejections for B and D do not change that conclusion because those arms are outside \(S\).
If the protocol had instead defined all four arms as the target set, the same results would satisfy the any-rejection rule. The conclusion changes because the success event changes—not because the observed decisions changed.
Checks Before Reporting Disjunctive Power
A disjunctive power statement is interpretable only if readers can tell which trial outcome counts as success. Before reporting it, verify that the target set and the arm-level decision rule were fixed in advance, and state the effect configuration under which the probability is evaluated. Also say whether “success” means any rejection in the target family or at least one correct detection among effective arms.
The earlier material on pairwise power describes power for a single comparison. In a multi-arm trial, listing those marginal powers is not itself a definition of disjunctive power: the trial-level event is a union of rejection events, and its probability concerns those events jointly. Do not infer a trial-level probability merely by adding pairwise powers. The exact joint calculation depends on the dependence structure and testing procedure.
| Question to specify | Why it matters |
|---|---|
| Which arms are in \(S\)? | Only events for those arms enter the stated success criterion. |
| What does \(R_j\) mean? | The rejection rule determines which outcomes count as arm-level success. |
| Under which effect configuration is power evaluated? | Power can differ when different arms have effects meeting the target. |
| Does success mean any rejection or a true detection? | In a mixed configuration, a null arm can be rejected by chance. |
| Was the criterion prespecified? | Changing the target set after seeing results changes the question being answered. |
A common failure is to define success after observing which arms look favorable. For example, a report may initially describe a three-arm family, then calculate “disjunctive power” only over the two arms with the largest observed effects. This does not estimate power for the prespecified three-arm criterion. It selects a different event using the results. The correction is to define \(S\) and the arm-specific rules before examining trial outcomes, and to report results for that definition.
Another mistake is to equate any statistically significant result with a true treatment effect. A rejection is a decision generated by a procedure, not proof that the corresponding effect is nonzero or clinically meaningful. The error control properties of the testing procedure address the frequency of false rejections under specified null configurations; they do not make a false rejection impossible in an individual trial.
Check Your Understanding
For each question, first identify the event that defines success. Solutions are provided below the questions.
- A protocol includes treatments A, B, and C in its target set and defines success as at least one adjusted rejection. Write the success event.
- In a four-arm trial, only arms A and C are prespecified for a particular claim. If B is rejected but A and C are not, is that claim successful?
- In a mixed configuration, B is the only treatment with an effect meeting the target. Write the event for detecting at least one truly effective treatment.
- Why is the sum of three pairwise powers not, by itself, the definition of disjunctive power for three arms?
- A team removes one arm from its target set after seeing that arm’s result was not significant. What is the problem with reporting the resulting power as the power for the original criterion?
Solutions. (1) \(D(S)=R_A\cup R_B\cup R_C\). (2) No. The claim’s set is \(S=\{A,C\}\), so its success event is \(R_A\cup R_C\); rejection of B does not meet it. (3) It is \(R_B\), because B is the only arm in \(E(\theta)\). (4) The definition is the probability of the union of rejection events; adding marginal probabilities does not specify their joint behavior and can count outcomes in overlapping events more than once. (5) The target event has been changed using observed results. The resulting quantity does not refer to the original prespecified criterion and should not be reported as its power.