Disjunctive Power Under Independent Tests

From Pairwise Power to Any-Success Power

In a multi-arm trial, a treatment may be judged successful if at least one of several prespecified comparisons succeeds. The probability of that outcome is not generally the power of any single comparison: it depends on how the comparison outcomes occur together. Under independence, however, the marginal pairwise powers are enough to calculate the probability of at least one success.

The calculation is straightforward once the relevant events are specified. Its main condition is easy to overlook: it is the success events that must be independent under the treatment-effect configuration of interest. Having several tests, or calculating their powers separately, does not establish that condition.

Definition: Consider a prespecified family of $m$ comparisons under a fixed treatment-effect configuration. Let $S_i$ be the event that comparison $i$ meets its specified success criterion, and let $\pi_i=P(S_i)$ be its pairwise power under that configuration. Disjunctive power is the probability of at least one success: $$ P\left(\bigcup_{i=1}^{m}S_i\right). $$ If the events $S_1,\ldots,S_m$ are mutually independent, then disjunctive power is $$ 1-\prod_{i=1}^{m}(1-\pi_i). $$

A success criterion might be rejection of the comparison’s null hypothesis using a prespecified test and decision rule. The $\pi_i$ values must therefore correspond to those rules, including any multiplicity adjustment that determines success. They are probabilities under the stated treatment-effect configuration, not probabilities that the null hypotheses are true.

For example, $\pi_i=0.70$ means that, if the assumed effects and other model conditions hold, comparison $i$ would meet its success criterion in 70% of repeated trials. Disjunctive power is instead the probability that at least one comparison meets its criterion in a repeated trial.

Why the Product Formula Holds

Let $F_i=S_i^c$ denote failure of comparison $i$ to meet its success criterion. No comparison succeeds exactly when every comparison fails. Thus the event of no success is the intersection of all the failure events. The complement rule then turns the probability of that intersection into the probability of at least one success.

\begin{aligned} P\left(\bigcup_{i=1}^{m}S_i\right) &=1-P\left(\bigcap_{i=1}^{m}F_i\right)\\ &=1-\prod_{i=1}^{m}P(F_i)\\ &=1-\prod_{i=1}^{m}(1-\pi_i). \end{aligned}

The second equality is where independence is used: if $F_1,\ldots,F_m$ are mutually independent, the probability that they all occur is the product of their probabilities. Independence of the success events implies independence of their complements, so it is enough to assume either set is mutually independent.

For two comparisons, the result is $1-(1-\pi_1)(1-\pi_2)$. For three, it is $1-(1-\pi_1)(1-\pi_2)(1-\pi_3)$. The formula scales directly to any finite number of comparisons. It does not require the pairwise powers to be equal.

Assumption to check: The product formula is exact when the final success or failure indicators are mutually independent under the configuration being evaluated. Independence of the underlying test statistics can support this condition when each decision depends only on its own statistic and a fixed threshold. It is not enough that the comparisons are described or tested separately.

Worked Calculations

Worked Example: Combining Three Unequal Pairwise Powers

Suppose a hypothetical trial has three prespecified treatment comparisons. Under a stated effect configuration, their pairwise powers are $0.70$, $0.60$, and $0.50$. Assume the three final comparison-success events are mutually independent. Find the disjunctive power.

The probabilities of failure are $1-0.70=0.30$, $1-0.60=0.40$, and $1-0.50=0.50$. Under the independence assumption, the probability all three fail is $$ P(F_1\cap F_2\cap F_3)=0.30\times0.40\times0.50=0.06. $$

Therefore, $$ P(S_1\cup S_2\cup S_3)=1-0.06=0.94. $$ The disjunctive power is 0.94, or 94%. In repeated trials under this effect configuration and these decision rules, approximately 94% would have at least one successful comparison, if the model and independence assumption hold.

Worked Example: Using Powers From Adjusted Decision Rules

In a second hypothetical three-comparison setting, pairwise powers for the prespecified final success rules are $0.82$, $0.65$, and $0.40$. These powers already reflect the specified decision rules, and suppose the resulting success events are mutually independent. Calculate disjunctive power.

The individual failure probabilities are $0.18$, $0.35$, and $0.60$. Hence $$ P(\text{no success})=0.18\times0.35\times0.60=0.0378. $$

Taking the complement gives $$ P(\text{at least one success})=1-0.0378=0.9622. $$ The disjunctive power is 0.9622, or about 96.2%. This calculation uses the powers for the stated final rules; substituting powers from unadjusted tests would answer a different question if the actual success rules use adjustment.

Worked Example: Finding the Pairwise Power Needed for a Target

A hypothetical design has three independent comparison-success events and aims for disjunctive power of at least 0.90. Suppose the design gives all three comparisons the same pairwise power $\pi$. What pairwise power reaches a disjunctive power of exactly 0.90?

Under the stated equality and independence assumptions, $$ 0.90=1-(1-\pi)^3. $$ Rearranging gives $(1-\pi)^3=0.10$, so $$ \pi=1-\sqrt[3]{0.10}\approx1-0.4642=0.5358. $$

Thus a pairwise power of approximately 0.536 for each comparison gives disjunctive power of 0.90 under this model. This is an algebraic target, not a guarantee that a feasible sample size or a particular trial’s actual decision rules will deliver it. A design calculation must still produce the stated pairwise powers under the intended model.

What Independence Means in a Multi-Arm Trial

The independence condition concerns outcomes of the final comparisons, viewed as random events over repeated trials. Formally, mutual independence means that for every subset $A$ of the comparison indices, $$ P\left(\bigcap_{i\in A}S_i\right)=\prod_{i\in A}P(S_i). $$ For the disjunctive formula, it is particularly the probability that all comparisons fail that is needed. Mutual independence of the success events guarantees the required product for failures.

Independence may be plausible when comparisons use independent data and each decision is based only on its own data through a fixed rule. But a multi-arm trial often compares several treatments with the same control group. The resulting comparison statistics can share information through the control observations, so the comparison outcomes need not be independent. Likewise, a joint multiple-testing procedure can make final rejection decisions depend on the set or ordering of several test results. Separate reported pairwise powers do not remove either source of dependence.

The powers must also refer to the same fixed configuration and trial design. If the assumed effects, sample sizes, variance model, or success rules differ between power calculations, the numbers cannot simply be treated as the marginal probabilities for one common probability calculation. In particular, power is configuration-specific: a comparison’s probability of success can change when the assumed effects for the arms change.

Diagnostics, Alternatives, and Limitations

Before multiplying failure probabilities, check what the events actually represent and how the analysis produces them. Ask whether the comparisons share participants or control observations, whether the final decisions use a joint adjustment, and whether any selection or stopping rule links the decisions. These features do not prove dependence in every possible design, but they are reasons to justify independence rather than assume it.

A practical check is to write down the success event for each comparison and identify the data and decision rule that determine it. If each event depends only on a separate independent data source and a fixed rule, independence may follow from the model. If an event depends on shared data or on other comparisons’ results, independence requires additional justification. Simulation under the full design can also assess joint success probabilities when an analytic joint distribution is available or difficult to work with.

When independence is justified, the product formula is exact and efficient: only the pairwise powers are needed. When it is not justified, the appropriate alternative is to calculate the joint probability of no success, or equivalently the probability of the union, from the joint distribution of the comparison outcomes. A simulation can estimate that probability by repeatedly generating complete trial datasets and applying the full decision procedure. The product of marginal failure probabilities is not a substitute for that joint calculation.

A useful check on any proposed disjunctive power is that it cannot be smaller than the largest pairwise power: the event of success on any one comparison is contained in the event of at least one success. It also cannot exceed 1. Under independence, the calculated value should satisfy these basic bounds. If the result violates either, there is an arithmetic or input error.

The principal limitation is that marginal powers alone do not generally determine the probability of at least one success. The product expression supplies that probability only with the independence condition. In shared-control designs or procedures with linked decisions, using it without justification may overstate or understate disjunctive power; the direction cannot be inferred from the pairwise powers alone.

Common Pitfall

Incorrect reasoning: “There are three separate tests, so the failures are independent; I can multiply their failure probabilities.” Separate tests can still use shared observations or be linked by the final decision procedure. The number of tests and the fact that each has its own pairwise power do not establish independence.

Correction: First define the final success events and assess their joint behavior under the stated configuration. Use $1-\prod_i(1-\pi_i)$ only when the events are mutually independent. Otherwise, use a joint probability calculation or a simulation that applies the complete trial design and decision rule.

Key takeaway: Under mutually independent comparison-success events, disjunctive power is one minus the product of the pairwise failure probabilities. The formula is simple; verifying independence and using powers for the actual success rules are the essential checks.

Check Your Understanding

Try each question, then compare your reasoning with the solutions.

Three independent comparison-success events have pairwise powers $0.75$, $0.50$, and $0.40$. What is the disjunctive power?
Two comparisons each have pairwise power $0.60$. Does that information alone determine disjunctive power? Explain.
A trial’s final decisions use a procedure in which each comparison’s rejection depends on the ordered set of test results. Can separate pairwise powers automatically be combined by multiplying failure probabilities? Why or why not?
For four independent comparisons with equal pairwise power $\pi$, write the equation for disjunctive power $0.95$. Give the resulting value of $\pi$ to three significant figures.

Solutions. (1) The failure probabilities are $0.25$, $0.50$, and $0.60$. Thus $P(\text{no success})=0.25\times0.50\times0.60=0.075$, and disjunctive power is $1-0.075=0.925$, or 92.5%. (2) No. The marginal powers do not specify how the success events occur together. If independence is additionally assumed, the disjunctive power is $1-(0.40)^2=0.84$; without that assumption or other joint information, it is not determined. (3) No. A decision that depends on the ordered set of results can link the final success events. The independence condition must be established for the final events; otherwise a joint calculation is needed. (4) The equation is $0.95=1-(1-\pi)^4$. Therefore $\pi=1-(0.05)^{1/4}\approx1-0.4729=0.527$. Each comparison needs pairwise power of about 0.527 under the stated independence and equal-power assumptions to yield 0.95 disjunctive power.