Conjunctive Power Under Independent Tests

Why Independence Matters for Conjunctive Power

In a multi-arm trial, a study may be considered successful only if every treatment-control comparison meets its prespecified success criterion. The probability of that all-success event is the trial’s conjunctive power. If the comparison outcomes are independent, this joint probability has a simple form: multiply the individual pairwise powers. When arms have unequal powers, each factor must be retained separately.

The formula is simple, but its condition is substantive. Knowing the power of each comparison does not by itself determine the probability that all comparisons succeed. The product follows only when the success events are independent under the treatment-effect scenario being considered. This tutorial derives that result, shows how to use unequal powers, and explains what to check before applying it.

Definition: Let $S_i$ denote the event that comparison $i$ meets its prespecified success criterion, for $i=1,\ldots,m$. The conjunctive success event is $\bigcap_{i=1}^{m}S_i$, and its probability under a specified treatment-effect scenario is the conjunctive power. The pairwise power for comparison $i$ under that scenario is $p_i=P(S_i)$.

A success event must be defined before calculating power. For example, it might mean that a treatment-control test rejects its null hypothesis using the planned significance threshold. If the design uses another prespecified criterion, that criterion defines $S_i$ instead. The $p_i$ values must be probabilities of these actual planned events, not powers calculated for a different test or threshold.

Deriving the Product Formula

For two comparisons, the multiplication rule for probabilities gives $P(S_1\cap S_2)=P(S_1)P(S_2\mid S_1)$. Independence means that success on the second comparison has the same probability whether or not the first comparison succeeds: $P(S_2\mid S_1)=P(S_2)$, when $P(S_1)>0$. Hence the joint success probability is $p_1p_2$. Applying the same argument successively gives the result for any finite number of comparisons.

\begin{aligned} P\left(\bigcap_{i=1}^{m}S_i\right) &=P(S_1)\,P(S_2\mid S_1)\cdots P\left(S_m\mid \bigcap_{i=1}^{m-1}S_i\right)\\ &=p_1p_2\cdots p_m, \end{aligned}

The second equality requires mutual independence of the success events: each event is independent of the collection of the other events, so the conditional probabilities in the first line reduce to their corresponding marginal probabilities. Pairwise independence alone is not generally enough to justify the product when there are three or more events.

Theorem: If $S_1,\ldots,S_m$ are mutually independent under the treatment-effect scenario of interest, and $p_i=P(S_i)$, then conjunctive power is $$ P\left(\bigcap_{i=1}^{m}S_i\right)=\prod_{i=1}^{m}p_i. $$

This result accommodates unequal powers without modification. It does not say that a common power should be used for all arms. If, for instance, one comparison is less powerful because its effect is smaller or its standard error is larger, its smaller $p_i$ lowers the probability that every comparison succeeds.

Worked Calculations With Unequal Powers

Worked Example: Three Independent Comparisons With Different Powers

Suppose a hypothetical trial has three prespecified treatment-control comparisons. Under a particular scenario, their pairwise powers are $p_1=0.80$, $p_2=0.70$, and $p_3=0.90$. Assume the three comparison success events are mutually independent under this scenario. What is conjunctive power?

The all-success event is $S_1\cap S_2\cap S_3$. Independence permits multiplication:

P(S_1\cap S_2\cap S_3) =p_1p_2p_3 =(0.80)(0.70)(0.90)=0.504.

Thus the hypothetical conjunctive power is $0.504$, or $50.4\%$. Each pairwise comparison has at least $70\%$ power, but the chance that all three succeed is lower because all three successes must occur together. This interpretation depends on the stated independence assumption.

Worked Example: Calculating Pairwise Powers Before Multiplying

Consider three hypothetical comparisons whose test statistics are independent under the scenario of interest. Each comparison uses a one-sided normal test that rejects for $Z_i>1.96$. Suppose $Z_i\sim N(\delta_i/SE_i,1)$ under the scenario, with standardized means $\delta_i/SE_i$ equal to $2.80$, $2.50$, and $3.24$. Here $\delta_i$ is the treatment effect in the direction of benefit and $SE_i$ is its standard error. Calculate each pairwise power and the conjunctive power.

For a standard normal variable with mean $\delta_i/SE_i$ and variance 1, the probability of exceeding 1.96 is one minus the standard normal cumulative distribution function at $1.96-\delta_i/SE_i$. Writing $\Phi$ for the standard normal cumulative distribution function, this is:

p_i=P(Z_i>1.96) =1-\Phi\left(1.96-\frac{\delta_i}{SE_i}\right) =\Phi\left(\frac{\delta_i}{SE_i}-1.96\right).

Substituting the first standardized mean gives $p_1=\Phi(2.80-1.96)=\Phi(0.84)\approx0.7995$. For the second, $p_2=\Phi(2.50-1.96)=\Phi(0.54)\approx0.7054$. For the third, $p_3=\Phi(3.24-1.96)=\Phi(1.28)\approx0.8997$. These are probabilities of crossing the stated one-sided threshold under the specified normal models.

The test statistics are stipulated to be independent, so their rejection events are independent as well. Therefore:

\begin{aligned} P(S_1\cap S_2\cap S_3) &=(0.7995)(0.7054)(0.8997)\\ &\approx 0.5074. \end{aligned}

The conjunctive power is approximately $50.7\%$. The calculation uses the normal model and threshold specified in the example; in an application, the test model, standard errors, and any multiplicity-adjusted thresholds must match the planned analysis.

Worked Example: Finding the Power Needed for One Comparison

A hypothetical design has three independent comparison success events. The first two pairwise powers are $0.90$ and $0.80$. If the target conjunctive power is $0.70$, what power is required for the third comparison?

Under independence, the target equation is:

$$ (0.90)(0.80)p_3=0.70. $$

The first two factors multiply to $0.72$, so:

p_3=\frac{0.70}{0.72}\approx0.9722.

The third comparison would need pairwise power of about $97.2\%$, assuming the first two powers remain as stated and independence holds. This is a demanding requirement: the first two comparisons together already have an all-success probability of $0.72$, leaving limited room for failure on the third comparison if the overall target is $0.70$.

Checking the Assumptions and Diagnosing Problems

The product formula applies to the success events, not merely to a list of marginal powers. A practical justification should address the following points.

Use the same scenario throughout. Each $p_i$ and the joint probability must refer to the same specified treatment-effect configuration and analysis plan.
Define the events consistently. Identify what counts as success for every comparison, including the test direction and the planned threshold or other decision rule.
Justify mutual independence under that scenario. Independence under the null does not automatically establish independence under the alternative scenario used to calculate power.
Distinguish independence from similar-looking evidence. Equal marginal powers do not imply independence. Nor does a small observed correlation, by itself, prove that success events are independent.

A particular concern in multi-arm trials is that comparisons can share observations, such as a common control group. Shared data can make comparison outcomes dependent. Independence might be credible in a hypothetical setup with separate, independent data for each comparison, but it should not be assumed simply because the comparisons are listed separately. If independence is not justified, multiplying the marginal powers is not a valid derivation of the joint power; a dependence-aware calculation is needed.

The product also has useful checks. Since each $p_i$ is between 0 and 1, the product cannot exceed any individual factor: $\prod_i p_i\leq p_j$ for every $j$. If the calculated conjunctive power exceeds the smallest pairwise power, there is an arithmetic or interpretation error. If all $m$ comparisons have the same power $p$ and are independent, the result reduces to $p^m$. That special case is a check, not a reason to replace unequal powers by their average.

Common Mistake and Practical Limitations

Wrong reasoning: “The three comparisons have powers $0.80$, $0.70$, and $0.90$, so their conjunctive power is $0.504$.” This conclusion leaves out the condition that makes multiplication valid. The three marginal powers alone do not establish the joint probability.

Correction: State that the success events are mutually independent under the specified scenario, justify that assumption, and only then multiply the three probabilities. If dependence is present or cannot be ruled out, the product should not be reported as the conjunctive power.

Even when independence is justified, the result answers a specific question: the probability that all the defined comparisons succeed under one specified scenario. It is not a probability that the treatment effects are true, and it does not describe the probability of at least one success. It can also be sensitive to a single weak comparison: because every comparison must succeed, a low pairwise power can substantially reduce the product.

Key takeaway: Under mutual independence of the prespecified comparison success events, conjunctive power is the product of their pairwise powers. Keep unequal powers separate, and treat independence as an assumption to justify—not as a consequence of knowing the marginal powers.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

Three mutually independent comparison success events have powers $0.75$, $0.80$, and $0.90$. What is the conjunctive power?
Two independent comparisons have powers $0.60$ and $0.85$. A colleague reports conjunctive power $0.85$. Identify and correct the error.
Two comparisons each have power $0.80$, and their success events are independent. What is the conjunctive power? Why is it not $0.80$?
Three pairwise powers are known, but two comparisons share data and independence has not been established. Can their product be reported as conjunctive power? Explain.

Solutions. (1) $0.75\times0.80\times0.90=0.540$, so conjunctive power is $54.0\%$. (2) The colleague has reported the larger marginal power rather than the joint all-success probability. Under independence, the correct value is $0.60\times0.85=0.510$, or $51.0\%$. (3) The value is $0.80^2=0.640$, or $64.0\%$, because both successes must occur; the probability of both is not the probability of either one alone. (4) No. Sharing data can induce dependence, and the marginal powers alone do not determine their joint probability. The product is justified only if mutual independence is established under the scenario of interest.