Defining Conjunctive Success for Effective Arms

Why the Definition of Conjunctive Success Matters

In a multi-arm trial, a treatment can be effective while another is not. A power calculation must therefore say exactly which treatment successes are required. Requiring every treatment arm to succeed is a different criterion from requiring success on every arm that truly meets the efficacy target. These criteria can give different power values for the same trial, same analysis, and same true effects.

The distinction is about how we define the trial-level success event. It does not change which hypotheses belong to the prespecified testing family, nor does it authorize selecting arms after seeing the results. As in the earlier material on power under mixed treatment-effect configurations, the true effect configuration is specified for evaluating the probability of success; it is not inferred from the observed trial outcomes.

“Effective” must have a statistical meaning fixed by the trial’s objectives. For example, it might mean that an arm’s true treatment effect is in the prespecified beneficial alternative, or that it exceeds a clinically relevant threshold. It should not mean merely that an observed estimate looks favorable. The set \(E\) is determined by the true parameter values and the stated target, not by which tests happen to be significant.

Each \(S_i\) also depends on the trial’s prespecified analysis and decision rule. It may mean that the hypothesis for arm \(i\) is rejected using a multiplicity-adjusted procedure, for example. The definitions below apply to whatever valid success rule has been specified; they do not assume that the arm-level events are independent.

Two Different Conjunctive Criteria

The all-arm criterion counts the trial as successful only if every arm succeeds, including arms that are truly ineffective under the configuration being evaluated. The effective-arm criterion requires all genuinely effective arms to succeed, while not requiring success on arms outside \(E\). Thus the latter is a criterion for detecting all true-effective arms, not necessarily for making correct decisions about every arm.

When \(E\) is nonempty and is a subset of the \(K\) arms, every all-arm success entails effective-arm success. In symbols, \(S_{\mathrm{all}}\subseteq S_E\). Therefore,

This inequality does not require independence. It follows simply because an event contained in another event cannot have greater probability. Equality can occur, for example, when every arm is effective, so the two events are identical. When one or more arms are ineffective, the all-arm criterion can be more demanding: it also requires success on those arms.

Deriving the Relationship When Ineffective Arms Are Present

Let \(N\) be the set of arms outside \(E\), so that \(N=\{1,\ldots,K\}\setminus E\). The all-arm event can be written as the intersection of effective-arm success and success on every other arm:

If \(P(S_E)>0\), the multiplication rule for conditional probability gives the exact relationship

The conditional probability on the right is between zero and one. This gives \(P(S_{\mathrm{all}})\leq P(S_E)\) and shows what drives the gap: it is the probability that every non-effective arm also succeeds, conditional on all effective arms succeeding. If \(P(S_E)=0\), then \(S_{\mathrm{all}}\subseteq S_E\) implies \(P(S_{\mathrm{all}})=0\), so the inequality still holds. No independence assumption is needed for this argument.

If there are no non-effective arms, the intersection over \(N\) imposes no additional requirement and the events coincide. If there are non-effective arms, multiplying marginal probabilities for all arms is justified only under an appropriate independence assumption. In general, the conditional-probability expression above remains valid, but its conditional probability must be obtained from the joint distribution or another justified calculation.

Worked Calculation From a Complete Set of Probabilities

The criterion can also be compared when probabilities are provided for the relevant intersections directly. This avoids an independence assumption and makes clear which event each reported power refers to.

How to Specify and Report the Criterion

A power statement is incomplete if it says only “conjunctive power” without identifying which arms must succeed. A clear report names the efficacy configuration, the success rule, and the event whose probability is being evaluated. The testing family and multiplicity procedure should remain the prespecified ones. Defining an effective-arm criterion for a particular true configuration does not remove other hypotheses from that family.

For example, a precise report might say: “Under the configuration in which arms 1 and 3 meet the efficacy target and arm 2 does not, the probability of declaring both effective arms successful under the prespecified procedure is 56.0%.” If the all-arm criterion is intended, the report should instead say that success on all three arms is required and give that probability. Such wording prevents the numerical result from being interpreted as a different success criterion.

Diagnostics, Limitations, and a Common Pitfall

A useful diagnostic is to check event inclusion before trusting numerical results. When \(E\) is a subset of all arms, effective-arm conjunctive power cannot be smaller than all-arm conjunctive power for the same configuration and success rules. If a calculation reverses that ordering, likely causes include using different procedures, confusing an intersection with a union, or attaching probabilities to the wrong arms.

• Do not assume independence automatically. Pairwise treatment comparisons can be dependent, so a product of marginal powers need not equal a joint power. Use a justified joint model, simulation, or the relevant conditional probabilities.
• Do not use observed results to define \(E\). The effective-arm set is fixed by the true configuration for the power calculation and the prespecified target. Selecting it after looking at the trial results changes the interpretation.
• Do not equate effective-arm success with complete trial correctness. The event \(S_E\) does not require non-effective arms to fail. A trial can satisfy this criterion and still declare an ineffective arm successful.

The last point is especially important. Effective-arm conjunctive power answers, “What is the probability that every truly effective arm succeeds?” It does not, by itself, answer, “What is the probability that every effective arm succeeds and every ineffective arm fails?” That stricter event is different. If the scientific objective requires protection against false declarations as well as detection of all effective arms, both requirements must be stated and evaluated rather than silently treating one as the other.

A further limitation is that effective-arm power is configuration-specific: changing which arms are truly effective changes \(E\), and potentially changes the probability. It is a useful criterion for describing performance under a specified configuration, but it is not a single universal summary of a design. The all-arm criterion is also configuration-specific in probability, although its event always includes success on every tested arm.