Reporting Multi-Arm Power Unambiguously

Why a Power Number Needs a Definition

A statement such as “the trial has 80% power” is incomplete when a trial compares several treatments. It might mean an 80% chance of detecting a benefit for one particular treatment, an 80% chance of detecting a benefit for at least one treatment, or an 80% chance of detecting benefits for all treatments in a specified set. These are different events and can have different probabilities under the same design.

A useful report makes the probability being calculated reconstructible. It identifies the success event, the arms that count toward success, the treatment effects assumed for each arm, the test-level powers under that configuration, and how dependence among the tests was handled. The result is then a probability of a stated event under a stated configuration, not a free-standing property of the design.

The event must follow the planned decision rule. For example, if success means rejecting the treatment-control null hypothesis using a prespecified one-sided test at a stated threshold, say so. If a treatment can be declared successful only after satisfying a sequential rule across stages, describe that rule as part of the event. A power result is only as clear as its definition of \(R_i\).

The effect configuration matters just as much. It specifies the assumed effect for each arm, including arms assumed to have no effect or an effect smaller than the target effect. If an analysis uses direction-specific tests, the assumed direction must be clear too. Test-level powers are conditional on these assumptions; they are not fixed characteristics of the arms.

A Reproducible Reporting Framework

A concise report can be organized around six questions. The answers need not appear in a rigid template, but each should be available to someone interpreting or reproducing the calculation.

This framework separates the target probability from the numerical method used to obtain it. A calculation might use a complement rule, a multivariate distribution, or another valid approach; the report still needs to identify the same event and assumptions. If dependence is left unstated, a reader cannot tell whether the joint value follows from independence or from a specified dependent model.

Worked Examples

Justifying the Event-Based Calculation

The reporting principle follows directly from probability. Once the success event \(A\) and effect configuration \(\boldsymbol{\theta}\) have been defined, power is \(P(A\mid\boldsymbol{\theta})\). For disjunctive success across a target set \(S\), the event is \(\bigcup_{i\in S}\{R_i=1\}\); its complement is that every target test fails. Therefore:

This identity does not assume independence. Independence is required only if the probability of the intersection on the right is replaced by the product of individual failure probabilities. Similarly, conjunctive power is \(P(\bigcap_{i\in S}\{R_i=1\}\mid\boldsymbol{\theta})\); it equals the product of marginal powers only under independence. Naming the event first prevents an unjustified product or complement calculation from being mistaken for a general result.

Diagnostics, Alternatives, and Limitations

A report can be checked for completeness before its numerical result is trusted. Ask whether the target set and success event agree; whether the effects assumed for each arm are explicit; whether the rejection rule is stated; and whether the dependence assumption matches the calculation. Also check that the reported joint probability is compatible with its marginal probabilities. For example, a probability of both successes cannot exceed either marginal probability.

Common warning signs include a single “power” percentage with no event, a power profile without a declared target criterion, and a product of marginal powers without an independence assumption. Another failure is to define a target set by arm labels while assuming an effect configuration that makes some of those arms ineffective, without explaining whether they still count toward success. If the event concerns only a subset, state that explicitly.

Reporting marginal powers separately is useful when readers need to compare arms, but it answers a different question from reporting joint power. A disjunctive criterion is appropriate when success requires evidence for at least one target treatment; a conjunctive criterion requires evidence for every target treatment; an at-least-\(k\) criterion requires a specified number of successes. None is universally preferable: the appropriate event follows from the trial objective.

A joint power calculation remains conditional on its assumptions. It does not guarantee that the assumed effects will occur, and it does not describe performance under effect configurations that were not evaluated. A transparent report therefore presents joint power alongside the effect configuration and enough detail about tests and dependence to show what the result means. In a multi-stage setting, the event must reflect the actual stagewise decision rule; a terminal rejection probability cannot automatically stand in for a different rule.

Common Mistake

Wrong reasoning: “Each of three arms has about 80% power, so the trial has 80% power.” This treats three marginal probabilities as though they identified one joint probability, without saying what event defines trial success.

Correction: State whether success means any, all, or a specified number of rejections among named arms. Then specify the effect configuration, the arm-specific decision rules, and the dependence assumption before calculating the joint probability. If the intended measure is simply a particular arm’s power, identify that arm rather than calling it the power of the whole multi-arm trial.