Power Under Uncertain Biomarker Prevalence

Why Prevalence Uncertainty Changes a Power Calculation

A power calculation for a biomarker-defined subgroup depends on how many trial participants are expected to be biomarker-positive. If the prevalence is uncertain, calculating power at one chosen estimate can conceal a meaningful range of possible performance. The same uncertainty can also affect an all-comer claim when treatment effects differ between biomarker-positive and biomarker-negative participants: the population-average effect then depends on the prevalence.

The aim is not to replace a design assumption with a more elaborate calculation for its own sake. It is to describe how trial performance changes across plausible prevalence values, and to make clear which summary is being reported. In particular, average power over a distribution of plausible prevalences is not the same quantity as power evaluated at the average prevalence.

Conditional Power Under a Planning Model

Begin with a simple model to make the role of prevalence explicit. Suppose the outcome \(Y\) is continuous, larger values are beneficial, and the treatment-control contrast is estimated using independent randomized groups. Assume a common within-group standard deviation \(\sigma\), balanced allocation within the population being tested, and a large enough sample for a normal approximation. The test is two-sided at significance level \(\alpha\), and the treatment effect under the planning alternative is \(\delta\). These assumptions are planning approximations; departures from them require a different standard error or power calculation.

For a total of \(m\) participants in the population being tested, balanced allocation gives approximately \(m/2\) per arm. The standard error of the difference in means is \(SE=2\sigma/\sqrt{m}\). If the estimated contrast is approximately normal with mean \(\delta\) and standard error \(SE\), the rejection region is outside \([-z_{1-\alpha/2}SE,z_{1-\alpha/2}SE]\). Its power is

Here \(\Phi\) is the standard normal cumulative distribution function. The formula includes both tails of the two-sided test. For a subgroup analysis in an all-comer trial with total sample size \(N\), the expected subgroup size under a fixed prevalence \(\pi\) is \(N\pi\). A first-order planning calculation substitutes \(m=N\pi\). An all-comer analysis uses \(m=N\), but its treatment effect may depend on \(\pi\).

Suppose the treatment effect is \(\delta_+\) in biomarker-positive participants and \(\delta_-\) in biomarker-negative participants. If the subgroup effects are defined on the same outcome scale and the target is the population-average treatment contrast, then

Thus the all-comer power at prevalence \(\pi\) is calculated using \(m=N\) and \(\delta=\delta_A(\pi)\), under the stated common-variance approximation. If the outcome variance differs by biomarker stratum, or the analysis uses a different estimand or standard error, the all-comer power calculation must reflect that model rather than blindly using this formula.

From Conditional to Prevalence-Averaged Power

Let \(\operatorname{Power}_+(\pi)\) denote subgroup power conditional on prevalence \(\pi\), and let \(\operatorname{Power}_A(\pi)\) denote all-comer power conditional on \(\pi\). With \(g(\pi)\) representing a defensible distribution of plausible prevalence values, the corresponding average powers are

A short justification follows directly from conditioning. Let \(R\) be the event that the specified test rejects \(H_0\). Averaging the conditional rejection probability over the prevalence distribution gives

This identity is useful only after the target distribution and the conditional power model have been specified. The distribution might represent uncertainty about which recruitment population will be encountered, or uncertainty in an estimate derived from external data. It should not be treated as an objective feature of the trial unless its source and interpretation are justified.

In practice, the integral can be evaluated numerically. For a discrete set of scenarios \(\pi_1,\ldots,\pi_K\), with weights \(w_k\) that sum to one, use

For a continuous distribution, numerical integration or simulation can be used. When using simulation, draw prevalence values from the specified distribution, calculate conditional power for each draw, and average those powers. Report the distribution and the calculation method so the result can be reproduced.

All-Comer Power When Effects Differ by Biomarker

An all-comer estimate can have a different relationship with prevalence. Under the common-variance planning model, its standard error depends on \(N\), but its population-average treatment effect changes with the proportion in each stratum. Consequently, a prevalence distribution can change all-comer power even when the planned total sample size is fixed.

Prevalence Uncertainty Is Not the Same as Count Variation

The calculations so far use the expected subgroup size \(N\pi\) at each prevalence. In an actual trial, the observed number of biomarker-positive participants can vary around that expectation. If each enrolled participant’s biomarker status is an independent Bernoulli outcome with probability \(\pi\), then the subgroup count \(M\), conditional on \(\pi\), follows \(M\sim\operatorname{Binomial}(N,\pi)\). This additional sampling variation is distinct from uncertainty represented by \(g(\pi)\).

A more complete average-power calculation can average first over possible subgroup counts and then over prevalence:

Here \(\operatorname{Power}_+(m)\) uses the actual subgroup size \(m\), the planned treatment allocation, and the specified analysis. For \(m=0\), a subgroup test cannot be performed, so its rejection probability is zero. In real calculations, small or uneven arm counts may also require methods other than the simple normal approximation. The formula makes clear why using \(N\pi\) is a convenient approximation rather than an exact account of trial-level power.

Choosing a Summary and Checking the Model

Prevalence-averaged power is one useful summary, not a universal replacement for other planning summaries. Its interpretation depends on the decision being made and on the credibility of the prevalence distribution.

Several checks make the calculation more informative. Report the assumed prevalence distribution, its source, its range, and the conditional power at meaningful points such as lower and upper quantiles. Check whether the distribution places substantial weight on prevalences where subgroup power is below the design target. For an all-comer claim, also check how the assumed effects in each biomarker stratum determine \(\delta_A(\pi)\). If the result changes sharply under small changes to the distribution or treatment-effect assumptions, present that sensitivity rather than only one average.

The normal approximation, common variance, treatment-effect specification, and allocation assumptions should be checked against the planned design. If allocation differs by stratum, the subgroup standard error must reflect the actual numbers assigned to each arm. If outcome variance varies across strata, a common-\(\sigma\) calculation may misstate all-comer precision. If prevalence differs between the planning source and trial recruitment population, an otherwise correct integration can still answer the wrong question. Simulation can accommodate a more detailed design, but does not repair unsupported assumptions: it should implement a clearly stated data-generating and analysis model.

Common Mistake

A common mistake is to calculate power once at the mean prevalence and label it “average power.” That substitution is generally invalid because power is a nonlinear function of both the effect and the standard error: in general, \(E\{\operatorname{Power}(\pi)\}\ne\operatorname{Power}\{E(\pi)\}\). The correction is to calculate conditional power across the prevalence distribution and average those values with the specified weights. Report power at mean prevalence separately if it is also of interest.

A second interpretive pitfall is to treat 80% prevalence-averaged power as meaning that every prevalence scenario has 80% power. It means only that the weighted average of conditional powers is 80%. The conditional powers may lie both above and below that value, so their range or selected quantiles should accompany the average when design decisions depend on avoiding poor-performing scenarios.