Conditional Bayesian Power Under a Fixed Treatment Effect

What Conditional Bayesian Power Measures

A Bayesian trial may declare success when the posterior probability that the treatment effect exceeds a clinically relevant threshold is sufficiently high. Before the trial, however, the data are not yet observed, so it is unknown whether that posterior rule will be met. Conditional Bayesian power is the probability of meeting the rule when the treatment effect is fixed at a specified design value. It is an operating characteristic of the entire design-and-decision procedure.

This differs from the posterior probability reported at the end of a trial. The posterior probability is calculated from the observed data under the analysis model. Conditional power instead considers repeated possible datasets generated under a specified true treatment effect and asks how often those datasets would lead the analysis to declare success. The fixed effect is not drawn from a prior distribution in this calculation.

Definition: Let $\theta$ denote the treatment effect, let $\theta_{\min}$ be the minimum effect regarded as clinically relevant, and let $\gamma$ be the prespecified posterior-probability threshold for success. For trial data $Y$, define $q(Y)=P(\theta>\theta_{\min}\mid Y)$ under the analysis model. The decision rule declares success when $q(Y)\geq\gamma$. At a fixed design value $\theta^*$, conditional Bayesian power is $$ \beta(\theta^*)=P_{\theta^*}\{q(Y)\geq\gamma\}. $$ The probability $P_{\theta^*}$ is taken over repeated trial data generated when the true treatment effect is $\theta^*$, with other data-generating conditions held at their specified values.

The notation $P_{\theta^*}$ emphasizes the source of randomness: the trial data vary across repetitions, while the design effect is held fixed. If nuisance parameters, such as a baseline event rate or a variance, also affect the sampling distribution, they must either be fixed at stated values or handled by an explicitly defined averaging or worst-case procedure. That choice changes the operating characteristic being reported.

Conditions for a Meaningful Calculation

The calculation requires a complete specification of both the trial that generates data and the Bayesian analysis that turns data into a decision. These are related but distinct pieces. The data-generating model represents the assumed truth for evaluating performance; the analysis model and its prior determine the posterior probability used by the decision rule.

Specify the design value. State $\theta^*$ and its units or scale, and state the clinical threshold $\theta_{\min}$. “A treatment effect of 0.3” is incomplete unless the effect scale and units are identified.
Specify the data-generating conditions. Define the randomization, sample size, outcome process, follow-up, missing-data handling, and any nuisance-parameter values or scenarios. The generated data should reflect the trial whose performance is being assessed.
Specify the analysis rule. State the likelihood, analysis prior, posterior quantity, and threshold $\gamma$. If success also requires a safety condition or another criterion, that condition is part of the decision event.
Apply the same procedure to every replicate. Each simulated dataset must be analyzed using the planned method, including any model-fitting, stopping, or data-processing rules that would be used in the trial.

A prior used for the posterior analysis does not turn conditional power into an average over possible treatment effects. A design prior may be used for a different operating characteristic, but conditional power at $\theta^*$ evaluates one specified data-generating value. Nor does fixing $\theta$ imply that the posterior after observing data must be concentrated at that value: the posterior is an analysis result, not a statement that the data-generating value was sampled from the analysis prior.

From a Decision Rule to a Probability

For every possible dataset $y$, the analysis produces $q(y)$, and the decision rule maps that posterior probability to an indicator: one if the trial succeeds, zero otherwise. Conditional power is the expected value of this indicator under the sampling distribution at $\theta^*$.

I(y)=\mathbf{1}\{q(y)\geq\gamma\}, \qquad \beta(\theta^*)=E_{\theta^*}[I(Y)] =\int \mathbf{1}\{q(y)\geq\gamma\}\,p(y\mid\theta^*)\,dy.

For discrete outcomes, the integral is a sum over possible datasets. For complex designs, it is often approximated by simulation. This formulation also clarifies a useful distinction: a posterior threshold such as $q(y)\geq 0.95$ is a rule applied to each dataset; the resulting power might be 0.70, 0.90, or another value. The two probabilities answer different questions.

With $M$ independent simulated trials generated at $\theta^*$, let $I_m$ equal one if replicate $m$ meets the decision rule and zero otherwise. The simulated power is the mean of these indicators:

\widehat{\beta}(\theta^*)=\frac{1}{M}\sum_{m=1}^{M}I_m.

Each $I_m$ is a Bernoulli outcome with success probability $\beta(\theta^*)$. Thus the estimator is unbiased, and its variance is $\beta(\theta^*)\{1-\beta(\theta^*)\}/M$. Replacing the unknown power by its estimate gives the usual estimated Monte Carlo standard error:

\widehat{SE}_{\mathrm{MC}} =\sqrt{\frac{\widehat{\beta}(\theta^*)\{1-\widehat{\beta}(\theta^*)\}}{M}}.

This standard error describes simulation error in estimating the operating characteristic. It is not the standard error of a treatment-effect estimate in one trial, and it does not represent uncertainty about the true effect.

Worked Example: Exact Enumeration From a Discrete Sampling Distribution

Consider a hypothetical design evaluated at a fixed treatment effect of $\theta^*=0.20$ units. Its fully specified sampling model reduces the final evidence from each trial to one of four ordered categories. For this hypothetical model, the analysis produces the listed posterior probabilities, and the sampling model gives the category probabilities at $\theta^*=0.20$. The success rule is $P(\theta>0\mid Y)\geq 0.95$.

Evidence category	Sampling probability at $\theta^*=0.20$	$P(\theta>0\mid Y)$	Meets rule?
A	0.18	0.910	No
B	0.27	0.947	No
C	0.34	0.963	Yes
D	0.21	0.982	Yes

The category probabilities sum to $0.18+0.27+0.34+0.21=1.00$, as required. Categories C and D meet the rule because their posterior probabilities are at least 0.95. Therefore the conditional power is the sum of their sampling probabilities:

\beta(0.20)=P_{\theta=0.20}(C\text{ or }D) =0.34+0.21=0.55.

Under these specified conditions, the design has a 55% probability of meeting its posterior success rule when the true treatment effect is 0.20 units. The value 0.963 in the table is a posterior probability conditional on category C; 0.55 is a repeated-trial probability conditional on the fixed design value.

Simulation in More Complex Designs

When the possible datasets cannot be enumerated, simulation estimates the same probability. Each replicate must begin with data generated under the fixed design value and the chosen nuisance-parameter scenario. The planned Bayesian analysis is then applied, and the binary success indicator is recorded. Averaging those indicators estimates conditional power.

Set the scenario and rule.
Record $\theta^*$, nuisance values, trial design, analysis prior, posterior success quantity, and decision threshold.

Generate a trial dataset.
Draw one dataset from the specified sampling model at $\theta^*$, using the planned allocation and other design features.

Analyze and classify.
Calculate the posterior quantity and set the replicate indicator to one if every required success condition is met; otherwise set it to zero.

Repeat and summarize.
Repeat for $M$ independent trials, report the proportion meeting the rule, and include Monte Carlo uncertainty.

Worked Example: Estimating Conditional Power by Simulation

A hypothetical trial is evaluated at a fixed treatment effect of $0.30$ points on its prespecified outcome scale, with nuisance parameters held at their design values. In each of $M=800$ simulated trials, the planned analysis calculates $P(\theta>0\mid Y)$; success requires this posterior probability to be at least $0.975$. Of the 800 simulated trials, 548 meet the rule.

The estimate is

\widehat{\beta}(0.30)=\frac{548}{800}=0.685.

The estimated Monte Carlo standard error is

\widehat{SE}_{\mathrm{MC}} =\sqrt{\frac{0.685(1-0.685)}{800}} =\sqrt{\frac{0.215775}{800}} \approx 0.0164.

A rough 95% Monte Carlo interval is $0.685\pm1.96(0.0164)$, or approximately $0.653$ to $0.717$. This interval describes numerical uncertainty from using 800 simulations, not uncertainty in the effect or an interval for an individual trial’s outcome. The result supports a conditional power estimate of about 68.5% under the stated fixed-effect and nuisance-parameter conditions.

Thresholds, Scenarios, and Comparisons

The posterior threshold is part of the design, not a cosmetic reporting choice. Raising $\gamma$ makes the success criterion more demanding: for any fixed set of simulated datasets, the set that passes a higher threshold cannot be larger. Consequently, the estimated power cannot increase when only $\gamma$ is raised and all other ingredients remain unchanged.

Worked Example: Comparing Posterior Success Thresholds

Suppose the same 500 simulated datasets are analyzed at a fixed effect value of $0.25$ units. Their posterior probabilities are evaluated using the same analysis model. A threshold of 0.95 is met in 356 replicates, a threshold of 0.975 in 318, and a threshold of 0.99 in 271. Because these are nested rules applied to the same replicates, the success counts must be nonincreasing as the threshold rises.

The estimated powers are $356/500=0.712$, $318/500=0.636$, and $271/500=0.542$, respectively. For the 0.975 rule, the Monte Carlo standard error is

\sqrt{\frac{0.636(1-0.636)}{500}} =\sqrt{\frac{0.231504}{500}} \approx 0.0215.

Thus, under this fixed-effect scenario, the estimated probability of meeting the 0.975 rule is 63.6%, with Monte Carlo standard error about 2.15 percentage points. The three estimates are not three different truths about the trial; they describe the operating characteristics of three different decision rules.

Conditional power is also different from assurance. Assurance averages success probability over a distribution of plausible treatment effects, typically a design distribution; conditional power evaluates one effect value. A calculation across a grid of fixed effects gives an operating-characteristic curve. Averaging the values on that curve requires explicit weights or a probability distribution and is no longer power conditional on a single design value. Likewise, reporting a worst-case value over nuisance scenarios answers a different question from power at one specified nuisance setting.

A frequentist power calculation is also a repeated-sampling probability of a success event, but its test statistic and rejection rule need not be the Bayesian posterior threshold used here. Comparisons are informative only when the decision events are stated: two values both called “power” may refer to different rules, estimands, or assumed data-generating scenarios.

Diagnostics and Limitations

The calculation is only as credible as its assumed data-generating process and implementation of the planned analysis. Check that the simulated treatment effect is fixed at the stated value in every replicate; confirm that nuisance parameters and allocation match the intended scenario; and verify that the posterior calculation uses the intended prior and success threshold. For simulation, inspect whether the estimate is stable as the number of replicates increases, and report $M$ and the Monte Carlo standard error. If the estimate is close to a target, more replicates may be needed to distinguish a small difference from simulation noise.

Other warning signs include a success rate that rises when only the posterior threshold is raised, category probabilities that do not sum to one in an exact calculation, or many failed fits that are silently excluded from simulation. A failed or nonconverged analysis needs a prespecified handling rule; dropping such replicates without accounting for them can bias the estimated operating characteristic. If trial rules include interim stopping, missing outcomes, or multiplicity adjustments, omitting them can make the estimated power describe a procedure that will not actually be used.

One design value does not cover uncertainty across effects. A favorable $\theta^*$ can yield high conditional power while performance is poor at smaller effects. Report relevant fixed-effect scenarios rather than implying one value represents all plausible effects.
Nuisance parameters can matter. Holding them fixed is a conditional statement. If performance varies materially across plausible nuisance values, report a grid, a worst-case summary, or a clearly defined average.
Posterior success is not truth. A trial that meets a posterior threshold has met the stated decision rule; this alone does not establish that the treatment effect truly exceeds the clinical threshold.
Simulation error is not model error. Increasing $M$ reduces Monte Carlo error but does not repair a misspecified likelihood, an unrealistic design scenario, or an inappropriate decision criterion.

The central reporting requirement is therefore to pair the power estimate with its conditions: the fixed treatment effect and scale, nuisance scenario, trial design, analysis prior and model, posterior success rule, and method of calculation. Without these details, a conditional power number cannot be interpreted or reproduced.

Check Your Understanding

Answer each question, then compare with the solutions.

A trial has a posterior success threshold of 0.95. What probability does conditional power calculate at a fixed treatment effect?
In an exact calculation, three outcome categories have probabilities 0.20, 0.35, and 0.45 at $\theta^*$. The first and third categories meet the success rule. Calculate conditional power.
In 1,000 independent simulations, 720 trials meet the decision rule. Calculate estimated power and its estimated Monte Carlo standard error.
Why is the posterior probability $P(\theta>\theta_{\min}\mid Y)$ from one observed trial not itself conditional power?
If the posterior threshold rises while all simulated posterior probabilities and data-generating conditions stay fixed, what must happen to the success count?

Solutions. (1) It is the probability, over repeated datasets generated at the fixed treatment effect and specified nuisance conditions, that the posterior probability meets or exceeds 0.95. (2) The success probability is $0.20+0.45=0.65$. (3) The estimate is $720/1000=0.720$, and the Monte Carlo standard error is $\sqrt{0.720(0.280)/1000}=\sqrt{0.0002016}\approx0.0142$. (4) The posterior probability conditions on one observed dataset and quantifies uncertainty about the effect under the analysis model; conditional power averages the decision indicator over possible datasets at a fixed true effect. (5) The count can only stay the same or decrease, because every dataset passing the higher threshold also passed the lower one, but not necessarily the reverse.