Hybrid Bayesian Success Criteria and Frequentist Constraints

Why Combine Bayesian and Frequentist Criteria?

A trial may define success using a posterior probability, such as requiring at least a 97.5% posterior probability that the treatment effect is positive. A sponsor or regulator may also require that the design control the probability of a false positive at a prespecified level and attain adequate power at a clinically relevant effect. These requirements answer different questions: the posterior rule describes how the trial decision is made from the observed data, while frequentist constraints describe how often that rule would succeed across repeated trials under specified true effects.

A hybrid design must satisfy both requirements for the same decision rule. It is not enough to quote the posterior threshold, or to choose a sample size that meets a power target under a different test. The analysis model, prior, success rule, sample size, and frequentist operating characteristics must all be specified consistently.

Model and Decision Rule

Consider a hypothetical, balanced, two-arm trial with a continuous endpoint. Let \(\theta\) be the treatment-minus-control difference in population means, measured in endpoint units. Assume independent normally distributed outcomes in the two arms with a common, known standard deviation \(\sigma\). With \(n\) participants per arm, the observed difference in sample means, denoted \(\bar{D}\), has sampling distribution

For the Bayesian analysis, use the proper prior \(\theta\sim N(0,\tau^2)\), independent of the trial data. This prior is centered at no treatment difference and has variance \(\tau^2\). Under this model, the posterior distribution is normal. Define \(w_n=\tau^2/(\tau^2+s_D^2)\). Then

Suppose the prespecified Bayesian success rule is \(P(\theta>0\mid\bar{D})>\gamma\), where \(\gamma\) is the chosen posterior-probability threshold. Write \(z_\gamma\) for the \(\gamma\) quantile of the standard normal distribution. Since the posterior probability is \(\Phi\bigl(w_n\bar{D}/\sqrt{w_ns_D^2}\bigr)\), the rule is equivalent to

Thus, under this particular normal model, the Bayesian rule is an observable-data cutoff. The cutoff depends on the sample size and on the analysis prior through \(w_n\). The formula assumes the stated endpoint model, known variance, proper normal prior, and a success criterion based on a positive effect. If success instead requires exceeding a clinically important margin, the posterior probability and cutoff must be defined relative to that margin.

Deriving the Frequentist Constraints

For the decision rule \(\bar{D}>c_n\), the probability of declaring success when the true effect is \(\theta\) is

This is the operating-characteristic function for this design: it gives the success probability at each specified true effect. For the one-sided null \(H_0:\theta\leq 0\), the probability of success is largest at the boundary \(\theta=0\), because the displayed expression increases as \(\theta\) increases. The type I error is therefore

At a prespecified alternative \(\theta=\delta>0\), the power is

A design with target type I error \(\alpha^*\) and target power \(1-\beta^*\) must satisfy \(\alpha_n\leq\alpha^*\) and \(\operatorname{Power}_n(\delta)\geq1-\beta^*\) for the same \(n\), prior, endpoint model, and success rule. If there are nuisance parameters, the required error-rate or power criterion must also state how they are handled, for example by taking a worst case over a specified range.

Worked Examples

What the Joint Check Does—and Does Not—Guarantee

These examples show why the two criteria must be evaluated together. A posterior probability threshold does not, by itself, equal a frequentist type I error rate. In the normal model above, the prior affects the relationship: for a fixed \(\gamma\), \(w_n<1\) makes the data cutoff more demanding than the large-sample cutoff \(z_\gamma s_D\). As \(n\) grows, \(w_n\) approaches 1, and the type I error approaches \(1-\gamma\). With \(\gamma=0.975\), that limit is 0.025, not 0.01. At small sample sizes, however, the error can be below 0.01; it is incorrect to claim it always exceeds 0.01.

A hybrid design differs from two common alternatives. A posterior-only design chooses the Bayesian rule without imposing a repeated-sampling error-rate or power requirement. A conventional frequentist design specifies a test and its error rates, without making a posterior-probability threshold its success definition. A hybrid design retains the stated posterior rule but checks frequentist behavior for that same rule. Its constraints may lead to a stricter posterior threshold, a different sample size, or a conclusion that the requested combination is infeasible under the assumed model.

The calculation also has limits. It relies on a known variance, normal sampling model, balanced allocation, and the specified normal analysis prior. Unknown variance, time-to-event outcomes, missing data, nonadherence, multiple endpoints, or a different analysis can change the operating characteristics. In such settings, the design must evaluate the actual planned analysis and decision rule; substituting the formulas here without checking their assumptions is not justified.

Diagnostics and Common Pitfalls

• Check that the estimand and success event match. The examples define \(\theta\) as treatment minus control and success as evidence that \(\theta>0\). Reversing the contrast or using a positive clinical margin changes the rule.
• Check the null being protected. For \(H_0:\theta\leq0\), the boundary \(\theta=0\) gives the maximum false-positive probability in this model. If the null or nuisance-parameter space differs, evaluate the appropriate supremum rather than only one convenient case.
• Do not assume a posterior threshold guarantees a chosen error rate. Calculate the type I error for the actual sample size and prior. In particular, a threshold of 0.975 does not generally impose a 0.01 type I error bound.
• Recheck all constraints after changing the design. Changing \(n\), \(\gamma\), allocation, or the prior changes the cutoff and can change both power and type I error. A design that meets one target may fail another.
• Inspect the model and computation. Check that variance and effect units agree, that the planned sample size is per arm or total as stated, and that numerical calculations use unrounded values until the final report.