Power Priors for Single-Arm Historical Benchmarks

Why Compare a Single-Arm Trial With a Historical Benchmark?

A single-arm trial estimates the response probability among participants receiving the study treatment, but it has no concurrent control group. A historical control rate can provide context: the question may be whether the current response probability is greater than a benchmark informed by earlier control data. A power prior offers one way to discount that historical evidence rather than treating it as fully equivalent to current information.

The resulting comparison is a posterior probability about two rates under a specified model. It is not, by itself, proof that treatment caused an improvement: differences in eligibility, care, outcome assessment, or calendar time can separate the historical benchmark from the rate that would have occurred in the current trial without treatment. The analysis therefore depends on both the probability model and the clinical credibility of the benchmark.

Model and Assumptions

Assume the historical outcomes are conditionally independent Bernoulli observations with common response probability \(p_H\), so \(X_H \mid p_H \sim \operatorname{Binomial}(n_H,p_H)\). Assign \(p_H\) a beta prior, \(p_H \sim \operatorname{Beta}(\alpha_H,\beta_H)\), with positive shape parameters. For a fixed discount \(a\), the historical contribution to the posterior is proportional to

The current outcomes are assumed conditionally independent Bernoulli observations with common response probability \(p_T\), so \(Y \mid p_T \sim \operatorname{Binomial}(n,p_T)\). Assign \(p_T \sim \operatorname{Beta}(\alpha_T,\beta_T)\). For the simple comparison developed here, the two prior distributions are independent and the two data sets have independent likelihoods conditional on their respective response probabilities.

These assumptions require more than correct arithmetic. The endpoint and response definition must be meaningful across data sets; the binomial model treats participants within each group as having the same response probability; and the fixed discount must be chosen or justified for the analysis. Most importantly, borrowing historical information is credible only to the extent that the historical rate is relevant to the current clinical setting. Discounting reduces its mathematical influence, but cannot repair a mismatched endpoint or a materially different patient population.

Updating the Historical Benchmark and Current Rate

Conjugacy makes both updates explicit. Multiplying the historical beta prior by the powered likelihood adds \(aX_H\) to the first shape parameter and \(a(n_H-X_H)\) to the second. The current binomial likelihood updates its beta prior in the usual way. Under the stated independence assumptions, the resulting posteriors are independent:

This calculation uses a fixed discount. When \(a=1\), the historical likelihood is used without discounting; when \(a=0\), it makes no contribution to the historical parameter’s posterior. For intermediate values, the historical successes and failures each contribute fractionally. This is the likelihood-weighting interpretation of the power prior; it does not make the historical participants actual participants in the current trial.

A derivation follows directly from the beta density. Ignoring factors that do not depend on \(p_H\), the posterior kernel is

This is the kernel of the beta distribution with the updated shape parameters shown above. Applying the same multiplication to the current prior and likelihood gives the stated posterior for \(p_T\). With independent priors and separate likelihoods, the joint posterior density factors into the product of these two beta densities.

Comparing the Two Posterior Rates

A direct Bayesian comparison averages over uncertainty in both rates. If \(f_T\) is the posterior density of \(p_T\), and \(F_H\) is the posterior cumulative distribution function of \(p_H\), independence gives

For each possible current rate \(t\), \(F_H(t)\) is the posterior probability that the historical rate is below \(t\); averaging that quantity over the posterior for \(p_T\) yields the comparison probability. In general, beta distributions with different shape parameters do not reduce this comparison to a simple beta posterior. Numerical integration or draws from the two posterior distributions can evaluate it. In a simulation calculation, draw \(p_T^{(m)}\) and \(p_H^{(m)}\) independently from their respective posteriors, then estimate the probability by the proportion of draws for which \(p_T^{(m)}>p_H^{(m)}\).

A Fixed Benchmark Is a Different Comparison

Sometimes the decision question is framed against a fixed clinical threshold \(p_0\), rather than an uncertain historical response probability. The posterior probability \(P(p_T>p_0 \mid Y)\) is then a one-posterior tail probability. It is not the same as \(P(p_T>p_H \mid X_H,Y)\): replacing \(p_H\) by its posterior mean ignores uncertainty in the benchmark and will generally change the result.

Interpretation, Diagnostics, and Limitations

The comparison probability is conditional on the likelihoods, priors, discount, and independence assumptions. A posterior probability such as 0.858 can be compared with a prespecified decision threshold, but that threshold is a decision rule, not the frequentist significance level \(\alpha\). The clinical consequences of false-positive and false-negative decisions, and the costs of treatment or further study, should inform the decision threshold.

Useful checks include verifying that the response counts lie between zero and their sample sizes, checking the discount and prior values used in both the reported posterior and any software, and confirming that the comparison integrates uncertainty in both rates when that is the intended estimand. Repeating the comparison over scientifically plausible discounts and prior choices can reveal whether the conclusion depends heavily on borrowing. A striking shift in the posterior probability as \(a\) changes is a sensitivity warning, not evidence that one discount is automatically correct.

The central limitation is that a historical comparison cannot rule out systematic differences between historical controls and current patients or care. A power prior changes the weight assigned to historical evidence; it does not model every source of between-study variation or remove confounding. Nor should the historical benchmark be confused with a directly observed concurrent control rate. Randomized comparisons address a different design question, while a fixed-rate analysis answers a different benchmark question.