Why Discount Historical Controls?
A small randomized trial may provide few current control observations, leaving substantial uncertainty about the control response rate. Historical controls can add information, but treating them as if they were randomized alongside current participants is risky: eligibility, supportive care, outcome assessment, and calendar time may differ. A power prior offers a transparent way to give historical data less than full weight. Its discount weight is an assumption to examine, not a guarantee that historical and current controls are comparable.
We develop the calculation for a binary endpoint where response is beneficial. Let \(x_T\) of \(n_T\) participants respond in the current treatment arm, and \(x_C\) of \(n_C\) respond in the current control arm. Let \(x_H\) of \(n_H\) respond among historical controls. The corresponding response probabilities are \(p_T\) and \(p_C\). The treatment effect is the risk difference \(\Delta=p_T-p_C\), measured in response-probability units.
Model and Posterior Calculation
Assume that current treatment outcomes are independent Bernoulli observations with probability \(p_T\), and that current and historical control outcomes are independent Bernoulli observations with probability \(p_C\). Thus their aggregated counts have binomial likelihoods. The central borrowing assumption is that, conditional on the model, historical and current controls share the same \(p_C\). This is a substantive exchangeability assumption: discounting the historical likelihood does not itself remove bias caused by differences between the historical and current settings.
For this case study, use independent uniform priors \(p_T\sim\operatorname{Beta}(1,1)\) and \(p_C\sim\operatorname{Beta}(1,1)\), and treat \(\delta\) as fixed. The treatment arm receives only its current randomized data. The control posterior combines the current control likelihood and the discounted historical likelihood:
Collecting the powers of \(p_C\) and \(1-p_C\) gives a beta posterior. More generally, with a \(\operatorname{Beta}(a_0,b_0)\) prior, its parameters are \(a_0+x_C+\delta x_H\) and \(b_0+n_C-x_C+\delta(n_H-x_H)\). The treatment posterior under the uniform prior is \(\operatorname{Beta}(1+x_T,1+n_T-x_T)\).
Under the model, the two posterior probabilities are independent because their priors and likelihood contributions factor by arm. The posterior distribution of \(\Delta\) is the distribution of the difference of these two beta-distributed probabilities; it is not generally itself a beta distribution. Its mean and variance are nevertheless straightforward to calculate. A \(\operatorname{Beta}(a,b)\) variable has mean \(a/(a+b)\) and variance \(ab/[(a+b)^2(a+b+1)]\). Independence then gives \(E(\Delta\mid\text{data})=E(p_T\mid\text{data})-E(p_C\mid\text{data})\) and \(\operatorname{Var}(\Delta\mid\text{data})=\operatorname{Var}(p_T\mid\text{data})+\operatorname{Var}(p_C\mid\text{data})\). These are summaries, not a claim that the difference is normally distributed.
Worked Example: A Half-Weight Historical Control
Suppose a hypothetical randomized trial has 8 responses among 12 current treatment participants and 3 among 12 current control participants. A hypothetical historical control set has 15 responses among 30 participants. Use \(\delta=0.5\) and the uniform priors above.
The treatment posterior is \(\operatorname{Beta}(9,5)\), with posterior mean \(9/14=0.6429\). For current control, the historical data contribute \(0.5(15)=7.5\) response counts and \(0.5(15)=7.5\) nonresponse counts to the beta parameters. Therefore,
The current-control posterior mean is \(11.5/29=0.3966\). The posterior mean risk difference is therefore \(0.6429-0.3966=0.2463\), or about 24.6 percentage points in favor of treatment. For its uncertainty, the treatment variance is \(9(5)/[14^2(15)]=0.01531\). The control variance is \(11.5(17.5)/[29^2(30)]=0.007978\). Thus the variance of the risk difference is \(0.01531+0.007978=0.02328\), and its posterior standard deviation is \(\sqrt{0.02328}=0.1526\).
For illustration, suppose the decision rule is to favor treatment when the posterior expected risk difference exceeds 0.25, a prespecified minimum expected benefit of 25 percentage points. Here \(0.2463<0.25\), so this rule does not favor treatment at \(\delta=0.5\). This is a decision under the stated rule, not evidence that treatment has no effect. A different utility threshold can produce a different decision.
Comparing Discount Weights
For a fixed prior and data set, changing \(\delta\) changes only the historical contribution to the control posterior. This sensitivity analysis makes the influence of the borrowing assumption visible. With the same hypothetical trial counts as above, the treatment posterior mean remains \(9/14=0.6429\) at every weight. The control posterior and the resulting mean risk difference are:
| Discount weight \(\delta\) | Control posterior | Posterior mean of \(p_C\) | Posterior mean of \(\Delta\) | Rule: mean difference > 0.25 |
|---|---|---|---|---|
| 0 | \(\operatorname{Beta}(4,10)\) | 0.2857 | 0.3571 | Favor treatment |
| 0.25 | \(\operatorname{Beta}(7.75,13.75)\) | 0.3605 | 0.2824 | Favor treatment |
| 0.5 | \(\operatorname{Beta}(11.5,17.5)\) | 0.3966 | 0.2463 | Do not favor treatment |
| 1 | \(\operatorname{Beta}(19,25)\) | 0.4318 | 0.2110 | Do not favor treatment |
For example, at \(\delta=0.25\), the control parameters are \(1+3+3.75=7.75\) and \(1+9+3.75=13.75\). The posterior mean difference is \(9/14-7.75/21.5=0.2824\). At \(\delta=1\), they are \(1+3+15=19\) and \(1+9+15=25\), giving \(9/14-19/44=0.2110\). The decision changes between the two lower and two higher weights under the stated rule. This is not a data-driven determination of the correct weight; it is evidence that the decision is sensitive to the borrowing assumption.
Worked Example: Deriving the Posterior Mean and Decision Rule
The general beta mean is \(a/(a+b)\). Under the uniform priors, the historical control likelihood contributes \(\delta x_H\) to the response parameter and \(\delta(n_H-x_H)\) to the nonresponse parameter. Their total contribution is \(\delta n_H\). Therefore, the current-control posterior mean is
This expression also checks the endpoint cases. Setting \(\delta=0\) leaves the historical counts out, yielding \((1+x_C)/(2+n_C)\). Setting \(\delta=1\) gives \((1+x_C+x_H)/(2+n_C+n_H)\), the mean that results from combining the two control data sets at full likelihood weight under this prior.
For the hypothetical trial, the numerator and denominator at \(\delta=0.5\) are \(1+3+7.5=11.5\) and \(2+12+15=29\), so the mean is \(11.5/29=0.3966\), as in the earlier calculation. Subtracting it from the treatment mean gives \(0.2463\). Since the rule is to favor treatment only if the posterior mean difference is greater than 0.25, the rule does not favor treatment at this weight. The calculation shows exactly how the posterior summary feeds into the decision; it does not turn the 0.25 threshold into a universal clinical standard.
What the Weight Does—and Does Not Do
The power prior has a useful interpretation for binomial data: at fixed \(\delta\), historical responders and nonresponders contribute fractional counts to the beta posterior parameters. The effective historical sample size in this likelihood contribution is \(\delta n_H\). This is a description of the model’s weighting, not a claim that the trial literally observed that many additional participants. In particular, the prior still contributes its own parameters, and the effective sample size does not quantify uncertainty about whether the historical and current populations are comparable.
Two simple alternatives clarify the role of the power prior. With no borrowing, set \(\delta=0\) and analyze current controls alone. With full pooling, set \(\delta=1\), which assumes the historical control likelihood is fully applicable. A fixed intermediate weight lies between these choices in the likelihood contribution. The advantage is transparent sensitivity analysis; the limitation is that the analyst must choose the weight, and a single weight cannot represent every possible pattern of historical-current discrepancy.
Worked Example: A Historical Control Rate That Conflicts With Current Data
Keep the hypothetical randomized trial at 8/12 treatment responses and 3/12 current control responses, but suppose the historical controls instead had 22 responses among 30 participants. With \(\delta=0.5\), historical data contribute \(0.5(22)=11\) response counts and \(0.5(8)=4\) nonresponse counts. The current-control posterior is
Its mean is \(15/29=0.5172\). The treatment posterior mean remains \(9/14=0.6429\), so the mean risk difference is \(0.6429-0.5172=0.1256\). Under the same 0.25 decision rule, treatment is not favored. The historical response rate, \(22/30=0.7333\), is markedly higher than the current control rate, \(3/12=0.2500\). The shift in the posterior control mean is a warning to investigate differences in eligibility, calendar time, outcome measurement, and standard of care. It is not a reason to assume that the model has repaired those differences.
Diagnostics, Assumptions, and Common Pitfalls
Before reporting a borrowed analysis, inspect whether the historical and current control groups could plausibly share a response probability. Compare their observed rates and the clinical features that affect response. Check whether endpoint definitions, follow-up, missing-data handling, and treatment context align. A large discrepancy is a reason to investigate and to report less-borrowing analyses, not a proof by itself that the historical data are unusable or exchangeable.
A common mistake is to say, “The historical sample is large, so pooling it fully must improve the analysis.” Full pooling can reduce posterior uncertainty under the shared-\(p_C\) model, but if the shared-probability assumption is false, it can pull the current-control estimate away from the relevant rate and distort the treatment effect. The correction is to state the exchangeability assumption, justify the chosen weight clinically, and show how conclusions change across plausible weights, including no borrowing. A smaller posterior standard deviation is not a diagnostic that the assumption holds.
Other limitations matter in practice. A fixed \(\delta\) does not estimate how compatible the historical data are with current data. The basic model has no covariate adjustment, so it cannot directly account for measured differences in patient mix. The beta-binomial model also presumes independent binary outcomes within each group and a common response probability within each modeled group. If the endpoint is not binary, if follow-up differs substantially, or if the data structure violates these assumptions, this particular calculation is not the appropriate model without modification.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- With a \(\operatorname{Beta}(1,1)\) prior, 2 responses among 10 current controls, and no borrowing, what is the control posterior and its mean?
- For 15 responses among 30 historical controls, how many response and nonresponse counts enter the likelihood contribution at \(\delta=0.4\)?
- In the main worked trial, what are the current-control posterior parameters at \(\delta=1\), and what is the posterior mean risk difference?
- Why can full pooling give a misleadingly precise estimate when historical and current controls differ?
Solutions. (1) The posterior is \(\operatorname{Beta}(1+2,1+8)=\operatorname{Beta}(3,9)\), with mean \(3/12=0.2500\). (2) The historical contribution is \(0.4(15)=6\) response counts and \(0.4(15)=6\) nonresponse counts. (3) The control posterior is \(\operatorname{Beta}(1+3+15,1+9+15)=\operatorname{Beta}(19,25)\); its mean is \(19/44=0.4318\). The treatment mean is \(9/14=0.6429\), so the mean risk difference is \(0.6429-0.4318=0.2110\). (4) Full pooling assumes the historical and current control probabilities are the same. If that assumption fails, the historical observations can pull the estimate away from the current control rate while the model still treats them as relevant information, understating uncertainty about the mismatch.