Why Historical Outcomes Matter at a Fixed Discount
A fixed discount determines how much a historical dataset contributes to a current control analysis, but it does not determine the direction of that contribution by itself. Historical outcomes with a high success rate can pull the estimated control success probability upward; outcomes with a low success rate can pull it downward. The size of the shift also depends on the amount of current information, the historical sample size, and the prior.
This tutorial quantifies that influence for a binary endpoint. The aim is not only to calculate a posterior at each chosen discount, but to compare it with the posterior based on current data alone. That comparison makes the historical dataset’s contribution explicit.
Model and Assumptions
Let \(p\) be the probability of success on a binary control endpoint. Suppose the current control group has \(y_c\) successes among \(n_c\) participants, and the historical control data have \(y_h\) successes among \(n_h\) participants. Use a proper beta prior, \(p\sim\operatorname{Beta}(a_0,b_0)\), where \(a_0>0\) and \(b_0>0\). The beta distribution is parameterized by its two shape parameters.
Assume the current and historical outcomes are conditionally independent Bernoulli observations given a common \(p\). In particular, this analysis assumes that the endpoint, population, and outcome interpretation are sufficiently aligned that the same success probability is a reasonable model for both datasets. Assume also that the historical dataset is represented by its binomial likelihood and that the discount \(\delta\) is fixed for the calculation, with \(0\leq\delta\leq1\). These are substantive modeling assumptions, not consequences of the arithmetic.
The power prior raises the historical likelihood to the power \(\delta\). With a fixed discount, the resulting posterior is
The current observations enter at full weight; the historical successes and failures enter at weight \(\delta\). At \(\delta=0\), the historical likelihood contributes nothing. At \(\delta=1\), it contributes at full likelihood weight under the common-\(p\) model. Intermediate values give fractional contributions to the beta shape parameters.
Write \(m_0\) for the current-only posterior mean and \(m_\delta\) for the posterior mean at discount \(\delta\). Define \(K=a_0+b_0+n_c\), the total current-only posterior shape, and \(p_h=y_h/n_h\), the historical observed success proportion when \(n_h>0\). The beta mean gives
Deriving the Mean Shift
The difference between the discounted and current-only posterior means has a particularly useful form. Substitute \(a_0+y_c=Km_0\) and \(y_h=n_hp_h\), then subtract the current-only mean:
Thus the mean shift has the sign of \(p_h-m_0\): the historical rate pulls the posterior mean toward itself. Its magnitude is the gap between the historical rate and the current-only posterior mean, multiplied by \(\delta n_h/(K+\delta n_h)\), a factor between zero and one. The historical data do not simply add a fixed amount to the estimate; their influence is moderated by both the discount and the current-only information.
This identity also gives useful checks. If \(\delta=0\), or if \(p_h=m_0\), the shift is zero. For a positive discount and nonzero historical sample size, \(m_\delta\) lies between \(m_0\) and \(p_h\). A calculation outside that range signals an arithmetic or model-parameterization error.
Worked Comparisons Across Fixed Discounts
Worked Example: Opposite Historical Success Rates
Suppose, hypothetically, that 8 of 20 current control participants respond. Use a \(\operatorname{Beta}(1,1)\) prior. Compare two hypothetical historical datasets, each with 10 participants: one with 7 responses and one with 2 responses. Assume the binary endpoint and populations are aligned as required by the model.
The current-only posterior is \(\operatorname{Beta}(1+8,1+12)=\operatorname{Beta}(9,13)\), so \(K=22\) and \(m_0=9/22=0.4091\). For the historical dataset with 7 responses, the posterior mean is
At \(\delta=0.1\), \(m_\delta=(9+0.7)/(22+1)=9.7/23=0.4217\), an increase of \(0.01265\) over the current-only mean. At \(\delta=0.5\), \(m_\delta=(9+3.5)/(22+5)=12.5/27=0.4630\), an increase of \(0.05387\). At \(\delta=1\), \(m_\delta=16/32=0.5000\), an increase of \(0.09091\).
For the historical dataset with 2 responses, the corresponding mean is \(m_\delta=(9+2\delta)/(22+10\delta)\). At \(\delta=0.1\), this is \(9.2/23=0.4000\); at \(\delta=0.5\), it is \(10/27=0.3704\); and at \(\delta=1\), it is \(11/32=0.3438\). The same discount and historical sample size produce shifts in opposite directions because the historical success rates lie on opposite sides of \(m_0\).
| Historical outcomes | Discount \(\delta\) | Posterior mean | Shift from \(m_0=0.4091\) |
|---|---|---|---|
| 7 responses in 10 | 0.1 | 0.4217 | +0.01265 |
| 7 responses in 10 | 0.5 | 0.4630 | +0.05387 |
| 7 responses in 10 | 1 | 0.5000 | +0.09091 |
| 2 responses in 10 | 0.1 | 0.4000 | -0.009091 |
| 2 responses in 10 | 0.5 | 0.3704 | -0.03872 |
| 2 responses in 10 | 1 | 0.3438 | -0.06534 |
Worked Example: Influence of One Changed Historical Outcome
Keep the current data and prior from the previous example, and fix \(\delta=0.4\). Compare two hypothetical historical datasets of 10 participants that differ only in one outcome: the first has 5 responses and the second has 6. This comparison isolates the effect of changing one historical failure to a success while holding the historical sample size fixed.
With 5 historical responses, the posterior is \(\operatorname{Beta}(1+8+0.4(5),\,1+12+0.4(5))=\operatorname{Beta}(11,15)\), with mean \(11/26=0.4231\). With 6 responses, it is \(\operatorname{Beta}(11.4,14.6)\), also with total shape 26, and mean \(11.4/26=0.4385\). The difference is \(0.4385-0.4231=0.01538\).
In general, at fixed \(n_h\) and fixed \(\delta\), replacing one historical failure with one success increases the posterior mean by \(\delta/(K+\delta n_h)\). Here that is \(0.4/(22+0.4(10))=0.4/26=0.01538\), matching the direct calculation. This is a change in the posterior mean, not a statement that one individual’s outcome determines the underlying probability.
Worked Example: Same Historical Rate, Different Sample Sizes
Again use 8 responses among 20 current controls and a \(\operatorname{Beta}(1,1)\) prior. At \(\delta=0.5\), compare hypothetical historical data with 7 responses among 10 participants and 14 responses among 20 participants. Both historical datasets have a 70% response rate, but one contains twice as many observations.
For 7 of 10, the posterior mean is \((9+0.5(7))/(22+0.5(10))=12.5/27=0.4630\). For 14 of 20, it is \((9+0.5(14))/(22+0.5(20))=16/32=0.5000\). The larger dataset exerts more influence because its discounted sample size is 10 rather than 5. Both means move upward from \(0.4091\) toward the same historical rate, but the larger dataset moves farther.
The calculation does not say the two datasets are equally informative merely because their observed proportions match. At a fixed discount, the historical sample size determines how much likelihood weight is contributed, while the observed rate determines the direction of the pull.
Interpreting Influence Without Overstating It
A posterior mean summarizes location, not the entire posterior distribution. The posterior variance is \(ab/[(a+b)^2(a+b+1)]\) for a \(\operatorname{Beta}(a,b)\) distribution. It depends on both posterior shape parameters, so two datasets can produce similar means but different posterior uncertainty. A mean shift alone does not determine a credible interval, a posterior probability of exceeding a clinical threshold, or a treatment decision.
The arithmetic also does not establish that historical and current participants truly share one response probability. If the endpoint definitions, patient mix, treatment context, or follow-up differ materially, the common-\(p\) assumption may be implausible. Discounting reduces the historical likelihood contribution; it does not automatically correct a mismatch or remove bias. The historical rate should therefore be interpreted alongside the assumptions that make borrowing defensible.
A useful diagnostic is to calculate the current-only mean, the historical rate, and the discounted mean together. Check that the discounted mean lies between the first two, then inspect whether the direction and size of the shift are clinically consequential. Also verify that historical successes and failures have not been interchanged, that \(n_h-y_h\) is used for historical failures, and that the chosen discount is applied to both outcome counts.
How This Comparison Differs from Alternatives
| Analysis | Historical contribution | What it answers |
|---|---|---|
| Current data only | None; equivalent to \(\delta=0\) | What does the posterior say without historical borrowing? |
| Fixed-discount power prior | Historical likelihood raised to a specified \(\delta\) | How does this chosen amount of borrowing shift the posterior? |
| Full pooling under the common-\(p\) model | Historical likelihood at full weight; equivalent to \(\delta=1\) | What follows if historical and current outcomes contribute equally per participant? |
Comparing these analyses at several fixed discounts is a sensitivity analysis: it shows how conclusions depend on the specified borrowing level. It does not choose the discount for the analyst, and it does not demonstrate that the common-probability assumption holds. A fixed-discount calculation is most informative when the range of discounts is justified and the resulting posterior changes are reported in context.
Common Mistake and Practical Limits
- Wrong reasoning: “The historical success rate is 70%, so the posterior mean must become 70%.” This ignores current information and the prior.
- Correction: The posterior mean is a weighted combination of the current-only mean and historical rate, with historical weight \(\delta n_h/(K+\delta n_h)\). Unless that weight is one, the posterior mean generally remains between the two rates.
- Wrong reasoning: “A discount of 0.5 means that exactly half of the historical patients are selected.”
- Correction: A fixed power-prior discount gives fractional likelihood contributions; it is not generally literal subsampling. In this beta-binomial model, it adds \(\delta y_h\) and \(\delta(n_h-y_h)\) to the beta shape parameters.
Finally, posterior influence is not a causal effect of historical outcomes on patients. It is a mathematical description of how the specified historical likelihood changes an estimate under the model. If the historical data are incompatible with the current setting, a numerical shift can be calculated correctly and still be scientifically misleading.
Check Your Understanding
For each question, use a \(\operatorname{Beta}(1,1)\) prior unless stated otherwise, and compare with the current-only posterior mean where requested.
- Current controls have 8 successes among 20 participants. Historical controls have 6 successes among 12 participants. Calculate the posterior mean at \(\delta=0.25\).
- Suppose the historical success rate equals the current-only posterior mean. What is the historical influence on the posterior mean at any fixed discount?
- With the same current data, historical controls have 12 successes among 20 participants. Calculate the posterior mean at \(\delta=0.5\).
- With the same current data, compare two historical datasets of 10 participants that differ by one success, at \(\delta=0.5\). By how much does changing one failure to a success change the posterior mean?
Solutions. (1) The current-only posterior has mean \(9/22\). The discounted mean is \((9+0.25(6))/(22+0.25(12))=10.5/25=0.4200\). (2) The identity \(m_\delta-m_0=\{\delta n_h/(K+\delta n_h)\}(p_h-m_0)\) gives zero because \(p_h=m_0\). (3) The posterior mean is \((9+0.5(12))/(22+0.5(20))=15/32=0.4688\). (4) The total posterior shape is \(22+0.5(10)=27\), so the change is \(0.5/27=0.01852\).