Historical Controls and Power Priors · Tutorial 342 of 696

Learning Discounts From Historical Data

Learn how to assign a prior to a power-prior discount, update it using current trial data, and recognize the assumptions and computational choices that govern the resulting borrowing.

Advanced 11 min read

What You'll Learn

Define a normalized power prior with an unknown discount parameter.
Derive how current trial data update the discount through a marginal likelihood.
Calculate conditional posterior distributions and compare borrowing across discount values.
Explain why normalization affects the intended prior distribution for the discount.
Identify assumptions, computational checks, and limitations of learning a discount.
Distinguish a learned discount from a fixed sensitivity-analysis value.

Why Treat the Discount as Unknown?

A fixed power-prior discount specifies in advance how much historical information contributes to the analysis. That can be useful for sensitivity analysis, but it leaves the choice of discount outside the model. Alternatively, a model can assign a prior distribution to the discount and let the current trial data update it. This approach represents uncertainty about how relevant the historical evidence is, but it does not make that uncertainty disappear: the resulting borrowing depends on the model, the prior for the discount, and the information in the current trial.

The key modeling question is not simply whether the historical and current response rates look similar. The discount governs how strongly the historical likelihood informs the parameter of interest. Once the discount is unknown, the analysis must specify a prior for it and account for the normalization of the historical-data contribution.

Definition: Let $D_0$ denote historical data, $D$ current trial data, $\theta$ a parameter shared by the two data sources, $L_0(\theta)=p(D_0\mid\theta)$ the historical likelihood, and $\pi_0(\theta)$ a baseline prior. A normalized power prior with discount $a\in[0,1]$ is $$ \pi(\theta\mid D_0,a)= \frac{L_0(\theta)^a\pi_0(\theta)} {C(a)},\qquad C(a)=\int L_0(\theta)^a\pi_0(\theta)\,d\theta. $$ If the discount is unknown, specify a prior density $\pi(a)$ on $[0,1]$, and use $\pi(\theta,a\mid D_0)=\pi(\theta\mid D_0,a)\pi(a)$.

The conventional interpretation is that $a=0$ discards the historical likelihood and $a=1$ uses it without discounting. Intermediate values temper the historical likelihood. These interpretations presume that the likelihood and baseline prior are properly specified and that the model uses the same $\theta$ to link historical and current outcomes. A discount is a modeling device; it is not itself a probability that the historical study is valid.

How Current Data Update the Discount

Suppose the current data have likelihood $L(\theta;D)$ and are conditionally independent of the historical data given $\theta$. Conditional on a particular $a$, current data update $\theta$ from the normalized power prior. To learn about $a$, integrate $\theta$ out of that update. The resulting marginal likelihood for the current data at a given discount is $$ m(D\mid a,D_0)=\int L(\theta;D)\pi(\theta\mid D_0,a)\,d\theta. $$ Consequently, the posterior density of the discount is $$ \pi(a\mid D,D_0)= \frac{m(D\mid a,D_0)\pi(a)} {\int_0^1 m(D\mid u,D_0)\pi(u)\,du}. $$

This formula makes the learning mechanism explicit. Current outcomes favor values of $a$ under which they are relatively probable after accounting for the historical data and the baseline prior. The historical data do not independently provide a second update to $a$ in this formulation: they enter through the conditional prior for $\theta$. The current likelihood supplies the evidence that changes the prior distribution of $a$.

Assumptions: The discount-learning calculation requires a specified historical likelihood, a baseline prior, and a proper prior for $a$. It also assumes that the current likelihood is correctly specified and that historical and current observations are conditionally independent given the shared parameter. If the studies differ in ways not represented by this model, a learned discount alone may not adequately describe those differences.

The normalization $C(a)$ is essential when $\pi(a)$ is intended to be the marginal prior distribution for the discount. Without it, the expression $L_0(\theta)^a\pi_0(\theta)\pi(a)$ generally does not have marginal distribution $\pi(a)$: integrating over $\theta$ multiplies $\pi(a)$ by $C(a)$. This is one reason to distinguish a normalized power prior from an unnormalized joint construction. The latter can be used as a model, but it implies a different prior for $a$ unless adjusted.

Worked Example: Updating the Discount With Binary Outcomes

Consider a hypothetical historical control group with 8 responses among 20 participants and a current control group with 4 responses among 10 participants. Let the response probability be $\theta$, use a baseline prior $\theta\sim\mathrm{Beta}(1,1)$, and place a uniform prior on $a\in[0,1]$. Assume independent binomial sampling with a common response probability. Find the ratio of posterior density at $a=1$ to posterior density at $a=0$.

For a given $a$, the historical contribution is proportional to $\theta^{8a}(1-\theta)^{12a}$. The binomial coefficient in the historical likelihood is constant in $\theta$ and cancels from the normalized conditional prior. Thus $$ \theta\mid D_0,a\sim\mathrm{Beta}(1+8a,1+12a). $$

The current-data marginal likelihood is beta-binomial: $$ m(D\mid a,D_0)= {10\choose4} \frac{B(1+8a+4,\,1+12a+6)} {B(1+8a,\,1+12a)}. $$ At $a=0$, this is the predictive probability of 4 responses in 10 trials under a uniform prior: $$ m(D\mid 0,D_0)={10\choose4}B(5,7)=\frac{1}{11}=0.09091. $$ At $a=1$, it is $$ m(D\mid 1,D_0)={10\choose4}\frac{B(13,19)}{B(9,13)} =\frac{210(9)_4(13)_6}{(22)_{10}} \approx 0.2072, $$ where $(z)_k=z(z+1)\cdots(z+k-1)$. Since the prior density is uniform, the posterior density ratio is $$ \frac{\pi(1\mid D,D_0)}{\pi(0\mid D,D_0)} =\frac{0.2072}{0.09091}\approx2.279. $$

The current data are about 2.28 times as probable under full borrowing as under no borrowing, so the posterior density is higher at $a=1$ than at $a=0$. This is a density comparison, not a posterior probability that $a$ equals either endpoint; under a continuous prior, each exact endpoint has probability zero. Nor does this endpoint comparison alone determine the posterior mean or the probability of a broad interval of discounts.

Conditional Posterior and Borrowing

For the same binary-outcome model, the conditional posterior for $\theta$ after observing current data is also available in closed form. If the current study has $y$ responses among $n$ participants, then $$ \theta\mid D,D_0,a\sim \mathrm{Beta}(1+8a+y,\,1+12a+n-y). $$ The discount therefore controls how much the historical response and nonresponse counts contribute to the posterior parameters. With a random discount, the posterior for $\theta$ is a mixture of these conditional beta distributions, weighted by $\pi(a\mid D,D_0)$. It is generally not itself a single beta distribution.

Worked Example: Conditional Posterior at an Intermediate Discount

Use the hypothetical counts from the previous example and examine $a=0.5$. This calculation is conditional on that discount; it does not claim that $a=0.5$ is the learned value.

The historical contribution is half-weighted, giving 4 effective historical responses and 6 effective historical nonresponses. After adding the current 4 responses and 6 nonresponses, the posterior is $$ \theta\mid D,D_0,a=0.5\sim \mathrm{Beta}(1+4+4,\,1+6+6)=\mathrm{Beta}(9,13). $$ Its posterior mean is $$ E(\theta\mid D,D_0,a=0.5)=\frac{9}{9+13} =\frac{9}{22}=0.4091. $$ Thus, conditional on $a=0.5$, the posterior mean response probability is 0.4091. The unit is a probability, so it has no physical units. This conditional calculation is not the overall posterior mean under a prior on $a$; that mean averages over the posterior distribution of $a$.

In computation, one can evaluate $m(D\mid a,D_0)$ over a grid or sample $a$ jointly with $\theta$. For a grid approximation, calculate the marginal likelihood at each grid value, multiply by the prior density, and normalize the resulting weights using numerical integration. For continuous sampling methods, evaluate the normalized conditional prior or its log density accurately. In either case, inspect whether the posterior for $a$ is concentrated, diffuse, or sensitive to the prior.

Worked Example: Why Normalization Changes the Discount Prior

For the hypothetical historical data of 8 responses among 20 participants, the full binomial likelihood is $$ L_0(\theta)={20\choose8}\theta^8(1-\theta)^{12}. $$ With the uniform baseline prior, the normalization at $a=0$ is $$ C(0)=\int_0^1 1\,d\theta=1. $$ At $a=1$, it is the prior predictive probability of the historical count: $$ C(1)=\int_0^1 {20\choose8}\theta^8(1-\theta)^{12}\,d\theta ={20\choose8}B(9,13)=\frac{1}{21}\approx0.04762. $$

If the joint expression were left unnormalized in $\theta$, integrating over $\theta$ would multiply the nominal discount density by $C(a)$. At these endpoints, the multiplier would be 1 at $a=0$ but only 0.04762 at $a=1$. Thus a nominally uniform $\pi(a)$ would not remain uniform as the marginal distribution of $a$. Dividing by $C(a)$ makes the conditional prior for $\theta$ integrate to 1 at each $a$, so that the specified $\pi(a)$ is indeed the marginal prior for the discount before current data are observed.

Diagnostics, Alternatives, and Limitations

A posterior for $a$ is meaningful only relative to the modeling choices that produced it. In particular, check that the outcome definition and sampling model are appropriate in both data sources, that the shared parameter is scientifically defensible, and that the baseline prior and the prior for $a$ are proper. In numerical work, verify the normalization and compare computations using finer grids, alternative quadrature settings, or independent sampling runs. A result that changes materially with numerical resolution is not ready for interpretation.

A broad posterior for $a$ is not necessarily a computational failure. The current trial may simply contain too little information to distinguish discounts. Conversely, a posterior concentrated near an endpoint can reflect strong data compatibility under the assumed model, but it does not prove that the studies are exchangeable or that the historical data are unbiased. Prior sensitivity is especially important when the current trial is small: repeat the analysis under plausible alternative priors for $a$, and report how the posterior for $\theta$ changes.

A fixed discount and a random discount answer different questions. A fixed value produces a conditional analysis at a chosen borrowing level and is useful for transparent sensitivity analysis. A prior on $a$ propagates uncertainty about the discount into the posterior, but requires a prior specification and marginal-likelihood computation. No borrowing is the boundary case $a=0$; full borrowing is the boundary case $a=1$. Comparing these cases with the random-discount result can reveal whether the conclusion depends strongly on the borrowing assumption.

There are important limitations. The discount is often weakly identified because a discrepancy between historical and current outcomes can arise from sampling variation, genuine population change, or model misspecification. A single scalar $a$ cannot diagnose which explanation is responsible. When the posterior is sensitive to $\pi(a)$, that sensitivity should be reported rather than hidden. Also, a high posterior density for a discount does not establish a causal or substantive explanation for the agreement between studies.

Common pitfall: “The posterior for $a$ is near 1, so the historical study has been shown to be exchangeable with the current trial.” This overstates what the model establishes. The posterior says that, under the specified likelihoods, baseline prior, and discount prior, current data support stronger historical weighting. It cannot by itself verify exchangeability or rule out shared sources of bias.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

What role does $C(a)$ play in a normalized power prior?
With a uniform prior for $a$, what does a posterior density ratio of 3 at $a=1$ versus $a=0$ mean?
For historical data with 6 responses and 14 nonresponses, a $\mathrm{Beta}(1,1)$ baseline prior, and current data with 3 responses and 7 nonresponses, write the conditional posterior for $\theta$ at $a=0.5$.
Why can a posterior for $a$ remain diffuse even when both studies are well conducted?

Solutions. (1) $C(a)$ makes the conditional prior for $\theta$ integrate to 1 at each discount, preserving the specified marginal prior $\pi(a)$. (2) The posterior density at full borrowing is three times that at no borrowing; this is not a probability that $a=1$, nor a probability that the historical study is valid. (3) Half-weighting the historical counts gives 3 responses and 7 nonresponses. Adding current counts and the baseline prior gives $\theta\mid D,D_0,a=0.5\sim\mathrm{Beta}(1+3+3,1+7+7)=\mathrm{Beta}(7,15)$. (4) The current data may not contain enough information to distinguish sampling variability from differences in the underlying response probability, so multiple discounts can remain compatible with the observations.

Back to Tutorials