Choosing Between Common and Study-Specific Weights

Why the Weight Structure Matters

A power prior lets historical data contribute less than current trial data by raising each historical likelihood to a discount weight. The choice between one shared weight and separate study-specific weights determines how that discount is allocated. A common weight applies the same fractional discount to every historical study. Study-specific weights allow the analyst to discount studies differently, for example because their measurement quality or relevance to the current trial differs.

Neither choice is merely a computational convenience. It encodes an assumption about the historical evidence. A common weight treats all included studies alike in discount terms, while study-specific weights assert that their contributions should be adjusted differently. In either case, the weights do not by themselves make the historical studies identical to the current trial, nor do they model differences in treatment effect or baseline risk.

Definition: Let \(D_j\) be the data from historical study \(j\), with likelihood \(L_j(\theta)=p(D_j\mid\theta)\) for a parameter \(\theta\) shared across studies. A common fixed discount \(a\), with \(0\leq a\leq1\), gives a power prior proportional to \(\pi_0(\theta)\prod_{j=1}^J L_j(\theta)^a\). Study-specific fixed discounts \(a_1,\ldots,a_J\), each between 0 and 1, give a power prior proportional to \(\pi_0(\theta)\prod_{j=1}^J L_j(\theta)^{a_j}\). Here \(\pi_0(\theta)\) is the baseline prior. A weight of 0 excludes that study's likelihood contribution; a weight of 1 uses its full likelihood contribution.

These formulations presume that the likelihoods are conditionally independent across historical studies given the shared parameter, and that each likelihood is correctly specified for its study. They also presume that it is scientifically meaningful to let those studies inform the same \(\theta\). The shared-parameter assumption is substantive: different weights can reduce a study's influence, but do not replace a model for genuine between-study variation.

What the Weights Change

For fixed weights, taking logarithms shows the allocation directly. With a common discount, the historical contribution to the log likelihood is \(a\sum_j\log L_j(\theta)\). With study-specific discounts it is \(\sum_j a_j\log L_j(\theta)\). Thus \(a\) scales every study's log-likelihood contribution by the same factor, whereas \(a_j\) lets those factors differ. A common weight does not give each study equal influence: a large or highly informative study may still contribute much more than a small one.

The distinction is especially transparent for binary outcomes. Suppose study \(j\) observes \(y_j\) events among \(n_j\) participants, with a common event probability \(\theta\). Its binomial likelihood, omitting a factor that does not involve \(\theta\), is \(\theta^{y_j}(1-\theta)^{n_j-y_j}\). Raising this likelihood to \(a_j\) multiplies both its event and non-event exponents by \(a_j\).

\begin{aligned} \pi(\theta\mid D_1,\ldots,D_J,a_1,\ldots,a_J) &\propto \theta^{\alpha_0-1}(1-\theta)^{\beta_0-1} \prod_{j=1}^J \left[\theta^{y_j}(1-\theta)^{n_j-y_j}\right]^{a_j}\\ &\propto \theta^{\alpha_0-1+\sum_j a_jy_j} (1-\theta)^{\beta_0-1+\sum_j a_j(n_j-y_j)}. \end{aligned}

The last expression is the kernel of a beta distribution. Therefore, for a \(\text{Beta}(\alpha_0,\beta_0)\) baseline prior and fixed weights, the posterior is \(\text{Beta}\left(\alpha_0+\sum_j a_jy_j,\ \beta_0+\sum_j a_j(n_j-y_j)\right)\). For a common weight, set every \(a_j=a\). This derivation also shows why the weights can be interpreted as fractional likelihood contributions in this model.

Worked Example: One Discount or Two in a Binary Model

Consider two hypothetical historical studies of an event. Study 1 observes 18 events among 30 participants; Study 2 observes 4 events among 20 participants. Use a \(\text{Beta}(1,1)\) baseline prior. Compare a common discount of 0.5 with study-specific discounts \(a_1=1\) and \(a_2=0.25\). The shared event-probability model and binomial likelihood are assumed adequate for this illustration.

Common discount. The weighted event count is \(0.5(18)+0.5(4)=11\), and the weighted non-event count is \(0.5(12)+0.5(16)=14\). Adding the baseline prior parameters gives a posterior \(\text{Beta}(1+11,1+14)=\text{Beta}(12,15)\). Its mean is \(12/(12+15)=12/27=0.4444\).

Study-specific discounts. The weighted event count is \(1(18)+0.25(4)=19\), and the weighted non-event count is \(1(12)+0.25(16)=16\). The posterior is \(\text{Beta}(1+19,1+16)=\text{Beta}(20,17)\), with mean \(20/(20+17)=20/37=0.5405\).

The different posterior means arise because the study-specific choice retains all of Study 1's contribution but discounts Study 2 more heavily. Under each specified model, the posterior mean is the posterior average event probability; neither calculation establishes that the studies truly share one event probability. The contrast is a reason to justify the weights and examine sensitivity, not evidence that one weight choice is automatically correct.

Different Weights with Continuous Outcomes

The same allocation principle applies to continuous outcomes. For example, suppose study means \(y_j\) arise from normal sampling distributions with known common variance \(\sigma^2\), conditional on a shared mean \(\theta\). If study \(j\) has sample size \(n_j\), its likelihood contribution has precision \(n_j/\sigma^2\). Raising it to \(a_j\) multiplies that precision by \(a_j\). Under a normal baseline prior \(N(\mu_0,\tau_0^2)\), the resulting posterior precision and mean are

\begin{aligned} \frac{1}{\tau_{\text{post}}^2} &=\frac{1}{\tau_0^2}+\sum_j\frac{a_jn_j}{\sigma^2},\\ \mu_{\text{post}} &=\frac{\mu_0/\tau_0^2+\sum_j a_jn_jy_j/\sigma^2} {1/\tau_0^2+\sum_j a_jn_j/\sigma^2}. \end{aligned}

Worked Example: Posterior Mean with Continuous Study Results

Suppose two hypothetical historical studies estimate a common continuous endpoint, measured in units, with known within-study variance \(\sigma^2=16\). Study 1 has \(n_1=20\) and mean \(y_1=10\); Study 2 has \(n_2=5\) and mean \(y_2=14\). Use the baseline prior \(N(10,100)\). Compare common weights \(a_1=a_2=0.5\) with study-specific weights \(a_1=0.8,\ a_2=0.2\). These calculations assume independent normal sampling conditional on the shared mean and the stated known variance.

Common weights. The posterior precision is \(1/100+[0.5(20)+0.5(5)]/16=0.01+12.5/16=0.79125\). The precision-weighted numerator is \(10/100+[0.5(20)(10)+0.5(5)(14)]/16=0.1+135/16=8.5375\). Thus the posterior mean is \(8.5375/0.79125=10.79\) units (to three significant figures).

Study-specific weights. The posterior precision is \(1/100+[0.8(20)+0.2(5)]/16=0.01+17/16=1.0725\). The numerator is \(10/100+[0.8(20)(10)+0.2(5)(14)]/16=0.1+174/16=10.975\). The posterior mean is \(10.975/1.0725=10.23\) units (to three significant figures).

Study 1's mean is close to the prior mean, while Study 2's is higher. The second weighting scheme gives more influence to Study 1 and less to Study 2, so the posterior mean moves closer to 10. These are posterior means under the stated model, not estimates that account for possible between-study heterogeneity.

Equal Total Borrowing Does Not Mean Equal Allocation

A common weight can sometimes be selected to match the total nominal effective sample size of a study-specific weighting scheme. That match does not reproduce how the information is distributed across studies. In a binomial model, \(\sum_j a_j n_j\) is the total weighted sample size in the likelihood exponents. In other models, effective sample size needs a model-specific definition; a single total should not be treated as a universal measure of information.

Worked Example: Matching Total Effective Sample Size

Suppose hypothetical studies have sample sizes \(n_1=50\) and \(n_2=10\). An analyst proposes weights \(a_1=0.8\) and \(a_2=0.2\). The weighted sample sizes are \(0.8(50)=40\) and \(0.2(10)=2\), for a total of 42.

A common weight \(a\) that matches that total must satisfy \(a(50+10)=42\). Therefore \(a=42/60=0.7\). This common weight assigns effective sample sizes \(0.7(50)=35\) and \(0.7(10)=7\). Both schemes total 42, but the common scheme allocates five fewer effective participants to Study 1 and five more to Study 2. It cannot reproduce both study-specific contributions: doing so would require \(a=0.8\) for Study 1 and \(a=0.2\) for Study 2 at once.

The total-weight match is only an arithmetic comparison under this binomial-style effective-size interpretation. If the studies have different outcome variances or designs, equal totals need not imply equal posterior precision.

Choosing and Checking a Weight Structure

A common discount is useful when the same degree of discounting is defensible for all included studies, or when a simple sensitivity parameter is desired. It is parsimonious: one value controls the historical contribution as a whole. Study-specific discounts are useful when there are defensible study-level differences in reliability or relevance. They can express distinctions that a single common value necessarily obscures.

These choices should be connected to the design and data, rather than selected solely because one yields a preferred posterior. Relevant considerations include differences in outcome definitions, follow-up, eligibility, clinical practice, data completeness, and measurement quality. A separate weight can reduce the influence of a less relevant study, but it does not correct bias in that study, harmonize incompatible outcomes, or account for unmeasured differences between populations.

Several checks help reveal when a proposed weighting structure is not doing what was intended:

Inspect study-level contributions. Compare the weighted likelihood contributions, not just the numerical weights. The same weight can leave studies with very different influence because their sample sizes and outcome patterns differ.
Vary plausible weights. Recalculate the posterior over justified common-weight values or plausible study-specific combinations. Large changes in the posterior summary indicate material dependence on the borrowing assumptions.
Look for conflict. Study results that differ substantially may make the shared-parameter assumption questionable. Downweighting a conflicting study can change the result, but does not demonstrate that the remaining studies are exchangeable with the current trial.
Check the model's data requirements. Incorrect likelihoods, incompatible endpoint definitions, or dependence among studies undermine the calculation regardless of whether weights are common or study-specific.

Common pitfall: “The weights sum to the same total, so the two approaches borrow the same information.” This is generally wrong. Equal totals do not imply equal study-level contributions, and studies can differ in sample size, precision, and observed outcomes. Compare the weighted likelihood terms and resulting posterior summaries; do not infer equivalence from a total alone.

A further distinction concerns whether the discounts are fixed or estimated. The calculations above condition on fixed weights. If weights are treated as unknown, their probability model and the normalization of the power prior matter; the posterior must account for uncertainty in them. Choosing weights after inspecting the outcomes without accounting for that selection can also make the apparent borrowing more favorable than the analysis warrants. Study-specific weights offer flexibility, but more flexibility does not automatically mean more reliable inference.

Compared with a common power prior, a study-specific power prior can represent differences in study-level borrowing, at the cost of requiring more justification and potentially more sensitivity analysis. A model that assigns separate study parameters or explicitly models between-study variation is a different approach: it addresses heterogeneity directly rather than only scaling likelihood contributions. Neither approach is a substitute for checking whether the historical evidence is relevant to the target question.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In the beta-binomial model, what posterior distribution results from fixed study-specific weights \(a_j\)?
Two studies have sample sizes 30 and 10. If both receive a common weight of 0.4, what are their weighted sample sizes, and why are their contributions not equal?
For sample sizes 50 and 10, what common weight matches the total weighted sample size from weights 0.8 and 0.2? Does it match each study's contribution?
Does assigning different weights to historical studies model between-study heterogeneity? State the distinction.

Solutions. (1) With baseline prior \(\text{Beta}(\alpha_0,\beta_0)\), the posterior is \(\text{Beta}\left(\alpha_0+\sum_j a_jy_j,\ \beta_0+\sum_j a_j(n_j-y_j)\right)\). (2) The weighted sample sizes are \(0.4(30)=12\) and \(0.4(10)=4\); the common fraction is the same, but the larger study contributes more observations. (3) The target total is \(0.8(50)+0.2(10)=42\), so the matching common weight is \(42/(50+10)=0.7\). It gives contributions 35 and 7, not 40 and 2. (4) No. Different weights scale contributions to a shared parameter; a heterogeneity model represents variation in parameters across studies.