Why Combine Expert Priors?
In a clinical trial, several experts may judge the plausible range of the same treatment effect differently. One clinician may emphasize a promising biological mechanism; another may give more weight to previous trials. Combining their judgments can provide a single prior distribution for analysis, but the result depends on the pooling rule. A rule can preserve distinct expert viewpoints as a mixture or construct a distribution that emphasizes their shared support. Neither operation is neutral: each changes how the combined prior represents the center, uncertainty, and disagreement among experts.
The experts must be judging the same estimand, on the same scale, and with the same relevant conditioning information. For example, judgments about a log hazard ratio cannot be pooled directly with judgments about a risk difference. Before pooling, each expert’s elicited beliefs must be represented as a proper probability distribution for the common parameter. The weights should be chosen and justified as part of the elicitation; they are not automatically determined by how many patients an expert has treated or by the sample size of a study the expert recalls.
A zero weight excludes an expert from the pool. For a linear pool, the combined distribution is a weighted mixture: a draw from it can be understood as first selecting an expert according to the weights, then drawing from that expert’s prior. For a logarithmic pool, the densities are combined multiplicatively and then normalized. Where an expert assigns zero density, a positive-weight logarithmic pool also assigns zero density. Thus the logarithmic pool is supported only where all positively weighted experts have positive density, whereas a linear pool can retain support supplied by any one expert.
Two Ways to Represent Probability Judgments
In a linear pool, each expert contributes a share of probability mass. If expert \(j\) assigns probability \(q_j\) to an event \(A\), the combined probability of that event is \(\sum_j w_jq_j\). The linear pool therefore averages event probabilities directly. For continuous parameters, disagreement among expert centers contributes to the spread of the mixture, in addition to the uncertainty within each expert’s distribution.
A logarithmic pool instead averages log densities. The normalizing constant matters: the weighted geometric mean of the densities will not generally integrate to one. The log pool is not simply the arithmetic average of expert distributions, and it does not usually preserve each expert’s probability for an event by averaging those probabilities. It emphasizes parameter values that receive support from the contributing experts, and can produce a relatively concentrated combined prior.
Worked Example: Pooling Beliefs About a Clinically Meaningful Benefit
Suppose two experts judge whether a treatment has a clinically meaningful benefit. Expert 1 assigns probability 0.80 to benefit and 0.20 to no benefit. Expert 2 assigns probabilities 0.40 and 0.60, respectively. Give each expert weight 0.50. Treat the two outcomes as the possible states of a discrete parameter.
Linear pool. The probability of benefit is the weighted average:
The probability of no benefit is \(0.50(0.20)+0.50(0.60)=0.40\). These probabilities sum to one, as required.
Logarithmic pool. The unnormalized weights on the two states are the weighted geometric means of the experts’ probabilities. For benefit this is \(\sqrt{0.80(0.40)}=\sqrt{0.32}\approx0.5657\); for no benefit it is \(\sqrt{0.20(0.60)}=\sqrt{0.12}\approx0.3464\). The normalizing constant is \(0.5657+0.3464=0.9121\), so
Both rules give benefit a higher pooled probability than no benefit, but the numerical results differ. Here the logarithmic pool assigns about 0.620 probability to benefit, compared with 0.600 under linear pooling. These are combined prior probabilities, not probabilities calculated from trial outcomes.
How the Rules Change the Center and Uncertainty
For a linear pool, the mean and variance have a simple interpretation. Write \(m_j\) and \(v_j\) for the mean and variance of expert \(j\)’s distribution, assuming these moments exist. The pooled mean is the weighted mean of the expert means. The pooled variance contains both each expert’s within-distribution variance and the squared distance from that expert’s mean to the pooled mean.
The second term in the variance formula is the contribution of disagreement in centers. Consequently, a linear pool can be more dispersed than any individual expert prior, even when the experts are individually quite certain. The mixture may also be multimodal if experts concentrate their beliefs in different regions.
Worked Example: Pooling Normal Priors for a Log Hazard Ratio
Two experts give normal prior distributions for a trial’s log hazard ratio \(\theta\). Expert 1 gives \(N(-0.10,0.04)\), and Expert 2 gives \(N(-0.30,0.09)\), where the second parameter is the variance. Use equal weights. Both judgments concern the same log hazard ratio, so the parameter and scale align. Assume the elicited normal priors are proper and that the two equal weights are the intended pooling choice.
Linear pool. The mean is \(m_L=0.50(-0.10)+0.50(-0.30)=-0.20\). The variance is the weighted average of each variance plus the squared difference from \(-0.20\):
Thus the mixture has mean \(-0.200\) and variance \(0.075\), or standard deviation \(\sqrt{0.075}\approx0.2739\) on the log-hazard-ratio scale. The variance includes the experts’ disagreement about the center.
Logarithmic pool. When all experts’ priors are normal, the log pool is also normal. Its precision (the reciprocal of its variance) is the weighted sum of expert precisions, and its mean is the precision-weighted average of their means. Here,
The logarithmic pool has mean \(-0.1615\) and standard deviation \(\sqrt{0.05538}\approx0.2353\), both on the log-hazard-ratio scale. It is more influenced by Expert 1’s mean because Expert 1 supplied the more precise distribution. The smaller log-pool variance does not include a separate between-expert disagreement term.
Why the Normal Log-Pool Formula Holds
For expert \(j\), a normal density with mean \(m_j\) and variance \(v_j\) has log density, up to a constant that does not depend on \(\theta\), equal to \(-(\theta-m_j)^2/(2v_j)\). The log of the unnormalized logarithmic pool is therefore, up to a constant,
Define \(A=\sum_j w_j/v_j\) and \(B=\sum_j w_jm_j/v_j\). Expanding the squares gives \(-\tfrac12(A\theta^2-2B\theta+C)\), where \(C=\sum_j w_jm_j^2/v_j\) does not depend on \(\theta\). Completing the square,
The terms independent of \(\theta\) are absorbed into the normalizing constant. The remaining kernel is a normal density with mean \(B/A\) and variance \(1/A\), giving the formulas used above. This calculation also shows why a precise expert can have greater influence on the log-pool center even when the nominal weights are equal: that expert contributes more precision.
Worked Example: When Experts Disagree Strongly
Consider two equally weighted expert priors for a log treatment effect: \(N(-0.20,0.01)\) and \(N(0.20,0.01)\). Both are proper normal distributions on the same parameter scale, and equal weights are assumed.
The linear pool has mean \(0.50(-0.20)+0.50(0.20)=0\). Its variance is
It retains probability mass around both experts’ centers. In fact, the mixture density at either expert mean is proportional to \(0.5(1+\exp(-8))\approx0.5002\), while its density at zero is proportional to \(\exp(-2)\approx0.1353\). Thus zero is not a density maximum: the mixture has two peaks, reflecting the distinct judgments.
For the log pool, the equal variances give mean \([0.50(-0.20)/0.01+0.50(0.20)/0.01]/[0.50/0.01+0.50/0.01]=0\) and variance \((0.50/0.01+0.50/0.01)^{-1}=0.01\). The log pool is a single normal distribution centered between the experts, with standard deviation 0.10. It does not preserve two separate peaks. In this example, the two rules tell meaningfully different stories about uncertainty: a linear pool retains the two opposing views, while a logarithmic pool represents a concentrated compromise.
Choosing and Checking a Pooling Rule
A pooling rule should match the intended interpretation of the combined prior. Use a linear pool when the combined prior should preserve expert-specific possibilities and represent disagreement as additional uncertainty. Use a logarithmic pool when the aim is to combine support multiplicatively, favoring parameter values that receive support from all positively weighted experts. These are not interchangeable ways to calculate the same answer.
Before using either result in a trial analysis, check that all experts addressed the same estimand and relevant information, that each prior is proper, and that the weights are nonnegative and sum to one. Inspect the individual priors and the pooled result, not just their means. For the linear pool, look for distinct modes and tails contributed by particular experts. For the log pool, check whether the intersection of expert supports is scientifically plausible and whether the combined distribution has become too concentrated. In particular, an expert assigning zero probability or density to a plausible region can eliminate that region entirely from a positive-weight log pool.
Weights express the chosen influence of each expert; they are not a correction for dependence between experts and do not turn elicited judgments into independent data. A sensitivity check using reasonable alternative weights can show whether the pooled prior is driven by one judgment. If disagreement is itself important to retain, the linear mixture makes that feature explicit; a single pooled distribution may be less informative than keeping expert-specific priors visible for discussion.
A common mistake is to assume that averaging prior means also averages uncertainty. Under linear pooling, the variance includes between-expert disagreement, as the formula shows. Under logarithmic pooling, disagreement does not enter through that same variance term, and the pool can be much narrower. Another mistake is to treat equal weights as universally fair. Equal weights are a substantive choice, not a default justified by the mathematics. State what the weights mean and check how the result changes under plausible alternatives.
A hierarchical model that represents expert-specific judgments is another possible approach when the experts’ differences are to be modeled rather than collapsed into one distribution. That approach requires additional modeling choices. It is not the same operation as either linear or logarithmic pooling.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Two experts assign probabilities 0.20 and 0.80 to a clinically meaningful benefit. With weights 0.25 and 0.75, what is the linear-pool probability of benefit?
- Two normal expert priors have means 1 and 3 and both have variance 0.16. With equal weights, calculate the log-pool mean and variance, then the linear-pool variance.
- One positively weighted expert assigns zero probability to an event. Can the log pool assign positive probability to it? Can the linear pool?
- Why can the linear pool be wider than the individual priors even when each expert’s prior is narrow?
Solutions. 1. \(0.25(0.20)+0.75(0.80)=0.65\). 2. Equal variances yield log-pool mean \(0.50(1)+0.50(3)=2\) and variance \((0.50/0.16+0.50/0.16)^{-1}=0.16\). The linear-pool variance is \(0.50[0.16+(1-2)^2]+0.50[0.16+(3-2)^2]=1.16\). 3. The log pool assigns zero probability because the product includes a zero factor; the linear pool can assign positive probability if another positively weighted expert assigns positive probability. 4. Its variance includes the weighted squared distances between individual expert means and the pooled mean, so disagreement adds to within-expert uncertainty.