Why Borrowing Reports Need More Than a Posterior Estimate
When a clinical-trial analysis borrows information from historical data, a posterior estimate alone does not show how much those data mattered. A reader needs to know what the borrowing model allowed, how its prior settings were chosen, whether the current and historical data showed conflict, and whether the scientific conclusion changes under reasonable alternatives. Reporting these elements makes the analysis assessable without implying that a diagnostic can certify a model as correct.
The examples below use a hypothetical continuous endpoint. Let \(\theta_C\) denote the current-trial mean treatment effect, and let \(y_C\) be its estimate with known sampling variance \(V_C\). Let \(\theta_H\) denote a historical treatment effect. The effect is measured in points on the endpoint scale; positive values favor treatment. These symbols are defined here for the examples, not as a substitute for specifying the actual estimand, population, treatment contrast, and endpoint in an analysis report.
Describe the Borrowing Model and Its Prior
For a robust mixture prior, a common form is a weighted mixture of an informative historical component and a broader component:
Here \(\pi\) is the prior probability of the informative component, \(v_I\) and \(v_R\) are component variances, and \(N(m,v)\) denotes a normal distribution with mean \(m\) and variance \(v\). The broader component is not automatically a no-borrowing component: it is still centered at \(\theta_H\), and its finite variance can affect the posterior. A report should give the component distributions and explain the scientific and computational rationale for their scales.
For a commensurate precision model, a typical conditional specification is
The precision \(\tau\) controls how tightly the current effect is centered on the historical effect: larger \(\tau\) means a smaller conditional variance and stronger potential borrowing. If \(\theta_C\) is measured in endpoint points, \(\tau\) is measured in inverse squared points. Reporting “a gamma prior on precision” is incomplete: give the gamma parameterization (including whether the second argument is a rate or scale), hyperparameters, units, and a useful summary such as prior quantiles. Explain how those choices were calibrated. For a robust model, also describe any component intended to protect against disagreement.
Worked Example: Reporting a Mixture Prior and Its Posterior Weight
Suppose the hypothetical historical mean is \(\theta_H=10\) points. The current estimate is \(y_C=14\) points, with \(y_C\mid\theta_C\sim N(\theta_C,1)\). Use the prior \(\theta_C\sim 0.5N(10,0.25)+0.5N(10,100)\). Assume the historical estimate is treated as fixed for this calculation and the current likelihood is normal with the stated known variance.
The current estimate has marginal distribution \(N(10,1.25)\) under the informative component and \(N(10,101)\) under the broad component. The Bayes factor for the informative component against the broad component is therefore the ratio of these marginal densities at 14:
With equal prior component probabilities, posterior odds equal the Bayes factor. Thus the posterior probability of the informative component is \(0.01617/(1+0.01617)\approx0.01591\), or about 1.6%. The component-specific posteriors are \(N(10.8,0.2)\) and \(N(13.9604,0.9901)\): for each component, these follow by multiplying the normal prior density by the \(N(\theta_C,1)\) likelihood. The mixture posterior mean is approximately \(0.01591(10.8)+0.98409(13.9604)=13.910\) points.
The report would give the component definitions, the prior weight, and this posterior component probability. It would not describe 1.6% as “the percentage of observations borrowed.” That is not what a mixture probability measures. The broad component also remains centered on the historical mean and therefore does not guarantee zero historical influence.
Summarize Borrowing Without Hiding Its Meaning
There is no single borrowing summary that applies to every model. In a mixture model, report posterior component probabilities and, where helpful, component-specific posterior estimates. In a commensurate model, report the prior and posterior distributions of \(\tau\), with interpretable quantiles, as well as summaries of the resulting current-effect posterior. If a conditional shrinkage weight is used, define precisely what it conditions on.
For example, suppose \(y_C\mid\theta_C\sim N(\theta_C,V_C)\) and \(\theta_H\) is treated as fixed. Under the commensurate normal model, the posterior mean is a precision-weighted average:
The conditional weight on \(\theta_H\) is \(w=\tau V_C/(1+\tau V_C)\). It is a model-specific shrinkage weight, not a universal effective sample size. When \(\tau\), \(\theta_H\), or other model parameters are uncertain, a single plug-in weight can obscure posterior uncertainty. Report the assumptions behind it and, where possible, summarize its posterior distribution rather than only evaluating it at a point estimate.
Worked Example: Interpreting a Precision Prior and Posterior
Suppose a hypothetical trial uses \(V_C=0.25\) points squared and the commensurate model \(\theta_C\mid\theta_H,\tau\sim N(\theta_H,1/\tau)\). The analysis specifies \(\tau\sim\operatorname{Gamma}(2,1)\), using shape-rate notation, so the prior mean precision is 2 inverse points squared. Assume hypothetical posterior draws have 10th, 50th, and 90th percentiles of \(\tau\) equal to 0.3, 2, and 8 inverse points squared.
Conditional on \(\tau\) and treating \(\theta_H\) as fixed, the shrinkage weight is \(w=\tau V_C/(1+\tau V_C)\). At the three reported precision percentiles, the corresponding weights are \(0.3(0.25)/(1+0.3(0.25))\approx0.0698\), \(2(0.25)/(1+2(0.25))=0.3333\), and \(8(0.25)/(1+8(0.25))=0.6667\). The posterior median conditional weight is therefore one third, and the weight varies substantially across the stated posterior range.
These calculations do not establish that one third of the current data came from the historical trial. They describe conditional shrinkage for this normal model and this current-data variance. A report should include the precision parameterization and units, the posterior precision summaries, and enough context to interpret the induced shrinkage.
Report Conflict Diagnostics With Their Assumptions
A conflict diagnostic quantifies a discrepancy under a stated reference model. It does not, by itself, determine whether borrowing should be retained. For a simple diagnostic, suppose the historical and current estimates are independent, unbiased normal estimates of the same effect under no temporal or population shift, with known standard errors \(SE_H\) and \(SE_C\). Then their difference has standard error \(\sqrt{SE_H^2+SE_C^2}\), and the standardized discrepancy is
A large absolute value indicates that the estimates are far apart relative to the assumed sampling variation. The assumptions matter: overlap between patients, different covariate distributions, changed outcome definitions, or different treatment implementations can make a simple comparison misleading. A discrepancy may reflect sampling variability, a population shift, outliers, or unmodeled design differences. Investigate these possibilities and report the diagnostic’s reference model and limitations.
Worked Example: A Standardized Historical-Current Discrepancy
In a hypothetical comparison, suppose the historical estimate is \(y_H=10\) points with \(SE_H=1\) point, while the independent current estimate is \(y_C=14\) points with \(SE_C=1\) point. Assume both estimates are unbiased normal estimates of the same treatment effect under the no-shift reference model.
Under the stated reference assumptions, the two-sided normal tail probability is \(2\{1-\Phi(2.828)\}\approx0.00468\). This indicates a discrepancy that would be unusual if the estimates measured the same effect and differed only through the specified sampling error. It is not the probability that the historical and current effects are equal, nor is it a rule that automatically rejects borrowing. The report should describe the assumed common-effect reference, the independence and standard-error assumptions, and relevant design or covariate differences to investigate.
Show Sensitivity of the Trial Conclusion
Sensitivity analysis compares the target posterior conclusions under a primary borrowing model and plausible alternatives. Useful alternatives can include no historical borrowing, a robust mixture with different prespecified component scales or weights, and a commensurate model with a different defensible precision prior. State which parts changed and which remained fixed. Compare quantities that matter to the trial decision, such as an effect estimate and interval, a posterior probability relative to a clinical threshold, or a predictive quantity. Reporting only a change in posterior mean may hide a meaningful change in uncertainty.
Worked Example: Comparing Conclusions Across Borrowing Choices
Return to the hypothetical normal model with \(y_C=14\), sampling variance 1, and mixture components \(N(10,0.25)\) and \(N(10,100)\). Suppose the clinical threshold is \(\theta_C>12\) points. Within the informative component, the posterior probability of exceeding 12 is approximately \(0.0036\); within the broad component it is approximately \(0.9756\), using their component posterior distributions from the earlier calculation.
With prior component weight \(\pi=0.5\), the posterior informative-component probability is about \(0.01591\). The overall posterior probability is therefore \(0.01591(0.0036)+0.98409(0.9756)\approx0.9601\). If the prior weight changes to \(\pi=0.95\), the posterior component odds are \(19(0.01617)\approx0.3072\), giving posterior informative-component probability \(0.3072/(1+0.3072)\approx0.2350\). The corresponding posterior probability of exceeding 12 is approximately \(0.2350(0.0036)+0.7650(0.9756)=0.7472\).
With no historical borrowing and a flat prior for \(\theta_C\), the posterior is \(N(14,1)\), so \(P(\theta_C>12\mid y_C)=1-\Phi((12-14)/1)\approx0.9772\). The threshold conclusion is sensitive to the mixture weight: the estimated probability is about 96.0% for \(\pi=0.5\), 74.7% for \(\pi=0.95\), and 97.7% with no borrowing. These are hypothetical calculations, not evidence about a real treatment. A report should present such differences transparently and explain which prior settings were scientifically defensible before seeing the current outcome.
Diagnostics, Alternatives, and Common Pitfalls
A useful report makes it possible to check both computation and modeling. For simulation-based fits, report the convergence and mixing checks appropriate to the method, along with effective sample sizes or Monte Carlo uncertainty for key posterior summaries. In mixture models, check whether sampling adequately explores the components and whether the reported component probabilities are stable. In commensurate models, inspect the posterior for precision parameters and whether it is driven by the likelihood or remains sensitive to the prior. A posterior precision concentrated at a boundary, poor chain mixing, or large changes under small prior adjustments calls for caution, not a selectively reported point estimate.
Conflict summaries and sensitivity analyses answer related but distinct questions. A discrepancy diagnostic evaluates data against a reference assumption; sensitivity analysis shows how conclusions change under alternative modeling choices. Neither is a mechanical model-selection procedure. Alternatives such as no borrowing, robust mixture borrowing, and commensurate borrowing encode different assumptions and can be compared, but no method removes the need to justify the historical data’s relevance to the current estimand.
- Pitfall: calling the broad component “no borrowing.” Its finite variance and shared center can still influence the posterior. State its distribution and quantify the resulting sensitivity.
- Pitfall: reporting a precision prior without its parameterization. A gamma shape-rate prior is not interchangeable with a shape-scale prior. Give the convention, hyperparameters, units, and prior implications.
- Pitfall: treating a small conflict tail probability as proof of incompatibility. It is conditional on the diagnostic’s reference assumptions; it is not the probability that the model is false.
- Pitfall: reporting only the primary fit. Without alternatives, readers cannot assess how dependent the conclusion is on historical borrowing.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Why is a broad but finite mixture component not necessarily equivalent to no borrowing?
- For \(V_C=0.5\) and \(\tau=2\), calculate the conditional historical shrinkage weight \(w\).
- If independent historical and current estimates differ by 3 points and each has standard error 1 point, calculate their standardized discrepancy.
- What must a report specify to make a gamma precision prior interpretable?
Solutions. (1) A finite-variance component centered on the historical effect still pulls the current effect toward that center; a broad component may reduce, but does not generally eliminate, that influence. (2) \(w=\tau V_C/(1+\tau V_C)=2(0.5)/(1+2(0.5))=1/2=0.5\). Under the fixed-\(\theta_H\) normal model, the conditional posterior mean assigns half its weight to the historical mean. (3) \(z=3/\sqrt{1^2+1^2}=3/\sqrt{2}\approx2.121\). This is a standardized discrepancy under the independent, unbiased, normal common-effect reference assumptions, not a probability that the effects are equal. (4) Specify the gamma parameterization, including shape and rate or scale, the hyperparameters, the precision units, and a useful summary of the prior’s implications for borrowing.