Eliciting a Power Prior Discount

Why Elicitation Requires More Than a Similarity Score

A historical control dataset may be relevant to a new trial without being fully comparable to its current control group. The control outcome could have been measured under an earlier standard of care, in a somewhat different patient population, or with a different follow-up process. A power prior provides a way to discount the historical likelihood, but it does not determine the discount from those clinical facts. The discount must be justified through an explicit judgment about how much the historical evidence should contribute under the assumptions of the analysis.

The purpose of elicitation is not to manufacture a single objectively correct number. It is to make the reasoning traceable: identify relevant differences, state how they affect the analysis, and select a defensible candidate or range of candidates for evaluation. A score that appears precise but has no stated connection to the likelihood is not a substitute for this reasoning.

The endpoints are useful anchors: \(a=0\) removes the historical likelihood contribution, and \(a=1\) uses it without discounting. Values between them attenuate that contribution. These anchors do not tell an analyst what value to use. Nor does a clinical rating such as “moderately similar” have a universal numerical conversion to \(a\). A conversion must be agreed in advance for the particular analysis and explained in terms of what the judgment means.

Before elicitation, specify the estimand and the outcome model. For instance, if \(\theta\) is a control event probability, the historical data must concern the same defined event, population target, time window, and treatment condition that the model uses. The sampling model must be adequate for the historical observations, and the parameter must have a meaningful common interpretation across the historical and current settings. Discounting addresses uncertainty about how much historical likelihood to use; it does not correct a misspecified model or remove systematic bias.

Questions That Make the Judgment Explicit

Elicitation is most useful when it focuses on specific sources of relevance and incompatibility, rather than asking an expert for a discount immediately. The questions below can be answered jointly by clinical, operational, and statistical experts. The answers should be recorded alongside the selected values.

• Population: How closely do eligibility criteria, disease severity, baseline risk, and care setting match the target population? Which differences are expected to change the control outcome?
• Intervention and care context: Was background or supportive care comparable? Have changes in standard practice plausibly altered the outcome during the historical period?
• Outcome definition: Do the event definition, assessment method, follow-up window, and ascertainment intensity align? Could different measurement create a systematic difference in observed rates?
• Data quality: Are missingness, adherence, and follow-up processes sufficiently understood? Is the historical estimate vulnerable to selection or reporting bias?
• Model and target: Does the same parameter describe the outcome in both settings, and is the intended trial estimand compatible with the historical data?
• Decision anchor: What would justify no borrowing, partial borrowing, or nearly full borrowing? What evidence would cause the proposed value to be reduced or set to zero?

The answers are not interchangeable. For example, a difference in the calendar period may be acceptable if care remained stable, while a changed event definition may make the historical rate uninformative for the current endpoint. Experts should state the direction and plausible size of each concern where possible. If a mismatch can be addressed by a prespecified model that genuinely aligns the data, document that assumption; do not assume that a lower discount alone resolves it.

From Judgments to Candidate Values

A practical elicitation plan uses a small number of clearly described scenarios. One scenario might represent strong comparability, another the most plausible case, and a third substantial but not disqualifying mismatch. For each scenario, the team states a candidate \(a\), explains the evidence behind it, and identifies what assumptions would need to hold. It is usually more informative to evaluate a reasoned range than to report one value with unjustified precision.

A protocol may use a structured scoring rule to support consistency across experts. If so, its dimensions, weights, and conversion to candidate discounts must be specified in advance and treated as a decision aid—not as a statistical law. The team should also ask experts to review the resulting candidates in likelihood terms: does the proposed contribution match the degree of reliance they intended? That check can reveal a mismatch between a qualitative label and its numerical implementation.

Justifying and Documenting the Candidates

A candidate discount should be supported by a chain of reasoning, not merely a label. The chain starts with the target estimand and historical sampling model, states which features match and differ, explains how each material difference affects confidence in using the historical likelihood, and ends with a prespecified candidate or range. The justification should also identify the evidence that would change the judgment, such as a verified change in standard care or a reanalysis using a harmonized endpoint.

A useful record includes the historical data source and outcome definition; the target population and estimand; the model assumptions; the elicitation participants and questions; the candidate values and their rationales; and the planned analyses across the selected values. Keeping the rationale separate from the numerical value makes later review possible. It also makes clear whether a changed result reflects new evidence, a revised assumption, or a different decision about borrowing.

The elicitation should be completed without choosing a value simply because it gives a preferred trial conclusion. If the rationale depends on the observed treatment effect, that dependence needs careful scrutiny: the discount is intended to reflect historical-to-current relevance, not to tune the analysis to a desired result. Independent review and prespecification help limit this risk.

Diagnostics, Alternatives, and Limitations

Several warning signs suggest that the elicitation needs revision. Experts may use the same word—such as “similar”—to mean different things; a rubric may obscure a serious endpoint mismatch by averaging it with several favorable ratings; or the candidate may be reported to two decimal places when the evidence supports only a broad range. Another warning sign is an assumption that a low discount fixes bias. A power discount scales the historical likelihood; it does not remove selection bias, repair inaccurate measurement, or make different populations identical.

A further check is to ask whether the candidate scenarios remain credible when each major clinical assumption is challenged. If all plausible scenarios depend on the historical and current outcomes sharing a parameter interpretation, but that condition is doubtful, model revision or exclusion may be more appropriate than discount elicitation. Similarly, an unresolved material data-quality problem should be described as a limitation, not hidden inside a numerical choice.

The main alternative is to use no historical borrowing, which avoids relying on historical outcomes but may discard relevant information. Full borrowing is a useful benchmark when strong comparability is defensible, but it is not a default merely because the endpoint names match. Other approaches, such as robust mixture priors or a model that explicitly accounts for relevant differences, may be considered when their assumptions fit the problem. They are not interchangeable with a power prior, and their use requires a separate justification.

Finally, expert elicitation does not eliminate uncertainty about relevance. A fixed candidate is conditional on the assumptions and judgments recorded at the time it is chosen. Reporting a central value together with lower and upper plausible values makes that dependence visible and supports a transparent sensitivity analysis. The rationale, not the apparent precision of the chosen number, is what makes the choice reviewable.