From a Plausible Range to a Probability Statement
An expert may say that a treatment effect is “probably between” two values, or provide a low, typical, and high estimate. Those answers can be useful, but they do not specify a prior distribution until the probabilities attached to the values are made clear. A lower and upper value might describe the middle 90% of the expert’s uncertainty, a wider range, or merely values the expert happens to find easy to name. These interpretations are not interchangeable.
The practical aim is to ask for judgments with explicit probability levels. For a parameter \(\theta\), one might ask for the value below which the expert believes there is 5% probability, the 50th-percentile value, and the value below which there is 95% probability. The resulting values describe a distribution of uncertainty about \(\theta\); they are not observations from the trial.
For a continuous distribution, the probability between the \(a\)- and \(b\)-quantiles is \(b-a\), for \(0<a<b<1\). In particular, the interval from the 5th to the 95th percentile contains 90% of the distribution, with 5% in each tail. This does not mean that the parameter is confined to that interval: 10% of the distribution remains outside it.
Before eliciting numbers, specify the parameter and its scale. In a trial, that could be a baseline event risk, a risk difference, a log hazard ratio, or a mean treatment effect in a stated unit. The earlier material on Eliciting Baseline Risks From Clinician Estimates addresses the clinical framing of baseline-risk judgments; here the focus is on making the uncertainty in any such judgment explicit through quantiles.
How to Ask for Quantiles
Use a question that names both the probability and the event. For example: “Imagine repeated draws from your current uncertainty about the 12-month event risk in the target population. What risk would be low enough that you think there is only a 5% chance the true risk is lower?” This asks for a 5th percentile. Ask the corresponding question for a 50th percentile and a 95th percentile, rather than simply asking for a “low,” “usual,” and “high” value.
A 50th percentile is a median, not necessarily a mean or the expert’s most likely value. A range should also be labelled by its probability content: “the middle 90%” has a precise interpretation, while “a plausible range” does not. If experts instead give a confidence level for their own judgment, clarify whether they mean a probability distribution over the parameter or a confidence interval from repeated sampling. In prior elicitation, the intended object is uncertainty about the parameter before observing the trial data.
Quantile judgments should be ordered. For a single parameter with a well-behaved continuous distribution, \(Q(0.05)\leq Q(0.50)\leq Q(0.95)\). If an expert gives values out of order, do not silently rearrange them: check whether the scale, event, or probability wording was misunderstood. Also ask whether each number is approximate. A judgment such as “about 0.20” should not be recorded as if it were known to four decimal places.
Worked Example: An Elicited Range for Baseline Risk
Suppose, hypothetically, a clinician is asked about the 12-month event risk in a specified trial population. The clinician reports a 5th-percentile risk of 0.06, a median risk of 0.12, and a 95th-percentile risk of 0.23.
The elicited quantiles are \(Q(0.05)=0.06\), \(Q(0.50)=0.12\), and \(Q(0.95)=0.23\). They are ordered, and the endpoint probabilities define a central interval from 0.06 to 0.23 with probability content \(0.95-0.05=0.90\), under the continuous-distribution interpretation.
In context, the elicitation assigns 90% prior probability to a 12-month event risk between 6% and 23%, with 5% below 6% and 5% above 23%. It does not say that risks outside this range are impossible. The median risk is 12%, but this alone does not imply that the mean risk is 12%.
The judgment is a set of quantile constraints, not a complete distribution. Many distributions can have these three quantiles and different shapes between or beyond them. A distributional family may be selected later, but the elicited probabilities should remain clear and should not be replaced by an assumption that the stated endpoints are hard limits.
Quantiles on a Transformed Scale
Experts may reason more naturally on one scale than another. For example, a treatment effect may be elicited as a log hazard ratio, while results are often reported as a hazard ratio. If \(g\) is a continuous, strictly increasing transformation, quantiles transform in the same order: the generalized \(p\)-quantile of \(g(\theta)\) is \(g(Q(p))\). This follows because \(g\) preserves order and, by continuity, maps the infimum defining \(Q(p)\) to the infimum defining the quantile of \(g(\theta)\).
Worked Example: Translating Log-Hazard-Ratio Quantiles
Suppose, hypothetically, an expert gives the 5th, 50th, and 95th percentiles of a log hazard ratio \(\theta\) as \(-0.40\), \(-0.12\), and \(0.08\), respectively. The hazard ratio is \(\exp(\theta)\), and exponentiation is strictly increasing.
Therefore the corresponding hazard-ratio quantiles are:
The central 90% interval on the hazard-ratio scale is approximately 0.6703 to 1.083. The median hazard ratio is approximately 0.8869. These values are dimensionless. If the endpoint is an adverse event, a hazard ratio below 1 represents a lower hazard under the treatment relative to the comparator; the interval nevertheless includes values on both sides of 1.
The calculation does not make the lower and upper values hard bounds. It carries the original 5th- and 95th-percentile meanings through a monotone transformation. For a decreasing transformation, the order of the endpoints reverses, so the quantile levels must be handled accordingly.
Checking a Proposed Distribution Against Elicited Quantiles
Quantiles can also reveal whether a convenient distributional assumption is compatible with the expert’s judgments. For example, a normal distribution is symmetric about its mean and median. If its median is \(m\), its equal-tail quantiles must be equally far from \(m\). This symmetry is a property of the proposed distribution, not a requirement that every expert’s uncertainty be symmetric.
For a continuous, symmetric distribution with a strictly increasing CDF, the quantiles satisfy \(Q(p)+Q(1-p)=2m\). To see why, symmetry gives \(F(2m-x)=1-F(x)\). If \(F(Q(p))=p\), then \(F(2m-Q(p))=1-p\); strict increase ensures that this value is the unique \((1-p)\)-quantile. The uniqueness condition matters. With flat portions of a CDF, a generalized inverse need not reflect in this way.
Worked Example: Checking a Normal-Distribution Assumption
Suppose, hypothetically, an expert gives the 10th percentile of a treatment effect as \(-4\) points, the median as 1 point, and the 90th percentile as 6 points. The treatment effect is measured in points on a clinical scale.
The endpoint distances from the median are \(1-(-4)=5\) points and \(6-1=5\) points. The values are therefore symmetric about the median. This is consistent with, but does not establish, a symmetric distribution.
For a normal candidate distribution, the standard-normal 90th percentile is approximately 1.2816. Since the median equals the mean for a normal distribution, the upper quantile obeys \(6=1+1.2816\sigma\). Thus \(\sigma=5/1.2816\approx3.901\) points. The corresponding lower quantile is \(1-1.2816(3.901)\approx-4.000\) points, matching the elicited value to rounding.
This calculation checks the compatibility of these quantiles with a normal candidate; it does not prove that the expert’s uncertainty is normal. Additional quantiles or probability questions could reveal skewness or heavier tails that these three values do not show.
Worked Example: A Symmetric Distribution With a Nonunique Median Quantile
Consider a hypothetical distribution that assigns probability one half uniformly over \([-2,-1]\) and probability one half uniformly over \([1,2]\). Its CDF is continuous and the distribution is symmetric about 0. At \(-1\), the CDF reaches 0.5, and it stays at 0.5 throughout the gap from \(-1\) to 1.
Under the generalized-inverse definition, \(Q(0.5)=\inf\{x:F(x)\geq0.5\}=-1\). Reflecting this quantile about 0 gives 1, not \(-1\). Yet the generalized-inverse 50th quantile remains \(Q(0.5)=-1\). Thus the identity \(Q(0.5)+Q(0.5)=0\) does not hold.
The symmetry center 0 is still a median in the ordinary sense: at 0, at least half the probability is at or below the value and at least half is at or above it. The example shows why symmetry statements about quantiles require uniqueness, and why a chosen generalized inverse should not be confused with every value that may qualify as a median.
Diagnostics, Alternatives, and Limitations
A useful elicitation process checks more than whether the values are numerically ordered. Confirm that the expert is answering for the same population, endpoint, follow-up time, and treatment contrast throughout. Check that the scale is clear, particularly for effects that can be represented as a difference, ratio, or logarithm. Ask what probability lies outside a proposed range and whether the expert’s tails are symmetric. These follow-up questions can expose a range that was intended as an approximate clinical envelope rather than as a probability interval.
Quantile questions are often clearer than asking for a range without a probability level. Direct probability questions, such as asking the probability that a risk exceeds a clinically important threshold, can be more informative when that threshold matters to the decision. Clinical scenarios can help experts reason about effects in context. These approaches answer different questions; a few quantiles do not, by themselves, determine a full prior distribution or all decision-relevant probabilities.
Common failure modes include eliciting endpoints before explaining the tail probabilities, treating “plausible” as “certain,” and fitting a symmetric distribution simply because the elicited lower and upper values happen to be available. Quantiles can also be difficult to elicit when experts have little experience thinking in probabilities. Use plain-language probability prompts, allow time for revision, and record uncertainty about the elicited values rather than presenting rough judgments as exact facts.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- An expert gives a 10th percentile of 0.04 and a 90th percentile of 0.18 for a clinical risk. What probability does the interval between them contain under a continuous distribution, and are values outside it impossible?
- A log risk ratio has 5th and 95th percentiles \(-0.20\) and \(0.30\). What are the corresponding risk-ratio quantiles, to three significant figures?
- A proposed symmetric distribution has a unique median quantile \(m=2\) and \(Q(0.10)=-3\). What must \(Q(0.90)\) be?
- Why should an expert’s “plausible range” not automatically be recorded as a 100% interval?
Solutions. (1) The interval contains \(0.90-0.10=0.80\), or 80%, of the distribution. The remaining 20% lies in the tails, so outside values are not impossible. (2) Exponentiation gives \(\exp(-0.20)=0.819\) and \(\exp(0.30)=1.35\), the 5th and 95th percentiles. (3) With symmetry and unique quantiles, \(Q(0.10)+Q(0.90)=2m\), so \(Q(0.90)=4-(-3)=7\). (4) “Plausible” does not specify zero probability outside the range; ask the expert what probability the endpoints represent before assigning quantile levels.