Why Elicit Quantiles?
In a clinical trial, a probability parameter might represent the event rate in a control group or the probability of response under a treatment. An expert may find it easier to express beliefs about such a parameter in probability language: for example, “there is a 90% prior probability that the event rate is below 0.30.” Translating several such statements into a beta prior makes the assumptions explicit and allows them to be examined before trial data are observed.
This tutorial considers a probability parameter \(\theta\), such as an event probability, with \(0<\theta<1\). The proposed prior is a beta distribution. The elicited statements are quantiles of that prior—not observed event rates, confidence limits, or posterior intervals. The approach assumes that the expert’s judgments can reasonably be summarized by a single beta distribution; quantiles that contradict one another, or that cannot be matched adequately by any beta distribution, should not be forced into a fit.
The parameters \(a\) and \(b\) are positive shape parameters. They are not the significance level \(\alpha\). Because a beta distribution has two shape parameters, two distinct quantile statements can be used to determine a candidate pair. More elicited quantiles provide checks on whether that pair represents the expert’s full set of judgments; they are not additional parameters that a beta distribution can freely accommodate.
Turning Elicited Statements Into Equations
Suppose an expert supplies two statements: the \(p_1\) quantile is \(q_1\), and the \(p_2\) quantile is \(q_2\). A candidate beta prior must satisfy
The elicited values must have \(0<p_1<p_2<1\) and \(0<q_1<q_2<1\). The second ordering follows because the beta density is positive throughout the interior of its support, so its cumulative distribution function is strictly increasing there. If the elicitation violates these orderings, no continuous beta distribution can satisfy it.
In general, there is no simple algebraic expression that solves these equations for \(a\) and \(b\). A numerical root finder or optimization routine is used. One convenient formulation minimizes the squared CDF residuals,
The search must enforce \(a>0\) and \(b>0\), for example by optimizing over \(\log(a)\) and \(\log(b)\). Numerical output is a candidate fit, not evidence by itself that the prior is appropriate. The fitted values should be substituted back into the equations, and the resulting density should be inspected for behavior inconsistent with the expert’s intended beliefs.
Write each statement as \(P(\theta\le q_i)=p_i\). Confirm that all probabilities and quantile values lie strictly between 0 and 1 and that their orderings agree.
Solve the two beta CDF equations for positive \(a\) and \(b\), using a numerical method if necessary.
Evaluate the fitted CDF at each elicited quantile. Compare any remaining elicited quantiles with the fitted distribution and decide whether the discrepancies are acceptable for the intended use.
Worked Quantile Fits
Worked Example: A Symmetric Prior for a Response Probability
Suppose an expert believes that the 25th percentile of a response probability \(\theta\) is 0.3264 and the 75th percentile is 0.6736. Find a beta prior that matches these two statements. These are hypothetical elicited beliefs.
The equations to solve are \(I_{0.3264}(a,b)=0.25\) and \(I_{0.6736}(a,b)=0.75\). A numerical solver gives \(a\approx2.00\) and \(b\approx2.00\). The proposed prior is therefore \(\theta\sim\mathrm{Beta}(2,2)\).
For these shape parameters, the density is \(6\theta(1-\theta)\), and integration gives \(F(x)=3x^2-2x^3\). Substitution checks the first elicited quantile:
The result rounds to 0.25, as elicited. The beta distribution is symmetric about 0.5, so \(F(0.6736)=1-F(0.3264)\approx0.7499\), which rounds to 0.75. Thus the fitted prior reproduces both statements to the displayed precision. The parameter \(\theta\) is a probability, so these quantiles are on the probability scale and have no additional measurement unit.
Worked Example: Fitting an Asymmetric Event-Rate Prior
Suppose an expert’s hypothetical judgments are that the 22.5159th percentile of an event probability \(\theta\) is 0.10, and the 56.3792th percentile is 0.20. Fit a beta prior and verify the result.
The two equations are \(I_{0.10}(a,b)=0.225159\) and \(I_{0.20}(a,b)=0.563792\). Numerical solution gives \(a\approx2.00\) and \(b\approx8.00\). The candidate prior is \(\theta\sim\mathrm{Beta}(2,8)\).
For integer shape parameters \(a=2\) and \(b=8\), the CDF can be written \(F(x)=1-(1-x)^8(1+8x)\). At the first elicited value,
At the second value,
Both CDF values match the elicited probabilities. The prior is asymmetric: its density places more weight near lower event probabilities than near higher ones. That shape is consistent with the ordering and values of these hypothetical judgments; it does not establish that the prior is clinically appropriate without considering the expert’s intended interpretation.
Why the Quantile Equations Work
The justification is the definition of a quantile combined with the beta CDF. For a continuous beta distribution with positive shape parameters, the density is positive at every \(\theta\) strictly between 0 and 1. Therefore its CDF increases strictly from 0 to 1. For any probability level \(p\) in that range, there is a unique \(q_p\) satisfying \(F(q_p\mid a,b)=p\). Thus each elicited probability statement gives one equation in the unknown shape parameters.
A pair of distinct statements provides two equations, which can determine two unknowns numerically when the equations have a suitable solution. It does not guarantee a well-behaved or unique numerical solution in every situation. The solver’s result must be checked by evaluating both residuals and, when possible, by trying reasonable starting values to detect instability or different solutions.
A further consequence is a direct compatibility check. If \(p_1<p_2\), then any beta fit must have \(q_{p_1}<q_{p_2}\). A violation is a logical contradiction, independent of numerical method. If the ordering is correct, the statements may still be incompatible with the beta family: the family has only two shape parameters and imposes a particular range of distribution shapes.
Worked Example: Detecting Incompatible Quantile Judgments
Suppose an expert states that the 56.3792th percentile of a hypothetical event probability is 0.20, but also states that its median is 0.25. Can any beta prior satisfy both claims?
The first statement requires \(F(0.20)=0.563792\), which is greater than 0.50. Since a beta CDF is strictly increasing, its median—the value \(q_{0.50}\) at which \(F(q_{0.50})=0.50\)—must be less than 0.20. The second statement instead requires \(q_{0.50}=0.25\), which would place the median above 0.20. These requirements contradict one another:
No beta distribution, and indeed no continuous distribution with a strictly increasing CDF, can meet both statements. The appropriate response is to revisit the elicitation—for example, check whether the probability level or quantile value was recorded incorrectly—not to make a numerical solver fit contradictory inputs.
Assessing More Than Two Quantiles
When an expert supplies three or more quantile statements, use two to produce a candidate fit and treat the rest as model checks. For each additional pair \((p_i,q_i)\), calculate \(I_{q_i}(a,b)\) from the fitted parameters and compare it with \(p_i\). Equivalently, calculate the fitted quantile at level \(p_i\) and compare it with \(q_i\). Report discrepancies on the probability scale or the \(\theta\) scale, whichever makes their practical size clearer.
Small discrepancies can arise from rounding elicited values or reporting only a few digits. Larger discrepancies indicate that the chosen pair of quantiles and the beta family do not summarize all the judgments well. Check first for basic ordering contradictions. Then check sensitivity to which two quantiles were used to fit the prior: if different pairs yield substantially different shape parameters, the elicitation may be weakly informative about the beta shape or may not be well represented by one beta distribution.
Inspecting the fitted density, or drawing prior values from it, can also reveal an unintended concentration near 0 or 1, excessive concentration in the middle, or tails that conflict with the expert’s account. These checks do not replace the quantile equations; they help determine whether the fitted family gives a sensible overall representation rather than matching only two selected points.
Comparison, Limitations, and Common Mistakes
Eliciting a beta prior from a mean and variance is an alternative when those summaries are easier for an expert to assess. The earlier material on eliciting beta priors from mean and variance gives that parameterization. Quantiles instead describe statements such as “the event rate is below this value with this prior probability,” which may be more natural to elicit. Neither approach is universally preferable: use summaries the expert can judge reliably, then check whether the resulting beta distribution captures other relevant beliefs.
Quantile elicitation also has limitations. Two quantiles determine a candidate beta fit but cannot establish that the entire distribution reflects the expert’s beliefs. Numerical results can be sensitive to elicitation rounding or to the selected quantiles. And if beliefs have pronounced features that a single beta density cannot represent, a forced beta fit can obscure rather than resolve the mismatch. In that case, return to the elicitation and consider whether the proposed distributional family is suitable before using the prior in a trial analysis.
- Mistake: treating a quantile as a probability statement about a fixed parameter. Saying “the 90th percentile is 0.30” means \(P(\theta\le0.30)=0.90\) under the prior distribution. It does not mean there is a 90% chance that a fixed, already-known value of \(\theta\) changes or lies randomly on either side of 0.30.
- Mistake: assuming any two elicited quantiles guarantee a sensible prior. They give equations for a candidate fit, but the fit can be numerically unstable or poorly representative. Check the residuals and the distribution’s implications.
- Mistake: fitting every elicited quantile as if the beta family had one parameter per statement. A beta prior has only two shape parameters. Fit two statements and assess the remaining statements for compatibility instead of expecting an exact fit to all of them.
- Mistake: overlooking quantile ordering. If a lower probability level is paired with a larger quantile value, the statements contradict the strictly increasing CDF and must be clarified before fitting.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- An expert says \(P(\theta\le0.15)=0.20\) and \(P(\theta\le0.40)=0.80\). Write the two equations for a beta prior’s shape parameters.
- Can the 70th percentile of a continuous beta distribution be 0.30 while its 40th percentile is 0.35? Explain.
- A numerical fit gives \(a=3\), \(b=7\). What should be checked before treating this as an adequate prior if four quantiles were elicited?
- Why does an additional elicited quantile generally serve as a compatibility check rather than as another freely fit parameter?
Solutions. (1) Solve \(I_{0.15}(a,b)=0.20\) and \(I_{0.40}(a,b)=0.80\), with \(a>0\) and \(b>0\). (2) No. Since \(0.40<0.70\), strict increase of the CDF requires the 40th percentile to be below the 70th percentile, but 0.35 is greater than 0.30. (3) Substitute the fitted values into the CDF equations for all four elicited quantiles, assess the residuals, and inspect whether the implied prior shape matches the intended beliefs; the two quantiles not used for fitting are important checks. (4) The beta family has only two shape parameters, so two quantile equations use those two degrees of freedom. Additional statements impose further constraints and may not be exactly reproducible by that family.