Worked Elicitation for a Binary Trial Endpoint

From an Expert Judgment to a Prior

For a binary clinical endpoint, each participant either experiences the event or does not. Before trial data are observed, an expert may be able to estimate the event probability for a clearly specified population, treatment condition, and follow-up period, and to describe how uncertain that estimate is. The task in this tutorial is to turn those judgments into a prior distribution for the event probability.

The endpoint must be defined before asking for a number. “The event rate” is not sufficiently precise if it could mean events by 30 days or by one year, or if the population or treatment condition is unclear. The elicited probability must refer to the same binary outcome and time horizon that the trial will analyze. The setup below also assumes that participants have the same event probability, conditional on the parameter, and that their binary outcomes are independent.

The parameter \(p\) is unknown but fixed for the modeled population and conditions; the prior describes uncertainty about its value. This is distinct from the Bernoulli variation in individual outcomes conditional on \(p\). It is also distinct from variation in event probabilities between participants. If meaningful patient-level heterogeneity is part of the trial model, a single common \(p\) may not be adequate.

The beta distribution is useful because it is defined on probabilities and can represent a range of shapes, from concentrated and approximately symmetric to strongly skewed. Its two parameters are not themselves the expert’s expected probability and uncertainty. Those are more naturally elicited on the probability scale, then translated into \(\alpha\) and \(\beta\).

Moment Matching for a Beta Prior

Suppose the expert’s judgments are summarized by a prior mean \(m\) and prior standard deviation \(s_p\), both on the probability scale. The notation \(s_p\) here means the standard deviation of the prior distribution for \(p\), not a sample standard deviation. Let \(v=s_p^2\) be the elicited prior variance. For \(p\sim\operatorname{Beta}(\alpha,\beta)\), the mean and variance are

These moment formulas can be used in reverse. Define the total concentration \(\kappa=\alpha+\beta\). Matching the beta mean to \(m\) gives \(\alpha=m\kappa\) and \(\beta=(1-m)\kappa\). Substituting those expressions into the variance formula gives \(v=m(1-m)/(\kappa+1)\), and hence the elicitation formulas below.

The variance condition is a coherence check, not an optional technical detail. A probability-valued parameter with mean \(m\) cannot have variance greater than \(m(1-m)\). If the elicited variance equals that upper bound, the moment-matching formula gives \(\kappa=0\), which is not a proper beta distribution. A variance greater than the bound gives negative concentration. Either result means the stated mean and uncertainty cannot define a beta prior; it does not mean that a negative beta parameter should be used.

The concentration \(\kappa\) describes how tightly the prior is distributed around its mean: holding \(m\) fixed, larger \(\kappa\) means smaller prior variance. Although \(\alpha\) and \(\beta\) are sometimes described as prior event and non-event counts, they are not literal observed participants. That shorthand must not replace the elicitation or the model assumptions.

Check the Elicitation Before Accepting the Parameters

A calculation that returns positive parameters is necessary but not sufficient. The elicitation should be reviewed for clarity, and the resulting distribution should be checked against the judgments that produced it. In particular, ask the expert to distinguish a central estimate from a likely range, and to state uncertainty about the underlying event probability rather than uncertainty about the outcome of one participant.

The beta family can be asymmetric, especially for probabilities near zero or one. Thus, matching a mean and standard deviation does not guarantee that the prior’s median, mode, or tails match an expert’s intended beliefs. If the expert gives quantiles or a range instead of a standard deviation, those statements should be fitted or checked directly; they should not be silently treated as a mean plus or minus a fixed number of standard deviations. A numerical fit can match selected quantiles, but it may not match the mean and variance at the same time.

Failure Modes and Alternatives

Several problems can be detected before the prior is used. First, check that the event definition, population, and follow-up time are unambiguous. A probability for a different time horizon is not a harmless change of units. Second, calculate the feasibility bound \(m(1-m)\) before interpreting the parameters. Third, verify the fitted moments by substituting the calculated \(\alpha\) and \(\beta\) back into the beta mean and variance formulas. Finally, examine the shape for compatibility with what the expert said, not just whether the parameters are positive.

A common mistake is to set \(\alpha=m\) and \(\beta=1-m\), reasoning that these parameters must encode the desired mean. That does give the correct mean, because their ratio is \(m\), but it fixes \(\alpha+\beta=1\) and therefore imposes a particular, generally very diffuse prior. For example, if \(m=0.20\), this choice gives \(\operatorname{Beta}(0.20,0.80)\), not a prior whose uncertainty has been elicited. The correction is to obtain an uncertainty statement as well and use both moments to determine the concentration and the two shape parameters.

The main alternatives serve different purposes. A uniform prior is a specific choice, \(\operatorname{Beta}(1,1)\), not a neutral substitute for eliciting uncertainty. A fixed probability omits uncertainty about \(p\). A beta prior is convenient for a probability parameter, but a single beta distribution may be unsuitable if expert beliefs are distinctly multimodal or cannot be represented adequately by two moments. Direct elicitation of quantiles, followed by fitting and checking an appropriate distribution, can be preferable in those cases. The choice should be documented with the endpoint definition, elicited judgments, parameterization, and checks, rather than justified only by mathematical convenience.