Why Compare These Priors?
A prior distribution can place its center near no treatment effect, or it can center on a treatment benefit that experts consider plausible. It can also be narrow or wide. These choices have different implications: a prior centered near no effect may still allow substantial benefit if it is wide, while a prior centered on benefit may remain uncertain if its spread is large. Comparing “skeptical” and “enthusiastic” priors therefore requires more than comparing their labels.
This tutorial uses a hypothetical randomized trial of a continuous symptom-severity endpoint measured in points, with lower scores indicating less severe symptoms. Let \(\delta\) be the treatment-group mean minus the control-group mean. Thus, negative values favor treatment. Suppose a reduction of at least 1 point is considered clinically important. The probability of that benefit under a prior is \(P(\delta\leq -1)\).
A useful comparison keeps the treatment-effect definition, outcome scale, and threshold fixed. It then examines prior centers and scales, and reports probabilities of clinically relevant events. The earlier material on elicitation motivates translating clinical judgments into distribution parameters; here the emphasis is on what those parameters imply when alternatives are placed side by side.
Comparing Centers and Scales
For transparent calculations, suppose each candidate prior is normal: \(\delta\sim N(m,\omega^2)\), where \(m\) is the prior mean and \(\omega\) is the prior standard deviation. The second argument in \(N(m,\omega^2)\) is the variance, so the prior standard deviation is \(\omega\), not \(\omega^2\). It describes prior uncertainty about the treatment effect; it is not the trial’s standard error.
The formula follows by standardizing the prior variable. Since \(\omega>0\), the inequality keeps its direction when both sides are divided by \(\omega\):
This gives a direct interpretation of each candidate prior. A more negative \(m\) shifts probability toward benefit. A larger \(\omega\) spreads probability farther from the center in both directions; it does not, by itself, express optimism. The standard deviation is therefore essential when interpreting a prior mean.
Worked Example: A Skeptical and an Enthusiastic Prior
Suppose the skeptical prior is \(\delta_{\mathrm{S}}\sim N(0,1.5^2)\), and the enthusiastic prior is \(\delta_{\mathrm{E}}\sim N(-1,1.5^2)\). Both have a standard deviation of 1.5 points, but the enthusiastic prior is centered 1 point in the beneficial direction. Compare their probabilities of any benefit, clinically important benefit of at least 1 point, and benefit of at least 2 points.
For any benefit, calculate \(P(\delta<0)\). Under the skeptical prior, symmetry around zero gives \(P(\delta_{\mathrm{S}}<0)=0.5000\). Under the enthusiastic prior,
For a clinically important benefit of at least 1 point, use the threshold \(-1\). The skeptical prior assigns
The enthusiastic prior assigns
For benefit of at least 2 points, the threshold is \(-2\). The skeptical prior gives \(\Phi(-2/1.5)=\Phi(-4/3)\approx 0.09121\). The enthusiastic prior gives \(\Phi((-2+1)/1.5)=\Phi(-2/3)\approx 0.2525\). Thus, in this comparison, the equal spreads make the change in probabilities attributable to the shift in center. These are prior probabilities about the treatment effect, not probabilities calculated from trial data.
Separate a Change in Spread From a Change in Center
A fair comparison often changes one feature at a time before presenting a pair of complete alternatives. Holding the center fixed isolates the effect of scale. For a prior centered at zero, increasing \(\omega\) puts more probability in both tails. It consequently raises the probability of a sufficiently large benefit and the probability of a sufficiently large harm. A wider prior centered at zero is not automatically more enthusiastic.
Worked Example: A Wider Prior Is Not Necessarily More Optimistic
Compare \(\delta_{\mathrm{N}}\sim N(0,0.75^2)\) and \(\delta_{\mathrm{W}}\sim N(0,1.5^2)\). Both are centered at no treatment effect, but the second is twice as wide. Calculate the probability of at least a 1-point benefit and at least a 1-point harm. Here harm of at least 1 point means \(\delta\geq 1\).
For the narrower prior, the probability of a 1-point benefit is \(\Phi((-1-0)/0.75)=\Phi(-4/3)\approx 0.09121\). By symmetry, its probability of harm of at least 1 point is also \(0.09121\). For the wider prior, the benefit probability is \(\Phi((-1-0)/1.5)=\Phi(-2/3)\approx 0.2525\), and symmetry again gives the same probability of harm.
The wider prior assigns more probability to either direction beyond 1 point: about 25.25% for benefit and 25.25% for harm, compared with about 9.12% for each under the narrower prior. If the event of interest is an effect at least 1 point from zero in either direction, the probabilities are \(2(0.09121)\approx 0.1824\) and \(2(0.2525)\approx 0.5050\), respectively. The comparison shows greater uncertainty, not a more favorable center.
A compact comparison table can help keep the interpretation clear. It should show the effect definition and clinical direction alongside the distribution parameters and the probability calculation, rather than using labels alone.
| Feature | Question to ask | Interpretation in this setting |
|---|---|---|
| Center \(m\) | Where is the prior located? | A more negative center favors a larger reduction in symptom severity. |
| Scale \(\omega\) | How dispersed is the prior? | A wider prior allows more extreme effects in both directions. |
| Clinical threshold | What effect counts as important? | For a benefit of at least 1 point, calculate \(P(\delta\leq -1)\). |
| Direction of harm | Which values are unfavorable? | Here, an effect of \(\delta\geq 1\) favors control by at least 1 point. |
Worked Example: The Clinical Threshold Changes the Comparison
Return to \(\delta_{\mathrm{S}}\sim N(0,1.5^2)\) and \(\delta_{\mathrm{E}}\sim N(-1,1.5^2)\). Calculate the probability of benefit at three thresholds: at least 0.5, 1, and 2 points. This checks whether the comparison depends on the chosen definition of clinically important benefit.
A benefit of at least 0.5 points corresponds to \(\delta\leq -0.5\). For the skeptical prior the probability is \(\Phi(-0.5/1.5)=\Phi(-1/3)\approx 0.3694\). For the enthusiastic prior it is \(\Phi((-0.5+1)/1.5)=\Phi(1/3)\approx 0.6306\).
At the 1-point threshold, the probabilities are \(\Phi(-2/3)\approx 0.2525\) for the skeptical prior and \(\Phi(0)=0.5000\) for the enthusiastic prior, as calculated above. At the 2-point threshold, they are \(\Phi(-4/3)\approx 0.09121\) and \(\Phi(-2/3)\approx 0.2525\), respectively.
The enthusiastic prior assigns greater probability to benefit at all three thresholds, but the numerical difference depends on the threshold. For example, at 0.5 points the difference is about 0.2611, while at 2 points it is about 0.1613. A conclusion such as “the enthusiastic prior is more favorable” should therefore be accompanied by the particular clinical event being compared.
Diagnostics, Alternatives, and Limitations
Before interpreting the comparison, check that both candidate priors refer to the same parameter, contrast direction, units, and population. A sign error can reverse which prior appears enthusiastic. Also verify that the stated spread is a standard deviation and that the variance used in \(N(m,\omega^2)\) is its square. Recalculate probabilities from the unrounded standardized values: for example, \(\Phi(-4/3)\approx 0.09121\), not a value obtained by rounding \(-4/3\) to \(-1.333\) and then reporting unsupported extra digits.
The normal model is an assumption, not part of the definition of skepticism or enthusiasm. It is useful when a continuous treatment effect is plausibly summarized by a center and a symmetric spread. It can be unsuitable when the effect is strongly asymmetric, has important bounds, or has tails inconsistent with clinical knowledge. In those cases, compare probabilities under an elicited distribution better suited to the parameter rather than forcing a normal approximation. A visually reasonable center and standard deviation do not guarantee reasonable tail probabilities.
Other useful comparison strategies answer slightly different questions:
- One prior specification: gives a single analysis choice, but does not show how conclusions depend on alternative clinical beliefs.
- A sensitivity set: compares several plausible priors, changing centers or scales in a structured way. This makes the range of assumptions explicit rather than treating one pair as exhaustive.
- A weakly informative prior: may limit implausibly extreme effects while leaving substantial uncertainty. It is not automatically neutral, and its implied probabilities should still be examined.
A skeptical prior does not mean that treatment is ineffective; it describes a distribution that gives specified weight to possible effects before observing the trial data. An enthusiastic prior is not evidence of efficacy either. The comparison is limited to the assumptions represented by the candidate distributions and the thresholds selected. It cannot establish which prior is correct, and it does not replace careful elicitation or justify choosing a prior merely because it produces a preferred result.
Check Your Understanding
Answer each question, then check the solutions below.
- For \(\delta\sim N(-0.5,1^2)\), where negative values favor treatment, calculate the probability of a benefit of at least 1 point.
- Two priors are centered at zero, but one has twice the standard deviation of the other. Does the wider prior necessarily express more enthusiasm for treatment? Explain.
- For the pair \(N(0,1.5^2)\) and \(N(-1,1.5^2)\), calculate the probability of a benefit of at least 1 point under each.
- Name two checks needed before comparing skeptical and enthusiastic priors.
Solutions. (1) The threshold is \(-1\), so \(P(\delta\leq-1)=\Phi((-1+0.5)/1)=\Phi(-0.5)\approx 0.3085\). (2) No. With the same zero center, the wider prior assigns more probability to both sufficiently large benefits and sufficiently large harms; it represents greater dispersion, not a more beneficial center. (3) The probabilities are \(\Phi(-2/3)\approx 0.2525\) and \(\Phi(0)=0.5000\), respectively. (4) For example, confirm that the priors use the same treatment-effect definition, units, and sign convention, and verify that the reported scale is the standard deviation rather than the variance.