Why Use a Sensitivity Set?
A prior distribution represents uncertainty about a parameter before observing the trial data. In clinical applications, that distribution may depend on expert judgments, earlier evidence, and assumptions about plausible treatment effects. Those inputs can be uncertain too. A sensitivity set makes that uncertainty visible by specifying a small number of clinically defensible alternatives and examining how each affects the analysis.
The point is not to generate many arbitrary priors or to choose whichever one produces a preferred conclusion. Instead, design the alternatives to answer a clear question: Does the analysis depend on where the prior is centered, how concentrated it is, or how much probability it assigns to distant effects? To make that comparison interpretable, vary one feature at a time where possible.
For example, let \(\theta\) be the log hazard ratio for an experimental treatment relative to control. Then \(\theta=0\) corresponds to a hazard ratio of 1, and \(\theta<0\) corresponds to a hazard ratio below 1. The sign and clinical meaning depend on how the outcome and treatment contrast are defined; state that direction before interpreting any prior. A prior centered at \(\mu\) on the log hazard ratio scale has median hazard ratio \(\exp(\mu)\) when the prior is symmetric and continuous.
A useful sensitivity set has a declared target parameter and scale, a short clinical rationale for each member, and a comparison that isolates a specified prior feature. Keep the likelihood, data, endpoint definition, estimand, and decision rule fixed when comparing posterior conclusions. Otherwise, a difference cannot be attributed specifically to the prior.
Designing Comparisons That Isolate One Feature
Location, scale, and tail behavior are related but distinct. Changing a prior center shifts its typical values. Changing its scale alters concentration around that center. Changing its tail behavior can alter the probability assigned to distant values even when center and variance are held approximately fixed.
A practical construction begins with a reference prior and then makes targeted alternatives. For a location comparison, keep the distributional family and scale fixed. For a scale comparison, keep the family and center fixed. For a tail comparison, keep the center fixed and, when feasible, match the variance so that the contrast is not merely a comparison of overall spread.
Name the parameter, its scale, the treatment contrast, and which values represent benefit, no effect, and harm.
Use the available elicited judgments and evidence to specify a reference center, uncertainty, and distributional shape.
Vary the center, scale, or tail behavior while holding other features fixed as much as possible.
Apply the same model and report how the target posterior summaries or decisions change, alongside the rationale for each prior.
The prior alternatives must be plausible, not merely mathematically convenient. A sensitivity set should not include values ruled out by the endpoint definition, treatment mechanism, or credible clinical knowledge. Conversely, it should not exclude uncertainty solely because extreme values seem inconvenient. The earlier material on eliciting expert judgments and avoiding implausible prior tails provides relevant background for judging these bounds.
Worked Example: Changing the Prior Center
Suppose experts consider three nearby centers plausible for \(\theta\), the log hazard ratio of a hypothetical treatment compared with control. Use normal priors with a common standard deviation of 0.20:
The parameter is dimensionless because it is a log ratio. The centers correspond to median hazard ratios of \(\exp(-0.10)=0.9048\), \(\exp(-0.20)=0.8187\), and \(\exp(-0.30)=0.7408\), respectively. These are hypothetical values; whether they are clinically plausible must be justified for the specific disease and treatment.
One way to make the location comparison concrete is to calculate the prior probability of benefit, \(P(\theta<0)\). For a normal prior with center \(\mu\) and standard deviation \(\sigma\), standardizing gives \(P(\theta<0)=\Phi((0-\mu)/\sigma)\), where \(\Phi\) is the standard normal cumulative distribution function.
The scale and family are identical, so this comparison examines the prior center. The probabilities describe prior mass below the no-effect value, not posterior evidence and not the probability that the treatment truly benefits patients after seeing trial data.
Worked Example: Changing the Prior Scale
Now suppose the clinically supported center is fixed at \(-0.20\), but experts disagree about the degree of uncertainty. Compare hypothetical normal priors with standard deviations 0.10, 0.20, and 0.35:
The prior probability of benefit is computed in the same way. For the narrow prior, \((0-(-0.20))/0.10=2.00\), so \(P(\theta<0)=\Phi(2.00)=0.9772\). For the intermediate prior, the standardized value is \(0.20/0.20=1.00\), giving \(\Phi(1.00)=0.8413\). For the broad prior, it is \(0.20/0.35=0.5714\), giving \(\Phi(0.5714)\approx0.7161\).
All three priors have the same center, so this comparison examines scale. A broader prior places more probability on values both above and below the center; it does not simply express less belief in benefit. In this example, the center is below zero, so widening the prior also brings more mass across zero.
Comparing Tail Behavior Fairly
Two priors can have the same center and nearly the same variance but differ in how much probability they assign far from that center. A mixture of a narrow and a broad normal component is one way to represent a concentrated body with a small allowance for distant values. The mixture weight and component scales require clinical justification; a broad component is not automatically plausible merely because it is mathematically possible.
For a mixture with component means equal to \(\mu\), the overall mean is \(\mu\), and its variance is the weighted average of the component variances. This follows by conditioning on which component generated the value: since both component means are \(\mu\), there is no additional between-component variance in the means.
where \(w\) is the weight of the broader component. To compare tails, choose a clinically meaningful distance \(c\) from the center and calculate \(P(|\theta-\mu|>c)\) under each prior. This is a prior probability about parameter values, not a probability about future trial observations.
Worked Example: Comparing Tails at Nearly Matched Variance
Consider the hypothetical mixture prior
and compare it with a normal prior \(N(\mu,0.1984^2)\). Both are centered at \(\mu\). The mixture variance is
Thus the mixture standard deviation is \(\sqrt{0.039375}\approx0.1984\), matching the comparison normal's standard deviation to four significant figures. The comparison therefore focuses on tail shape rather than a substantial difference in variance.
Take \(c=0.50\) log-hazard-ratio units from the common center. Under the normal prior, the standardized distance is \(0.50/0.1984\approx2.520\), giving
For the mixture, calculate the two-sided probability in each component and weight the results. In the narrow component, \(0.50/0.15=3.333\), so the probability is \(2\Phi(-3.333)\approx0.00086\). In the broad component, \(0.50/0.60=0.8333\), so it is \(2\Phi(-0.8333)\approx0.40466\), or \(0.405\) to three significant figures. Therefore,
The mixture assigns about \(2.105\%\) probability to values more than 0.50 from its center, compared with about \(1.173\%\) under the variance-matched normal prior. This difference shows what the comparison examines: tail probability at the chosen distance, with the center and variance held fixed to rounding precision. It does not establish that either tail is clinically appropriate; that judgment comes from the elicitation and scientific context.
Assumptions, Diagnostics, and Alternatives
A sensitivity set is informative only if its alternatives address genuine uncertainty and are compared consistently. Before interpreting results, check that each prior is defined on the same parameter scale, uses the same treatment direction, and respects the parameter's support. Check calculations of centers, variances, and threshold probabilities, especially for mixtures. For posterior comparisons, use the same likelihood, observed data, model assumptions, estimand, and decision threshold for every member of the set.
Warning signs include alternatives that change several features at once without explanation; a set so wide that some members are clinically indefensible; a narrow set that does not cover credible disagreement; and a large list of priors selected after inspecting the trial result. A further diagnostic is to examine whether the stated clinical rationale actually supports the numerical values. For instance, labeling a prior “broad” does not explain why its component standard deviation or mixture weight is credible.
There are several alternatives to a sensitivity set, but they answer different questions. A single elicited prior gives one primary analysis and can be appropriate when the evidence and elicitation are sufficiently compelling, although it hides dependence on that choice. A sensitivity set keeps a small number of alternatives visible and compares results under each. A hierarchical model or formal prior averaging can assign probabilities to uncertainty among models or priors, but requires additional modeling assumptions; listing candidate priors does not do this automatically.
A sensitivity set also has limits. It cannot guarantee that the true prior is among its members, repair a misspecified likelihood, or resolve uncertainty that was never considered. Its conclusions depend on which alternatives were selected and which summaries were compared. Report the set, the clinical rationale, the feature each comparison varies, and any material changes in conclusions. Do not summarize a result as “robust” without specifying the range of priors and the outcomes for which it was robust.
Common Mistake
Another frequent error is calling a tail comparison a pure tail-shape comparison when the priors also have very different variances. Matching variances, or explicitly acknowledging the variance difference, prevents that interpretation from being overstated. Likewise, a collection of many alternatives is not necessarily more rigorous: a small, prespecified, interpretable set is often easier to justify and audit.
Check Your Understanding
Answer each question, then compare with the solutions below.
- For a location comparison, what should generally be held fixed?
- A normal prior on a log hazard ratio has center \(-0.25\) and standard deviation \(0.25\). What is its prior probability that \(\theta<0\)?
- Why can a mixture and a normal prior with nearly equal variances still have different probabilities of distant effects?
- Does a sensitivity set assign probabilities to its alternative priors by itself?
Solutions. (1) Keep the distributional family and scale fixed, and vary the center; this isolates location. (2) Standardize: \((0-(-0.25))/0.25=1\), so \(P(\theta<0)=\Phi(1)=0.8413\). (3) Variance summarizes average squared distance from the center but does not uniquely determine tail shape; a small broad mixture component can increase distant-value probability. (4) No. It is a collection for comparison unless a separate model explicitly assigns weights or probabilities to its members.