Skeptical Priors for Treatment Effects

What Makes a Prior Skeptical?

A clinical trial may estimate a treatment effect that is compatible with benefit, no effect, or harm. Before observing the trial results, a skeptical prior expresses caution about large departures from no effect. It does not declare that the treatment is ineffective, and it does not make the trial data irrelevant. Instead, it assigns more prior probability to effects near the null value than to effects far from it.

To state such a prior clearly, first define the treatment-effect parameter and its direction. For example, let \(\theta\) be the mean outcome in the treatment group minus the mean outcome in the control group, measured in the outcome’s original units. If lower values are better, negative values favor treatment, positive values favor control, and \(\theta=0\) is no difference in means. The same reasoning applies to other effect parameters with a zero-valued null, provided the parameter’s scale and interpretation are stated.

Definition: A symmetric skeptical prior centered at the null is a prior distribution for a treatment-effect parameter \(\theta\) that is centered at its null value, here \(\theta=0\), and assigns decreasing plausibility to departures farther from zero. A common specification is \(\theta\sim N(0,\tau^2)\), where \(\tau>0\) is the prior standard deviation. The prior is symmetric about zero; \(\tau\), in the units of \(\theta\), determines how broadly it spreads.

The normal distribution is a convenient example, not the definition of skepticism. What matters for the interpretation is the prior’s center and the amount of probability it assigns to small and large effects. A prior centered at zero expresses no prior preference for either direction when it is symmetric. Its scale expresses how strongly it regards substantial departures from zero as plausible. A smaller \(\tau\) concentrates more probability near zero; a larger \(\tau\) is less skeptical about large effects.

This specification rests on substantive and modeling assumptions. The effect must be defined on a scale for which zero represents the null of interest. Symmetry assumes that, before considering the trial data, departures of equal magnitude in either direction are equally plausible. A normal prior also has support over the whole real line, so it is appropriate only when that support is acceptable for the chosen parameter or when probability assigned to implausible values is negligible. The prior should describe beliefs about the same treatment contrast and population that the trial’s estimand targets.

Choosing the Scale From Plausible Departures

The scale \(\tau\) should be elicited in interpretable units, not chosen merely because a software default is available. For a normal prior, about 68% of prior probability lies within one standard deviation of zero, and about 95% lies within roughly 1.96 standard deviations. These statements allow a clinician or trial team to describe a plausible range and translate it into \(\tau\). They are probability statements, not hard limits: values beyond the range retain some prior probability.

More generally, if the team judges that a central proportion \(q\) of prior effects should lie between \(-a\) and \(a\), then for \(\theta\sim N(0,\tau^2)\), \(P(-a\leq\theta\leq a)=q\). Let \(z_{(1+q)/2}\) be the corresponding standard-normal quantile. The implied scale is \(\tau=a/z_{(1+q)/2}\). This calculation makes the elicitation statement and the resulting prior consistent.

Worked Example: Setting a Prior Range in Outcome Units

Suppose a hypothetical trial compares a treatment with control on a symptom score measured in points, and \(\theta\) is the treatment-minus-control difference. Lower scores are better, so negative effects favor treatment. The team wants a symmetric prior that assigns approximately 95% probability to effects between \(-6\) and \(6\) points. Assume a normal prior centered at zero.

For a normal distribution, the central 95% interval is approximately \(-1.96\tau\) to \(1.96\tau\). Setting its upper endpoint to 6 gives \(1.96\tau=6\), so \(\tau=6/1.96=3.061\) points. The proposed prior is therefore \(\theta\sim N(0,3.061^2)\), with variance approximately \(9.370\) square points.

This prior assigns about 95% probability to a treatment effect between \(-6\) and \(6\) points and about 2.5% to effects below \(-6\) and 2.5% to effects above \(6\). The interval is not a boundary: the prior still allows values outside it. Because the distribution is symmetric, the prior probability of a negative effect is 0.5. The choice is skeptical about large departures, but does not favor benefit over harm.

The elicited range should be checked against clinical knowledge and the model’s parameter support. A range in outcome units is easier to review than an unexplained standard deviation. If a scale change is made, the prior must be converted too: a prior stated in points cannot be used unchanged when the parameter is represented in different units. If the effect parameter cannot physically take negative values, a symmetric distribution on the real line may be unsuitable even if its center seems appealing.

Worked Example: Eliciting From a Probability Statement

In another hypothetical study, suppose the treatment effect is a difference in a continuous outcome, measured in points, with zero representing no difference. Investigators judge that 90% of prior effects should be within 4 points of zero. They are willing to use a symmetric normal prior.

For a central 90% normal interval, the standard-normal quantile is approximately 1.645. Thus \(4=1.645\tau\), giving \(\tau=4/1.645=2.432\) points. The prior is \(\theta\sim N(0,2.432^2)\).

The same prior also lets us quantify a narrower range. The standardized value for 2 points is \(2/2.432=0.822\). Using the standard normal cumulative distribution function \(\Phi\), \(P(|\theta|\leq2)=2\Phi(0.822)-1\approx2(0.7944)-1=0.5888\). Thus about 58.9% of prior probability lies within 2 points of zero, while 90% lies within 4 points. The calculation clarifies what the elicited 90% statement implies at a smaller distance; it does not turn either range into a cutoff.

How the Prior Combines With Trial Evidence

The effect of a skeptical prior can be seen under a simple approximate likelihood. Suppose an estimate \(y\) of \(\theta\) has known standard error \(s\), and the sampling model is \(y\mid\theta\sim N(\theta,s^2)\). Assume the prior is \(\theta\sim N(0,\tau^2)\), and that the likelihood and prior refer to the same effect parameter. Under these assumptions, the posterior is normal, with a mean between zero and the observed estimate. Its variance and mean are

\operatorname{Var}(\theta\mid y)=\left(\frac{1}{s^2}+\frac{1}{\tau^2}\right)^{-1}, \qquad E(\theta\mid y)= \left(\frac{1}{s^2}+\frac{1}{\tau^2}\right)^{-1} \frac{y}{s^2} = \frac{\tau^2}{\tau^2+s^2}y.

The prior contributes precision \(1/\tau^2\), while the likelihood contributes precision \(1/s^2\). The posterior mean is a precision-weighted compromise: the smaller \(\tau\), the more weight the prior gives to zero. This result depends on the stated normal model and known standard error; it is not an exact description of every trial analysis.

Define the estimand and null.
State what \(\theta\) compares, its units, which direction favors treatment, and why \(\theta=0\) represents no effect.

Elicit a plausible range.
Ask how large a departure from zero is plausible before seeing the trial data, and what central probability should lie within that range.

Translate and check.
For a normal prior, convert the range to \(\tau\), then inspect the implied probabilities, units, symmetry, and support.

Assess data sensitivity.
Fit the trial model and compare the inference under reasonable alternative skeptical scales, keeping the likelihood and estimand fixed.

Worked Example: Shrinkage With a Skeptical Prior

Consider hypothetical continuous-outcome trial results summarized by an estimated treatment-minus-control difference \(y=-3.0\) points with standard error \(s=2.0\) points. Negative values favor treatment. For illustration, assume the approximate likelihood \(y\mid\theta\sim N(\theta,2.0^2)\) and use the skeptical prior \(\theta\sim N(0,2.5^2)\). Both the estimate and prior concern the same difference in mean outcome, so their units and estimand align.

The posterior precision is \(1/2.0^2+1/2.5^2=1/4+1/6.25=0.4100\) per square point. Therefore the posterior variance is \(1/0.4100=2.439\) square points, and the posterior standard deviation is \(\sqrt{2.439}=1.562\) points.

The posterior mean is \(({-3.0}/4)/0.4100=-0.7500/0.4100=-1.829\) points. It is closer to zero than the observed estimate because the prior contributes information centered at zero. Under this posterior, the probability of a negative effect is \(\Phi(1.829/1.562)=\Phi(1.171)\approx0.879\). In this hypothetical analysis, the posterior assigns about 87.9% probability to a lower mean score under treatment. This is a posterior probability under the stated model and prior, not a probability that the null is true.

Assumptions, Diagnostics, and Alternatives

A skeptical prior is useful when there is a defensible reason to expect that very large effects are less plausible before the trial results are known. It can reduce the influence of noisy estimates, but its implications must be checked rather than assumed benign. Prior predictive checking means using the prior together with the trial’s data model to examine the outcomes it could generate before observing the data. If those implied outcomes are clinically impossible, the prior or likelihood model needs reconsideration.

Several checks are particularly important:

Check the center and direction. A symmetric prior centered at zero gives equal prior probability to positive and negative effects. If the parameter’s sign convention is reversed, the clinical interpretation reverses too.
Check the scale in context. Report \(\tau\) in the effect’s units and state an interpretable prior probability range. A number without units or elicitation rationale is difficult to evaluate.
Check support and implied outcomes. A normal prior allows arbitrarily large positive and negative values. If such values are impossible or lead to nonsensical trial predictions, use a model and prior compatible with the parameter’s support.
Check sensitivity. Refit under other plausible values of \(\tau\). If the main conclusion changes substantially, report that dependence rather than presenting one scale as uniquely determined.
Check that the prior is genuinely prior information. Do not tune its scale after seeing the observed treatment effect merely to obtain a preferred conclusion.

A broad, weakly informative prior is an alternative when the available information supports ruling out extreme values but does not justify strong concentration near zero. A directional prior has a different premise: it favors one direction before seeing the data, rather than expressing symmetric caution about departures. Neither alternative should be substituted silently for a skeptical prior. The choice should reflect the available prior knowledge and be made explicit.

A continuous skeptical prior, such as \(N(0,\tau^2)\), assigns probability zero to the exact value \(\theta=0\). It can put substantial probability near zero, but it does not assign positive probability to a point null. Thus it represents uncertainty about the size and direction of an effect, not a prior model in which no effect is a separate possibility. Skepticism also does not guarantee that posterior conclusions will be conservative: sufficiently informative trial data can move the posterior away from zero, and a poorly chosen prior can distort inference.

Common Pitfall

Common mistake: “The prior is skeptical, so the effect is assumed to be zero,” or “95% of the prior is within the stated range, so effects outside that range are impossible.”

Correction: A continuous prior centered at zero expresses greater prior density near the null, not certainty that the null holds. A central 95% prior interval leaves 5% probability outside its endpoints. The prior’s scale should be interpreted as a graded statement about plausible departures, then checked against the data model and reasonable alternatives.

Check Your Understanding

Answer each question, then compare with the solutions below.

A treatment-minus-control effect is measured in minutes, with negative values favoring treatment. What is the null value, and what does a symmetric prior centered there imply about prior direction?
For a normal prior, a central 95% range is specified as approximately \(-8\) to \(8\) units. Find \(\tau\).
Does a continuous normal prior centered at zero assign positive probability to the exact null value? Explain.
In the normal likelihood example, what happens to the posterior mean’s weight on the observed estimate when \(\tau\) increases?

Solutions. (1) The null is \(\theta=0\) minutes, representing no mean difference. Symmetry gives equal prior probability to effects favoring either group; it does not favor treatment. (2) \(1.96\tau=8\), so \(\tau=8/1.96=4.082\) units. (3) No. A continuous distribution assigns probability zero to any single exact value, including zero, although it can assign substantial probability to values near zero. (4) The weight is \(\tau^2/(\tau^2+s^2)\). As \(\tau\) increases, that weight increases toward one, so the posterior mean moves closer to the observed estimate and the prior exerts less shrinkage.