Prior Sensitivity Analysis For Regression Conclusions

Why Examine Prior Sensitivity?

A Bayesian regression conclusion depends on both the likelihood, which describes how the data relate to the parameters, and the prior, which represents information or assumptions specified before observing the data. A prior can have little effect when the likelihood is informative, but it can matter substantially when the data are weakly informative. Prior sensitivity analysis makes that dependence visible: keep the data and model fixed, replace the prior with other substantively defensible alternatives, and compare the resulting posterior conclusions.

The goal is not to search for a prior that produces a preferred answer. It is to learn whether reasonable prior choices lead to meaningfully different conclusions about a coefficient’s sign, size, uncertainty, or implications for outcomes. The examples below use normal regression with known error variance, where the posterior calculations are explicit. The same comparison logic applies when posterior summaries are obtained by simulation.

Definition: A prior sensitivity analysis compares posterior quantities under alternative prior distributions while holding the observed data and likelihood model fixed. The alternatives should be justified as substantively plausible before their consequences are interpreted.

What Is Held Fixed, and What Is Compared?

Suppose the regression model is $y_i=\alpha+\beta x_i+\varepsilon_i$, with independent errors $\varepsilon_i\sim N(0,\sigma^2)$. For the first example, take $\sigma^2$ as known, center the predictor so that $\sum_i x_i=0$, and use a prior on the slope $\beta$. These assumptions make the slope’s likelihood separate from the intercept’s likelihood. They are simplifying assumptions for the calculations, not a claim that the error variance is known in every application.

For this model, the slope estimate $\hat{\beta}$ has sampling distribution $N(\beta,SE^2)$, where $SE$ is its known standard error under the specified design and error variance. If the prior is $\beta\sim N(m_0,v_0)$, the posterior is normal. Its mean is a precision-weighted average of the data estimate and the prior mean; its variance reflects both sources of information.

Normal-normal update: If $\hat{\beta}\mid\beta\sim N(\beta,SE^2)$ and $\beta\sim N(m_0,v_0)$, then $$ \beta\mid\text{data}\sim N(m_1,v_1),\qquad v_1=\left(\frac{1}{SE^2}+\frac{1}{v_0}\right)^{-1},\qquad m_1=v_1\left(\frac{\hat{\beta}}{SE^2}+\frac{m_0}{v_0}\right). $$

A short derivation shows why the update has this form. Up to a constant that does not depend on $\beta$, the likelihood times prior density is proportional to $$ \exp\left\{-\frac{1}{2}\left[\frac{(\hat{\beta}-\beta)^2}{SE^2}+\frac{(\beta-m_0)^2}{v_0}\right]\right\}. $$

Collecting the terms involving $\beta^2$ gives coefficient $1/SE^2+1/v_0$. Completing the square in $\beta$ therefore gives posterior variance $v_1=(1/SE^2+1/v_0)^{-1}$ and posterior mean $v_1(\hat{\beta}/SE^2+m_0/v_0)$, as stated. This also shows why a more concentrated prior has more influence: its precision $1/v_0$ is larger.

Worked Example: Comparing Slope Posteriors

Suppose hypothetical regression data, measured on a substantively chosen scale, give $\hat{\beta}=2.00$ outcome units per predictor unit with $SE=0.80$. Compare two zero-centered priors, $N(0,4)$ and $N(0,1)$. Their prior standard deviations are 2 and 1 coefficient units, respectively; assume both ranges are defensible for the application.

For the $N(0,4)$ prior, the posterior variance and mean are $$ v_1=\left(\frac{1}{0.80^2}+\frac{1}{4}\right)^{-1} =\left(1.5625+0.25\right)^{-1} =\frac{16}{29}=0.5517, \qquad m_1=0.5517\left(\frac{2.00}{0.64}\right)=1.724. $$ Thus the posterior standard deviation is $\sqrt{0.5517}=0.7428$. The 95% equal-tailed credible interval is approximately $$ 1.724\pm1.96(0.7428)=[0.2683,3.180]. $$ The posterior probability that the slope is positive is $P(\beta>0\mid\text{data})=\Phi(1.724/0.7428)$, approximately 0.990.

For the $N(0,1)$ prior, the update gives $$ v_1=\left(1.5625+1\right)^{-1}=\frac{16}{41}=0.3902, \qquad m_1=0.3902(3.125)=1.220. $$ The posterior standard deviation is $0.6247$, and the 95% credible interval is $$ 1.220\pm1.96(0.6247)=[-0.0044,2.444]. $$ Here $P(\beta>0\mid\text{data})=\Phi(1.220/0.6247)$, approximately 0.975. Both priors yield a positive posterior mean and a high posterior probability of a positive slope, but the more concentrated prior pulls the estimated magnitude closer to zero and just barely places zero inside the 95% interval.

Compare Conclusions, Not Just Posterior Means

A sensitivity analysis is more informative when it examines several aspects of the conclusion. A posterior mean alone can conceal changes in uncertainty or sign probability. For each defensible prior, consider the coefficient’s posterior mean or median, a credible interval, and a posterior probability relevant to the claim, such as $P(\beta>0\mid\text{data})$. If a practical threshold matters, also report a probability such as $P(\beta>c\mid\text{data})$, where $c$ is specified in the coefficient’s units and has a substantive interpretation.

A credible interval is a posterior probability statement conditional on the model, data, and prior: a 95% posterior credible interval contains 95% of the posterior probability for the parameter. It is not a statement that there is a 95% chance that a fixed parameter lies in this particular interval under repeated sampling. When comparing intervals across priors, note both movement in their centers and changes in their widths or overlap with substantively important values.

Worked Example: A Weakly Informative Likelihood

Now consider a different hypothetical regression, with $\hat{\beta}=0.70$ and $SE=1.00$, in the same coefficient units. Compare priors $N(0,4)$ and $N(0,0.25)$, with standard deviations 2 and 0.5. Suppose each represents a plausible, though contrasting, degree of prior concentration around no effect.

Under $N(0,4)$, $$ v_1=\left(1+\frac{1}{4}\right)^{-1}=0.8000,\qquad m_1=0.8000(0.70)=0.5600. $$ The posterior standard deviation is $\sqrt{0.8}=0.8944$, so the 95% credible interval is $$ 0.5600\pm1.96(0.8944)=[-1.193,2.313]. $$ The posterior probability of a positive slope is $\Phi(0.5600/0.8944)$, approximately 0.734.

Under $N(0,0.25)$, $$ v_1=\left(1+4\right)^{-1}=0.2000,\qquad m_1=0.2000(0.70)=0.1400. $$ The posterior standard deviation is $0.4472$, giving the interval $$ 0.1400\pm1.96(0.4472)=[-0.7365,1.017]. $$ The posterior probability of a positive slope is $\Phi(0.1400/0.4472)$, approximately 0.623. Neither analysis gives strong evidence for a positive slope, and both intervals include zero. However, the estimated magnitude and sign probability vary appreciably with the prior. That variation is itself an important result: the data do not dominate the prior enough to make the sign conclusion insensitive.

Prior Choices Also Affect Predictions

A coefficient’s prior sensitivity can matter for predicted outcomes, not just for the coefficient summary. For a new observation at predictor value $x_*$, a posterior predictive distribution includes uncertainty about the regression parameters and, when predicting a new outcome rather than its conditional mean, the regression error. Consequently, prediction intervals need not change in exactly the same way as coefficient intervals.

For a normal model with known $\sigma^2$, independent normal posteriors for $\alpha$ and $\beta$, and a new independent error, the predictive mean at $x_*$ is $E(\alpha\mid\text{data})+x_*E(\beta\mid\text{data})$. Its predictive variance is $\operatorname{Var}(\alpha\mid\text{data})+x_*^2\operatorname{Var}(\beta\mid\text{data})+\sigma^2$, when the posterior covariance between $\alpha$ and $\beta$ is zero. If that covariance is nonzero, the variance also includes $2x_*\operatorname{Cov}(\alpha,\beta\mid\text{data})$.

Worked Example: Comparing Predictions Under the Two Slope Priors

Continue the first hypothetical regression, and suppose the centered design gives an intercept estimate of 10.00 outcome units with standard error 0.50. Let the common intercept prior be $N(0,100)$, independent of the slope prior, and let the known regression error standard deviation be 2.00 outcome units. Compare predictions at $x_*=1$ under the two slope priors from the first example.

The intercept posterior variance is $(1/0.50^2+1/100)^{-1}=1/4.01=0.2494$, and its mean is $0.2494(10/0.25)=9.975$. Under the $N(0,4)$ slope prior, the predictive mean for a new outcome is $9.975+1.724=11.699$, and its predictive variance is $$ 0.2494+0.5517+2.00^2=4.8011. $$ The predictive standard deviation is $2.191$, so the approximate 95% posterior predictive interval is $$ 11.699\pm1.96(2.191)=[7.405,15.994] $$ outcome units.

Under the $N(0,1)$ slope prior, the predictive mean is $9.975+1.220=11.195$, and the predictive variance is $$ 0.2494+0.3902+4.00=4.6396. $$ The predictive standard deviation is $2.154$, giving the approximate 95% posterior predictive interval $$ 11.195\pm1.96(2.154)=[6.973,15.416] $$ outcome units. The intervals overlap substantially, but their centers differ because the slope posterior differs. These intervals describe uncertainty about a future outcome, including regression error; they are not intervals for the conditional mean alone.

A Practical Sensitivity Workflow

For more complicated regressions, posterior distributions may be available only through MCMC draws. The comparison still follows the same logic: define the alternative priors, fit the same likelihood under each, verify that the resulting chains are adequate for the summaries, and compare posterior quantities using consistent definitions. Convergence and effective sample size matter because noisy Monte Carlo estimates can disguise or exaggerate differences between priors.

Specify plausible alternatives.
Use the coefficient’s econometric scale and substantive context to justify prior locations, scales, and any constraints. Avoid selecting alternatives only because they produce a desired conclusion.

Keep the analysis otherwise fixed.
Use the same observations, likelihood, covariates, and treatment of other parameters. If these also change, the comparison no longer isolates prior sensitivity.

Recompute relevant posterior summaries.
Compare coefficient magnitudes, credible intervals, sign or threshold probabilities, and predictions at substantively meaningful predictor values.

Describe the range of conclusions.
State which claims remain similar across priors and which change. If using simulation, consider Monte Carlo uncertainty in each reported summary before attributing small differences to the priors.

Common Pitfalls and Interpretation

A common mistake is to call one prior “weak” or “uninformative” without checking what it implies on the coefficient’s actual scale. A prior that appears broad in one parameterization may be concentrated after a change of units or predictor scaling. Prior alternatives should therefore be interpreted in the units readers use. If the predictor is rescaled, transform the prior accordingly when the intention is to represent the same substantive belief.

Another mistake is to change the prior and the model at the same time, then attribute every posterior difference to prior choice. Sensitivity analysis asks what follows from changing a specified prior while keeping the likelihood model and data fixed. It does not by itself address whether the likelihood model is adequate, whether covariates should be included, or how to average over different models. Those are distinct questions.

Finally, prior sensitivity is not a reason to select whichever posterior summary looks most decisive. Report the alternatives and their consequences transparently. A conclusion that is stable across plausible priors is more robust to prior specification than one that reverses or changes substantially. A prior-sensitive conclusion should be presented with that qualification, rather than as though the data alone determined it.

Key takeaway: Recompute the posterior under substantively plausible priors and compare the quantities that support the regression claim. Stability across those choices supports a robust conclusion; meaningful variation signals that the prior assumptions remain consequential.

Check Your Understanding

Answer these questions using the definitions and calculations above.

In a prior sensitivity analysis, which elements of the regression analysis should remain fixed when the prior is changed?
Why does a more concentrated normal prior have greater influence on the posterior mean in the normal-normal update?
In the first worked example, how did the 95% credible interval change when the slope prior standard deviation decreased from 2 to 1?
What does a posterior probability such as $P(\beta>0\mid\text{data})$ mean, and what does it not mean?
Why is a posterior predictive interval for a new outcome generally wider than an interval that accounts only for uncertainty in the conditional mean?