Eliciting Regression Priors On Econometric Scales

Why Elicit Priors on Econometric Scales?

A coefficient is not meaningful apart from its units. A prior that says a coefficient is probably between -1 and 1 could be weakly informative if the predictor is measured in thousands of dollars, but extremely concentrated if the predictor is measured in single dollars. Prior elicitation therefore begins with the substantive question: what change in the outcome is plausible for a meaningful change in the predictor?

In Bayesian regression, this question is answered before observing the outcome data. A prior should express defensible knowledge or uncertainty about coefficient values, and a prior predictive check should reveal the outcomes those choices imply. The aim is not to make the prior fit the observed sample. It is to make its scale and implications transparent enough to assess.

Model, Units, and Coefficient Meaning

For clarity, consider a linear regression with one predictor, observed outcomes $y_i$, predictor values $x_i$, and conditionally independent normal errors:

y_i=\alpha+\beta x_i+\varepsilon_i,\qquad \varepsilon_i\mid \sigma^2\sim N(0,\sigma^2).

Here $\alpha$ is the conditional mean outcome at $x_i=0$, $\beta$ is the change in the conditional mean outcome for a one-unit increase in $x_i$, and $\sigma$ is the conditional error standard deviation. The likelihood assumes the specified linear conditional mean and, for this model, normal errors with a common variance. A prior is then placed on the unknown parameters; for example, one might use independent normal priors for $\alpha$ and $\beta$, together with a separate prior for $\sigma$.

Definition: A coefficient prior on an econometric scale is a probability distribution for a regression coefficient whose location and spread are chosen by reasoning about changes in the outcome for specified, substantively meaningful changes in the predictor. A prior predictive distribution is the distribution of outcomes obtained by drawing parameters from their prior and then drawing outcomes from the model conditional on those parameters.

The coefficient’s units are outcome units per predictor unit. If income is recorded in dollars, a coefficient might be dollars of outcome per dollar of income; if income is recorded in thousands of dollars, it is outcome units per thousand dollars. These are different numerical coefficients describing the same relationship. A prior specification must match the chosen coding and units.

Worked Example: Converting a Belief to a Coefficient Scale

Suppose a hypothetical model relates annual household spending, in dollars, to annual income. An increase of $10,000 in income is thought likely to be associated with an increase of about $600 in spending, with substantial uncertainty. The predictor is recorded in thousands of dollars, so a one-unit increase means $1,000 more income.

The implied coefficient center is $600/10=60$ dollars of spending per $1,000 of income. Suppose the substantive judgment is that a change of $10,000 in income would usually correspond to a spending change between $0 and $1,200. On the coefficient scale, that means a change per $1,000 between $0 and $120. A simple initial prior is $\beta\sim N(60,30^2)$: its central 95% interval is approximately $60\pm1.96(30)$, or $1.2$ to $118.8$ dollars per $1,000.

This prior is not uniquely implied by the judgment. It is a transparent approximation whose implications can be checked. In particular, its 95% interval is about $12 to $1,188 in spending per $10,000 of income, close to the stated substantive range. The small difference arises because a normal distribution assigns some probability outside any finite interval.

Changing Predictor Units or Centering

Changing units changes the numerical coefficient and requires changing its prior accordingly. If $x^*=x/c$, then $x=cx^*$, so the same model can be written with coefficient $\beta^*=c\beta$. Thus, if $\beta\sim N(m,s^2)$, the equivalent prior is $\beta^*\sim N(cm,c^2s^2)$. Keeping the same numerical prior after rescaling the predictor would describe a different substantive belief.

Centering also changes the intercept. If $x^*=x-c$, then $x=x^*+c$, and

\alpha+\beta x=(\alpha+c\beta)+\beta x^*.

The slope is unchanged, but the new intercept is $\alpha^*=\alpha+c\beta$: the expected outcome at the original predictor value $x=c$. If $\alpha$ and $\beta$ have independent priors, $\alpha^*$ and $\beta$ will generally be dependent. This is one reason to choose a useful predictor origin before assigning intercept priors.

Worked Example: Rescaling and Centering a Predictor

Suppose a hypothetical model uses age in years and has $y=\alpha+\beta x+\varepsilon$, where $y$ is a health expenditure measure in dollars. The prior is $\beta\sim N(4,2^2)$, so the expected change is $4 per additional year and the prior standard deviation is $2 per year.

If age is instead recorded in decades, let $x^*=x/10$. The equivalent coefficient is $\beta^*=10\beta$, so $\beta^*\sim N(40,20^2)$. The expected change per decade is $40, and its prior standard deviation is $20. Using $N(4,2^2)$ for the per-decade coefficient would instead imply a much smaller effect.

Now center age at 50 years by defining $z=x-50$. Then $y=(\alpha+50\beta)+\beta z+\varepsilon$. The slope prior is still $N(4,2^2)$, while the intercept now represents expected expenditure at age 50. If $\alpha$ and $\beta$ were independent, the centered intercept $\alpha+50\beta$ would have covariance $50\operatorname{Var}(\beta)=50(4)=200$ with $\beta$. Centering therefore changes the joint prior structure as well as the intercept’s interpretation.

Encoding Magnitude and Sign Beliefs

A normal prior is often a convenient starting point for an unconstrained coefficient: $\beta\sim N(m,s^2)$. Its location $m$ represents the prior center, while $s$ controls spread. If prior judgments are expressed through a plausible interval, a rough calibration is to choose $m$ as the interval midpoint and $s$ so that the interval covers a stated amount of prior probability. For a normal prior, approximately 95% of the mass lies between $m-1.96s$ and $m+1.96s$.

Sign beliefs should be expressed as probabilities, not just words. For example, a normal prior with positive mean assigns more than half its mass to positive coefficients, but it does not rule out negative values. If $\beta\sim N(m,s^2)$, then

P(\beta>0)=\Phi\left(\frac{m}{s}\right),

where $\Phi$ is the standard normal cumulative distribution function. This calculation makes the strength of the sign belief explicit. A prior that must assign zero probability to one sign requires a distribution with restricted support, such as a truncated distribution; that is a stronger modeling choice than merely expressing a high probability of a sign.

Worked Example: Checking the Sign Probability

Suppose a hypothetical analyst expects a policy coefficient to be positive and specifies $\beta\sim N(0.4,0.2^2)$, with the coefficient measured in percentage points of employment per policy unit. The implied probability of a positive coefficient is

P(\beta>0)=\Phi\left(\frac{0.4}{0.2}\right)=\Phi(2)\approx0.9772.

Thus the prior assigns about 97.7% probability to a positive coefficient and about 2.3% to a negative one. Its approximate 95% interval is $0.4\pm1.96(0.2)$, or $0.008$ to $0.792$ percentage points per policy unit. If the analyst intended only a mild preference for a positive sign, this prior may be too concentrated on positive values; if a sign reversal is genuinely thought nearly impossible, the remaining negative mass should still be considered rather than silently ignored.

Prior Predictive Simulation

Coefficient priors are difficult to evaluate in isolation. A modest-looking slope can generate extreme outcomes when multiplied by a predictor far from zero, or when combined with a diffuse intercept and error distribution. Prior predictive simulation translates parameter choices back to the scale of the outcome.

For a chosen predictor value $x$, the procedure draws parameters from their joint prior and then draws a possible outcome from the model. Repeating this gives a distribution of outcomes before using the observed outcomes to update beliefs. For the normal linear model above, if $\alpha$, $\beta$, and $\varepsilon$ are mutually independent with $\alpha\sim N(a,A^2)$, $\beta\sim N(m,s^2)$, and $\varepsilon\sim N(0,\sigma^2)$ for fixed $\sigma$, then

Y\mid x \sim N\left(a+mx,\ A^2+x^2s^2+\sigma^2\right).

The mean follows by adding the means of $\alpha$, $\beta x$, and $\varepsilon$. Independence makes their variances additive, giving $A^2+x^2s^2+\sigma^2$. With uncertain $\sigma$, or dependent coefficient priors, the predictive distribution need not be normal; simulation remains available by drawing jointly from the specified prior and likelihood.

Choose predictor values.
Use values representing meaningful cases, including values near the edges of the range where the model will be used.

Draw prior parameters.
For each simulation, draw the intercept, slopes, and error-scale parameter from their joint prior, respecting any prior dependence.

Draw outcomes from the model.
At each selected predictor value, calculate the conditional mean and draw an outcome using the stated error model.

Inspect ranges and patterns.
Summarize simulated outcomes with quantiles, plots, or domain-specific thresholds; investigate implausible values and trends before fitting the model.

Worked Example: Simulating Outcomes at Two Predictor Values

Consider a hypothetical model for a test score, with scores measured in points and $x$ denoting weekly study hours:

Y=\alpha+\beta x+\varepsilon,\quad \alpha\sim N(50,5^2),\quad \beta\sim N(2,0.5^2),\quad \varepsilon\sim N(0,4^2).

Assume these three quantities are independent and that the error standard deviation is fixed at 4 points. At $x=10$, the prior predictive mean is $50+2(10)=70$ points. Its variance is $5^2+10^2(0.5^2)+4^2=25+25+16=66$, so its standard deviation is $\sqrt{66}\approx8.124$ points. The approximate central 95% range is $70\pm1.96(8.124)$, or 54.1 to 85.9 points.

At $x=20$, the mean is $50+2(20)=90$ points, while the variance is $25+20^2(0.25)+16=141$, with standard deviation $\sqrt{141}\approx11.87$. The approximate central 95% range is $90\pm1.96(11.87)$, or 66.7 to 113.3 points. The upper end exceeds the possible score of 100. That may indicate that the prior or the linear-normal outcome model is unsuitable for this outcome; it should not be dismissed merely because the data have not yet been used. A simulation with many prior draws would show the same issue as a nontrivial fraction of impossible scores above 100.

What a Prior Predictive Check Can and Cannot Tell You

A prior predictive check asks whether the model and priors jointly generate outcomes that are plausible before seeing the sample. It can expose coefficient scales that imply implausibly large effects, intercept priors that place typical outcomes in the wrong region, or error scales that allow excessive variation. Checks should be made at several predictor values, not only at $x=0$, because slope uncertainty grows in its contribution to predictive variance as $|x|$ increases.

The check does not establish that the model is correct, nor does it test whether the observed data fit. It is a diagnostic of prior implications. If the implications are unreasonable, revise the substantive specification, the predictor coding, the prior, or the outcome model, and then examine the consequences again.

Common Mistake: Treating a Numerical Prior as Scale-Free

A common error is to say that $\beta\sim N(0,1)$ is “weakly informative” without stating the units of $x$ and $y$. The correction is to state what a one-unit increase in $x$ means, translate plausible outcome changes into coefficient values, and check predictive outcomes over the range of $x$ that matters. A second error is to interpret a prior sign probability as certainty: $P(\beta>0)=0.977$ still leaves 2.3% probability on negative values.

Key takeaway: Elicit coefficient priors in the units of the model, transform them when predictors are rescaled or centered, and use prior predictive simulation to assess the joint implications for outcomes. A prior is interpretable only together with its parameterization and model.

Check Your Understanding

Answer each question by linking the prior to the model’s units and implications.

A predictor changes from dollars to thousands of dollars. How must the slope prior’s mean and variance change to preserve the same substantive belief?
If $\beta\sim N(0.3,0.15^2)$, what is the formula for $P(\beta>0)$, and what does that probability describe?
Why can centering a predictor change the relationship between the intercept and slope priors?
What quantities are drawn in a prior predictive simulation, and why are outcomes examined at more than one predictor value?
A prior predictive simulation for a bounded outcome frequently produces values outside its possible range. Name two aspects of the specification that may need reconsideration.