Posterior Summaries For Coefficients And Marginal Effects

From a Posterior Distribution to an Econometric Summary

A fitted Bayesian regression produces a posterior distribution, not just a single coefficient estimate. That distribution describes uncertainty about the coefficients conditional on the specified model, prior, and observed data. Posterior summaries compress it into quantities that are easier to interpret, while retaining different aspects of its shape.

Let \(\boldsymbol{\beta}\) denote a vector of regression coefficients and \(y\) the observed outcomes. The posterior distribution is written \(p(\boldsymbol{\beta}\mid y)\). If the posterior is represented by \(S\) draws \(\boldsymbol{\beta}^{(1)},\ldots,\boldsymbol{\beta}^{(S)}\), a summary can be calculated directly from those draws. The draws might come from an analytically available posterior or a simulation method; the calculations below treat them as representative draws from the posterior.

Definition: A posterior summary is a numerical description of a posterior distribution conditional on the model, prior, and observed data. A posterior mean averages a coefficient over that distribution; a posterior median divides its probability mass in half; a credible interval or region contains a stated amount of posterior probability.

Posterior Means, Medians, and Credible Intervals

For coefficient \(\beta_j\), the posterior mean is \(E(\beta_j\mid y)\), and the posterior median is a value with at least half the posterior probability on either side. With \(S\) draws, the posterior mean is estimated by their arithmetic average:

\widehat{E}(\beta_j\mid y)=\frac{1}{S}\sum_{s=1}^{S}\beta_j^{(s)}.

The sample median of the draws estimates the posterior median. The mean uses all draw values and can be pulled toward a long tail; the median is less sensitive to extreme values. They coincide for a symmetric posterior, but need not do so for a skewed one.

A \(100(1-\alpha)\%\) equal-tailed credible interval places \(\alpha/2\) posterior probability below its lower endpoint and \(\alpha/2\) above its upper endpoint. Its endpoints are the posterior quantiles at \(\alpha/2\) and \(1-\alpha/2\). For example, a 95% equal-tailed interval uses the 2.5th and 97.5th percentiles. A highest-posterior-density (HPD) region is a region containing the specified posterior probability whose points have posterior density at least as high as points outside it. For a unimodal distribution, the HPD region is often a single interval; for a multimodal distribution it can consist of separate pieces.

The interpretation is conditional: given the specified model, prior, and observed data, the posterior probability that the coefficient lies in a 95% credible interval is 0.95. This is not a claim that the fixed coefficient has a 95% probability of being in the interval under repeated sampling.

Worked Example: Summarizing a Slope Posterior

Suppose a hypothetical Gaussian regression has a continuous outcome measured in thousands of dollars and a predictor measuring years of experience. After specifying the likelihood and prior and conditioning on the observed data, consider these ten illustrative posterior draws for the experience slope, in thousands of dollars per year:

\(0.4,\ 0.6,\ 0.7,\ 0.8,\ 0.9,\ 1.0,\ 1.1,\ 1.2,\ 1.4,\ 1.9.\)

Posterior mean. The draw average is

\widehat{E}(\beta_{\text{experience}}\mid y) =\frac{0.4+0.6+0.7+0.8+0.9+1.0+1.1+1.2+1.4+1.9}{10} =\frac{10.0}{10}=1.00.

Posterior median. With ten ordered draws, the median is the average of the fifth and sixth values: \((0.9+1.0)/2=0.95\) thousand dollars per year.

Illustrative equal-tailed interval. Using the empirical quantile convention that the \(q\)-quantile is the smallest draw whose cumulative empirical probability is at least \(q\), the 10th percentile is 0.4 and the 90th percentile is 1.4. Thus the draw-based 80% equal-tailed interval is \([0.4,1.4]\) thousand dollars per year. With only ten draws this is a coarse approximation; more draws would generally be needed for stable tail quantiles.

The posterior mean slope is 1.00 thousand dollars per additional year of experience, under this model and conditional on these data. The interval describes uncertainty in that slope, not the range of individual outcomes.

Joint Posterior Regions

Separate intervals describe one coefficient at a time. When the question concerns two or more coefficients together, use a joint credible region. A 95% joint region contains 95% of the posterior probability for the coefficient vector as a whole. It is not equivalent to putting a separate 95% interval around every coefficient: the probability of lying in all those intervals simultaneously is generally not 95%.

For a bivariate normal posterior with mean vector \(\boldsymbol{\mu}\) and covariance matrix \(\boldsymbol{\Sigma}\), a 95% credible ellipse is defined by a quadratic distance from the mean. In two dimensions the cutoff is the 95th percentile of a chi-squared distribution with 2 degrees of freedom, approximately 5.991:

(\boldsymbol{\beta}-\boldsymbol{\mu})^\mathsf{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{\beta}-\boldsymbol{\mu}) \leq 5.991.

The covariance matrix matters: it accounts for posterior dependence between the coefficients and determines the ellipse’s orientation and shape. For a non-normal posterior, a joint region can instead be constructed from posterior draws, for example by finding a region of high posterior density that contains the desired probability. Its exact construction depends on the posterior’s shape.

Worked Example: Checking a Point Against a Joint Region

Suppose a hypothetical two-coefficient posterior is bivariate normal with mean vector \((0.50,0.20)\) and covariance matrix

\boldsymbol{\Sigma}= \begin{pmatrix} 0.04 & 0\\ 0 & 0.01 \end{pmatrix}.

The posterior standard deviations are 0.20 and 0.10, and the covariance is zero. The 95% joint ellipse is therefore

\left(\frac{\beta_1-0.50}{0.20}\right)^2+ \left(\frac{\beta_2-0.20}{0.10}\right)^2 \leq 5.991.

For the point \((0.80,0.25)\), the left-hand side is \((0.30/0.20)^2+(0.05/0.10)^2=1.5^2+0.5^2=2.50\). Since \(2.50<5.991\), the point lies inside the 95% joint region.

For \((0.95,0.30)\), it is \((0.45/0.20)^2+(0.10/0.10)^2=2.25^2+1^2=6.0625\). Since \(6.0625>5.991\), that point lies outside the region. These are geometric checks against the specified posterior region, not tests of a null hypothesis.

Transforming Draws for Nonlinear Marginal Effects

A marginal effect is the change in a model’s conditional mean associated with a small change in a predictor, holding other covariates fixed. In a linear model without interactions, a slope coefficient itself is the marginal effect. In a nonlinear model, the effect usually depends on covariate values and on several coefficients. In that case, compute the effect for each posterior draw, then summarize the resulting effect draws.

For example, let \(Y_i\) be binary and suppose its conditional probability follows a logistic regression:

P(Y_i=1\mid x_i,z_i,\boldsymbol{\beta}) =\Lambda(\beta_0+\beta_1x_i+\beta_2z_i), \qquad \Lambda(u)=\frac{1}{1+\exp(-u)}.

Assume the outcomes are conditionally independent Bernoulli observations given the covariates and coefficients, and that this logistic specification is the model being summarized. Differentiating the conditional probability with respect to the continuous predictor \(x_i\) gives:

\frac{\partial P(Y_i=1\mid x_i,z_i,\boldsymbol{\beta})}{\partial x_i} =\beta_1\Lambda(\eta_i)\{1-\Lambda(\eta_i)\}, \qquad \eta_i=\beta_0+\beta_1x_i+\beta_2z_i.

Because this expression is nonlinear in the coefficients, substituting posterior mean coefficients generally does not produce the posterior mean marginal effect. The appropriate draw-based calculation evaluates the derivative at each coefficient draw, for the covariate values of interest. The resulting effect draws approximate the posterior distribution of the marginal effect.

Worked Example: A Logistic Marginal Effect at Fixed Covariates

Consider five hypothetical draws from the posterior of a logistic model. We want the effect of a one-unit increase in \(x\) at \(x=1,z=0\). For each draw, calculate \(\eta=\beta_0+\beta_1\), then \(\beta_1\Lambda(\eta)\{1-\Lambda(\eta)\}\).

Draw	\(\beta_0\)	\(\beta_1\)	\(\eta\)	Marginal effect
1	-1.0	0.8	-0.2	0.1980
2	-0.8	1.0	0.2	0.2475
3	-1.2	1.2	0	0.3000
4	-0.6	0.6	0	0.1500
5	-1.4	1.4	0	0.3500

For draw 1, \(\Lambda(-0.2)\approx0.4502\), so the marginal effect is \(0.8(0.4502)(1-0.4502)=0.1980\). For draw 2, \(\Lambda(0.2)\approx0.5498\), giving \(1.0(0.5498)(0.4502)=0.2475\). For draws 3–5, \(\eta=0\), so \(\Lambda(0)=0.5\). Their effects are respectively \(1.2(0.5)(0.5)=0.3000\), \(0.6(0.5)(0.5)=0.1500\), and \(1.4(0.5)(0.5)=0.3500\).

The mean effect is \((0.1980+0.2475+0.3000+0.1500+0.3500)/5=1.2455/5=0.2491\). The median is 0.2475. Under the empirical quantile convention used above, the 10th and 90th percentiles are 0.1500 and 0.3500, giving an illustrative 80% equal-tailed interval of \([0.1500,0.3500]\). With only five draws, this interval is very coarse.

At these covariate values, the posterior mean marginal effect is approximately 0.249 probability points per one-unit increase in \(x\), conditional on the model and data. It is a change in the model-implied conditional probability, not a causal effect unless the design and assumptions support a causal interpretation.

Why Transform Draws Rather Than Only Point Estimates?

In general, applying a nonlinear function to a posterior mean is not the same as averaging that function over the posterior. For the logistic effect, the two calculations are:

g\{E(\boldsymbol{\beta}\mid y)\} \quad\text{and}\quad E\{g(\boldsymbol{\beta})\mid y\},

where \(g\) maps coefficients to the marginal effect at specified covariates. They need not agree because \(g\) is nonlinear. Transforming each draw captures how uncertainty in every relevant coefficient jointly affects the effect, including coefficient dependence. It also allows the effect posterior to be skewed or otherwise unlike a normal distribution.

For the same reason, report the covariate values at which a nonlinear marginal effect is evaluated, or state how effects are averaged across observations. A marginal effect at a chosen profile and an average marginal effect over a population are different estimands. Their units should also be explicit: in the example, probability points per one-unit increase in \(x\).

Common Mistakes

Calling a credible interval a confidence interval. A Bayesian credible interval summarizes posterior probability conditional on model, prior, and data; its interpretation differs from the repeated-sampling interpretation of a frequentist confidence interval.
Reading marginal intervals as a joint statement. Two separate 95% intervals do not generally enclose 95% posterior probability jointly. Use a joint region when the claim concerns coefficients together.
Evaluating only at mean coefficients. For a nonlinear effect, the effect at posterior mean coefficients generally differs from the posterior mean effect. Transform each draw and summarize those transformed values.
Reporting an effect without its evaluation point or units. State the covariate profile or averaging procedure and describe the effect in context.

Key takeaway: Posterior means and medians summarize coefficient location; credible intervals and joint regions summarize uncertainty. For nonlinear marginal effects, apply the effect formula draw by draw before calculating summaries.

Check Your Understanding

Try these questions using the definitions and calculations above.

What posterior quantity does the arithmetic average of coefficient draws estimate?
How does a 95% equal-tailed credible interval divide posterior probability between its two tails?
Why are two separate 95% marginal intervals not generally a 95% joint credible region?
For a logistic model, why can the marginal effect at posterior mean coefficients differ from the posterior mean marginal effect?
What additional information should accompany a reported nonlinear marginal effect?