Proper and Improper Priors for Variance Components

Why Propriety Matters

A variance-component prior describes uncertainty about how much groups differ. When the data contain little information about that variability, the prior can have a substantial influence on the posterior. Before interpreting posterior summaries, however, a more basic question must be answered: does the posterior distribution have a finite normalizing constant? If not, it is not a probability distribution, and posterior means, credible intervals, and simulation-based summaries do not have their usual meanings.

This question is especially important in weakly informed GLMMs: for example, when there are few groups, little within-group replication, or outcome patterns that provide little information about between-group variation. A prior may be deliberately left unnormalized, but then propriety of the posterior must be established rather than assumed.

Definition: A prior density $p(\theta)$ is proper if its integral over the parameter space is finite and equals one. A nonnegative prior kernel whose integral is infinite is called improper. Given likelihood $L(\theta;y)$, the posterior is proper exactly when its normalizing constant $$ m(y)=\int L(\theta;y)p(\theta)\,d\theta $$ is finite and strictly positive. For an improper prior, this integral is evaluated using its specified kernel; the arbitrary multiplicative constant cancels from the posterior only if the integral is finite.

Here $\theta$ includes the variance component and any other unknown parameters. Propriety is a property of the joint posterior, not just of the marginal prior chosen for one variance. An improper prior on a fixed effect, intercept, or another variance component can also affect the normalizing integral.

Recognizing Proper and Improper Variance Priors

For a random-intercept GLMM, let $b_j$ denote the random intercept for group $j$, with $b_j\sim N(0,v)$, where $v=\tau^2>0$ is the variance component. A prior must be integrated over its full parameter space to determine whether it is proper. The fact that a density is finite at each individual value does not establish that its total integral is finite.

Worked Example: Checking Prior Integrals

Compare two candidate priors for $v>0$: an exponential prior with rate 1, $p(v)=e^{-v}$, and the kernel $q(v)=1/(1+v)$.

For the exponential prior, $$ \int_0^\infty e^{-v}\,dv =\left[-e^{-v}\right]_0^\infty =1. $$ It is proper and already normalized.

For the second kernel, $$ \int_0^\infty \frac{1}{1+v}\,dv =\left[\log(1+v)\right]_0^\infty =\infty. $$ It is improper. In particular, its integral diverges in the upper tail; it is not a probability distribution and cannot be used to compute ordinary prior probabilities without further modification.

Both expressions are nonnegative, and both have finite values at every finite $v>0$. Those facts are not enough: the integral over the entire variance space decides propriety.

A prior can also be proper while concentrating probability in regions that may be implausible for the application. Propriety is a mathematical normalization condition, not a guarantee of good calibration or useful information. Conversely, an improper prior is not automatically unusable for posterior estimation: in some models and with some data, the likelihood makes the posterior integral finite. That must be checked for the specified model and data.

A Weakly Informed GLMM Where the Posterior Fails to Normalize

Consider one group with $m$ binary observations and a known fixed intercept of zero. Conditional on the group random intercept $b$, suppose $Y_i$ are independent Bernoulli variables with $$ P(Y_i=1\mid b)=\operatorname{logit}^{-1}(b), \qquad b\sim N(0,v). $$ Suppose all $m$ observations are successes. After integrating out $b$, the likelihood for $v$ is $$ L(v;y)=E\left[\operatorname{logit}^{-1}(B)^m\right], \qquad B\sim N(0,v). $$ The group outcome can be explained by a large positive random intercept, but with only one group the data provide limited information about how much variation there is across groups.

Worked Example: An Improper Posterior in a One-Group Model

Use the model above with $m=3$ successes and the improper prior kernel $q(v)=1/(1+v)$. The posterior normalizing integral is $$ Z=\int_0^\infty L(v;y)\frac{1}{1+v}\,dv. $$ To check its upper tail, write $B=\sqrt{v}Z_0$, where $Z_0\sim N(0,1)$. As $v$ tends to infinity, $\operatorname{logit}^{-1}(\sqrt{v}Z_0)^3$ tends to 1 when $Z_0>0$, and to 0 when $Z_0<0$. Since this quantity is bounded between 0 and 1, dominated convergence gives $$ \lim_{v\to\infty}L(v;y)=P(Z_0>0)=\frac12. $$ Thus, for sufficiently large $v$, $L(v;y)$ is bounded below by a positive constant, such as $1/4$. Consequently, the tail of $Z$ is bounded below by $$ \int_M^\infty \frac{1}{4(1+v)}\,dv=\infty $$ for any sufficiently large $M$. The posterior is therefore improper.

The likelihood does not decay quickly enough to offset the prior's nonintegrable upper tail. This example does not say that one group always makes every posterior improper. It establishes that this particular likelihood, data pattern, and prior kernel do not produce a normalized posterior.

This argument illustrates a general test: examine the likelihood and prior together in every potentially problematic direction. A prior that is improper in a tail may lead to a proper posterior if the likelihood decays sufficiently rapidly there; if the likelihood stays bounded away from zero along a tail with infinite prior mass, the posterior integral diverges.

An Improper Prior Can Produce a Proper Posterior

The reverse outcome is possible. Keep the one-group model with known intercept zero, but observe one success and one failure in that group. Given $b$, the likelihood is $\operatorname{logit}^{-1}(b)\{1-\operatorname{logit}^{-1}(b)\}$. The opposing outcomes are less compatible with an increasingly extreme group intercept, so the integrated likelihood decreases as $v$ grows.

Worked Example: A Proper Posterior From an Improper Tail Prior

For one success and one failure, let $$ L(v;y)=\int_{-\infty}^{\infty} \operatorname{logit}^{-1}(b)\{1-\operatorname{logit}^{-1}(b)\} \frac{1}{\sqrt{2\pi v}}\exp\left(-\frac{b^2}{2v}\right)\,db. $$ Use the same prior kernel $q(v)=1/(1+v)$. For $0<v\leq 1$, the likelihood is a probability and hence $L(v;y)\leq 1$. Therefore, $$ \int_0^1 L(v;y)\frac{1}{1+v}\,dv \leq \int_0^1 \frac{1}{1+v}\,dv =\log 2<\infty. $$ For $v\geq 1$, the normal density in the integral is at most $1/\sqrt{2\pi v}$. Also, $$ \int_{-\infty}^{\infty}\operatorname{logit}^{-1}(b) \{1-\operatorname{logit}^{-1}(b)\}\,db=1, $$ because the integrand is the derivative of $\operatorname{logit}^{-1}(b)$. It follows that $L(v;y)\leq 1/\sqrt{2\pi v}$, and so $$ \int_1^\infty L(v;y)\frac{1}{1+v}\,dv \leq \frac{1}{\sqrt{2\pi}}\int_1^\infty v^{-3/2}\,dv =\frac{2}{\sqrt{2\pi}}<\infty. $$ The full normalizing integral is finite. It is also strictly positive, since the likelihood and prior kernel are positive for $v>0$. Thus this improper prior gives a proper posterior for these data and this model.

This calculation is specific to the stated likelihood and data. It does not justify using the same improper prior in a different GLMM, or with additional unknown parameters whose own tails could make the joint integral diverge.

Proper Priors, Likelihood Bounds, and Their Limits

A useful sufficient condition applies in many discrete-outcome GLMMs. If the joint prior over all unknown parameters is proper, the likelihood is bounded above by a finite constant $C$, and the likelihood is positive on a set with positive prior probability, then $$ 0<m(y)=\int L(\theta;y)p(\theta)\,d\theta \leq C\int p(\theta)\,d\theta=C<\infty. $$ The posterior is proper under these conditions. For a finite Bernoulli or binomial data set, the likelihood as a probability is at most 1. This makes proper priors a practical safeguard against one important source of posterior impropriety, provided the full joint prior is proper.

This bound is not a universal theorem that a proper prior always yields a proper posterior. The likelihood may be unbounded in some models or parameter regions; normal models with unknown residual scale can have this issue as a variance approaches a boundary. The integral must be checked under the likelihood actually being used. A proper prior also does not repair model misspecification, weak scientific justification, or poor prior calibration.

Worked Example: A Proper Prior With No Variance Information

Return to a single binary observation $Y=1$ in one group, with known intercept zero and $B\sim N(0,v)$. Give $v$ the proper exponential prior $p(v)=e^{-v}$, $v>0$. The marginal likelihood is $$ L(v;Y=1)=E\left[\operatorname{logit}^{-1}(B)\right]=\frac12. $$ To see the equality, the distribution of $B$ is symmetric about zero and $\operatorname{logit}^{-1}(b)+\operatorname{logit}^{-1}(-b)=1$. The evidence is therefore $$ m(Y=1)=\int_0^\infty \frac12 e^{-v}\,dv =\frac12. $$ The posterior density is $$ p(v\mid Y=1)=\frac{(1/2)e^{-v}}{1/2}=e^{-v},\qquad v>0. $$ It is proper, but it is exactly the prior: this observation provides no information about $v$ in this model. Propriety guarantees a normalized posterior, not that the data have meaningfully updated it.

Checking Propriety in Practice

When a variance-component prior is improper, a responsible analysis needs an argument about the full posterior integral. A computational routine that returns finite estimates is not that argument. Markov chain Monte Carlo methods can produce apparently stable traces over a finite run even when the target has infinite mass; convergence diagnostics assess simulation behavior, not whether a normalizing constant exists.

Write down the joint kernel.
Include the likelihood, the variance-component prior, and priors for every other unknown parameter. State the parameter domain and the parameterization, such as variance $v$ or standard deviation $\tau$.

Identify weakly informed directions.
Consider large variance, small variance, extreme fixed effects, and combinations of parameters that leave fitted probabilities nearly unchanged. Sparse groups, few clusters, and separated binary outcomes can make some directions especially weakly informed.

Bound or evaluate each tail.
Show that the joint kernel has a finite integral in each problematic region, or show that it has a divergent lower bound. If analytic work is difficult, numerical integration can reveal warning signs but does not prove finiteness over an unbounded domain.

Use a proper alternative when needed.
Choose a scientifically defensible proper prior and examine its implications. A proper prior makes the prior mass finite; under a bounded likelihood and the positivity conditions above, it also gives a proper posterior.

An improper prior may be used for posterior estimation if propriety has been established, but its arbitrary multiplicative constant creates a further limitation: marginal likelihoods and Bayes factors involving that prior are not defined without a specified normalization. A proper prior avoids that particular ambiguity. Proper priors and improper priors are therefore alternatives with different requirements, not interchangeable labels for the same analysis. In either case, the prior should be assessed for its substantive implications; prior predictive checks are useful for that purpose.

Common Mistakes and Practical Limits

Assuming the likelihood automatically fixes an improper prior. It may, but the one-group all-success example shows that the likelihood can approach a positive limit while prior mass remains infinite.
Assuming an improper prior always makes the posterior improper. The mixed-outcome example gives a finite posterior integral despite an improper upper-tail prior.
Treating a proper posterior as evidence of strong information. A normalized posterior can equal the prior, as in the one-observation example.
Using successful computation as a propriety test. Finite chains cannot establish that the target distribution has finite total mass.
Checking only the variance marginal. Propriety concerns the joint integral; other improper priors or weakly informed parameters may cause failure.

Key takeaway: Posterior propriety requires a finite, positive integral of the likelihood times the full prior kernel. Improper variance-component priors require a model- and data-specific justification; a proper prior is a useful safeguard, but it does not by itself guarantee good information or a well-behaved analysis.

Check Your Understanding

Answer each question, then compare with the concise solutions below.

What condition on the normalizing integral defines a proper posterior?
Why is the kernel $1/(1+v)$ on $v>0$ improper?
In the one-group all-success example, what feature of the likelihood causes the posterior integral to diverge under that kernel?
Can an improper prior produce a proper posterior? What must be shown?
Does a proper posterior imply that the data informed the variance component?

Solutions. (1) The integral of the likelihood times the prior kernel must be finite and strictly positive. (2) Its integral is $\log(1+v)$ evaluated from zero to infinity, which diverges. (3) The integrated likelihood tends to $1/2$, so it does not offset the kernel's infinite upper-tail mass. (4) Yes; the full likelihood-prior integral must be shown finite and positive for the specified model and data. (5) No; a posterior can be proper yet equal the prior when the likelihood provides no information about that parameter.