Jacobians for Prior Transformations of Variance Components

Why the Parameter Scale Matters

A random-effect distribution can be described by its variance, standard deviation, precision, or log-standard deviation. These are different coordinates for the same underlying quantity, but a probability density is defined relative to a particular coordinate. Consequently, writing the same-looking prior formula on two different scales generally does not describe the same prior distribution.

For example, a prior density on a random-effect standard deviation cannot simply be reused as a density on its log standard deviation. The change of variables contributes a Jacobian factor. This adjustment preserves probabilities when a density is expressed using a new coordinate; it does not change the probability model or add new prior information.

Consider a GLMM whose random effect $b_j$, conditional on a variance component $v$, has distribution $b_j\mid v\sim N(0,v)$. Define the standard deviation $\tau=\sqrt{v}$, the precision $\lambda=1/v$, and the log standard deviation $\eta=\log(\tau)$. The parameters satisfy $v>0$, $\tau>0$, $\lambda>0$, and $\eta\in\mathbb{R}$. The same principles apply to each scalar variance component in a model.

Definition: If a continuous parameter $\theta$ has density $p_\theta(\theta)$, and $\phi=g(\theta)$ is a one-to-one differentiable transformation, then the density on the $\phi$ scale is $$ p_\phi(\phi)=p_\theta\!\left(g^{-1}(\phi)\right)\left|\frac{d\,g^{-1}(\phi)}{d\phi}\right|. $$ The absolute derivative is the Jacobian for this one-dimensional transformation. It adjusts the density so that probabilities over corresponding intervals are unchanged.

Deriving the Adjustments

For variance and standard deviation, $v=\tau^2$. Since $dv/d\tau=2\tau$, a prior specified as a density $p_v(v)$ induces this density on the standard-deviation scale:

p_\tau(\tau)=p_v(\tau^2)\left|\frac{d(\tau^2)}{d\tau}\right| =2\tau\,p_v(\tau^2),\qquad \tau>0.

For the log standard deviation, $\tau=e^\eta$ and $v=e^{2\eta}$. Therefore, the relevant derivatives are $d\tau/d\eta=e^\eta$ and $dv/d\eta=2e^{2\eta}$. A prior given on either of those original scales transforms as follows:

p_\eta(\eta)=p_\tau(e^\eta)e^\eta =p_v(e^{2\eta})\,2e^{2\eta},\qquad -\infty<\eta<\infty.

For precision, $\lambda=1/v$, so $dv/d\lambda=-1/\lambda^2$. Equivalently, $d\lambda/dv=-1/v^2$. Thus the density transformations are

p_v(v)=p_\lambda(1/v)\frac{1}{v^2}, \qquad p_\lambda(\lambda)=p_v(1/\lambda)\frac{1}{\lambda^2}.

The negative derivative in the first relationship reflects that precision decreases as variance increases. A density uses the absolute derivative, so the Jacobian factor is positive.

Justification: For a short interval of values of $\phi$, the corresponding interval on the $\theta$ scale has width approximately $\left|d\theta/d\phi\right|d\phi$. Its probability is therefore approximately $p_\theta(\theta)\left|d\theta/d\phi\right|d\phi$. Equating this with $p_\phi(\phi)d\phi$ yields the change-of-variables formula. More exactly, the same result follows by transforming the probability integral over an interval.

The adjustment applies to the prior density, and therefore to the posterior density when the posterior is expressed on the new scale. If $L(y\mid\tau)$ is the likelihood for observed data $y$, then the posterior kernel on the standard-deviation scale is $L(y\mid\tau)p_\tau(\tau)$. On the log-standard-deviation scale, the same likelihood is evaluated at $\tau=e^\eta$, and the transformed kernel is $L(y\mid e^\eta)p_\tau(e^\eta)e^\eta$. The likelihood is not itself multiplied by a Jacobian; the Jacobian enters because the density is now with respect to $d\eta$, not $d\tau$.

Worked Transformations

Worked Example: A Prior Given on Log Standard Deviation

Suppose a hypothetical model assigns $\eta=\log(\tau)\sim N(-0.2,0.5^2)$. Find the induced density for $\tau$ at $\tau=1.5$, and compare the two prior densities at corresponding parameter values.

First, the corresponding log standard deviation is $\eta=\log(1.5)\approx0.4055$. The normal density at this value is

p_\eta(0.4055) =\frac{1}{0.5\sqrt{2\pi}} \exp\left[-\frac12\left(\frac{0.4055-(-0.2)}{0.5}\right)^2\right] \approx 0.3833.

Because $d\eta/d\tau=1/\tau$, the density on the $\tau$ scale is the density on the $\eta$ scale divided by $\tau$:

p_\tau(1.5)=\frac{p_\eta(\log(1.5))}{1.5} \approx\frac{0.3833}{1.5} \approx 0.2555.

For illustration, suppose the likelihood at $\tau=1.5$ is $L(y\mid 1.5)=0.600$. The unnormalized posterior density at this point is approximately $0.600(0.2555)=0.1533$ on the $\tau$ scale. On the $\eta$ scale, at $\eta=\log(1.5)$, it is $0.600(0.3833)\approx0.2300$. These pointwise density values differ because they are densities per unit of different coordinates. Their difference is exactly the Jacobian factor; probabilities assigned to corresponding intervals remain the same.

Worked Example: Transforming a Variance-Scale Prior

Suppose, hypothetically, that a variance $v$ has an exponential prior with rate $2$, so $p_v(v)=2e^{-2v}$ for $v>0$. Find the induced standard-deviation density and evaluate both densities at the corresponding values $v=0.25$ and $\tau=0.5$.

Since $v=\tau^2$, the Jacobian is $dv/d\tau=2\tau$. Thus

p_\tau(\tau)=2e^{-2\tau^2}(2\tau)=4\tau e^{-2\tau^2},\qquad \tau>0.

At $\tau=0.5$, the corresponding variance is $v=0.5^2=0.25$. On the variance scale,

p_v(0.25)=2e^{-2(0.25)}=2e^{-0.5}\approx1.2131.

The Jacobian at $\tau=0.5$ is $2(0.5)=1$, so $p_\tau(0.5)=1.2131(1)\approx1.2131$. Direct substitution gives the same result: $4(0.5)e^{-2(0.5)^2}=2e^{-0.5}\approx1.2131$. The numerical equality at this particular point is due to the Jacobian being one there; it is not generally true that densities on the two scales have equal numerical values.

Worked Example: Transforming a Precision Prior to Variance

Suppose a hypothetical precision $\lambda$ has a uniform prior on $(0,4)$, with density $p_\lambda(\lambda)=1/4$ within that interval and zero elsewhere. Find the induced variance density and evaluate it at $v=0.5$.

Because $v=1/\lambda$, the corresponding precision is $\lambda=1/v$. The support $0<\lambda<4$ becomes $v>1/4$: the upper precision bound translates into a lower variance bound, while positive precision can approach zero. The density transformation is

p_v(v)=p_\lambda(1/v)\left|\frac{d(1/v)}{dv}\right| =\frac14\frac{1}{v^2},\qquad v>\frac14.

At $v=0.5$, the corresponding precision is $\lambda=2$, which lies in $(0,4)$. Therefore,

p_v(0.5)=\frac14\frac{1}{(0.5)^2} =\frac14(4)=1.

As a normalization check, $\int_{1/4}^{\infty}(1/4)v^{-2}\,dv=1$. The transformed density is not uniform on the variance scale, even though the original precision density is uniform.

Several Variance Components

Suppose a model has $K$ positive variance components $v_1,\ldots,v_K$, with corresponding standard deviations $\tau_k=\sqrt{v_k}$. For the joint transformation, the Jacobian matrix is diagonal, with diagonal entries $dv_k/d\tau_k=2\tau_k$. Its determinant is the product of those entries. If the joint variance density is $p_{\boldsymbol v}$, then

p_{\boldsymbol\tau}(\boldsymbol\tau) =p_{\boldsymbol v}(\tau_1^2,\ldots,\tau_K^2) \prod_{k=1}^{K}2\tau_k.

This product rule does not require independent priors. It transforms a joint density whether or not its components are dependent. Independence, if assumed, is a separate property of the prior.

For instance, if $v_1$ and $v_2$ are independent exponential variables with rates $1$ and $0.5$, respectively, then $p_{v_1,v_2}(v_1,v_2)=e^{-v_1}(0.5)e^{-0.5v_2}$. At $\tau_1=0.5,\tau_2=2$, the variances are $0.25$ and $4$, and the joint Jacobian is $(2(0.5))(2(2))=4$. The transformed joint density is

p_{\tau_1,\tau_2}(0.5,2) =\left[e^{-0.25}(0.5)e^{-2}\right](4) \approx 0.2108.

The product of the two transformed marginal densities gives the same result: $[e^{-0.25}(1)] [0.5e^{-2}(4)]\approx(0.7788)(0.2707)=0.2108$.

Choosing and Checking a Parameterization

The parameter used to describe a prior should be distinguished from the parameter used to compute or report a model. A prior may be specified for a scientifically interpretable standard deviation, while software uses $\eta=\log(\tau)$ internally to keep the scale positive. If a density is evaluated as a density with respect to $d\eta$, the factor $d\tau/d\eta=\tau$ is required when the intended prior was specified with respect to $d\tau$. Omitting it changes the prior distribution.

A useful check is to verify that the transformed density integrates to one over its transformed support. Another is to transform probability statements: for example, a probability over $\tau\in[a,b]$, where $0<a<b$, must equal the probability over $\eta\in[\log(a),\log(b)]$. For multiple components, check that the joint Jacobian includes every transformed coordinate. In computation, also check whether the software expects the target density with respect to the unconstrained coordinate or automatically applies a transformation adjustment; silently applying an adjustment twice is as wrong as omitting it.

Direct specification on the variance or standard-deviation scale can be convenient when that is the scale on which prior information is expressed. Log-standard-deviation coordinates are useful for unconstrained computation, while precision may be natural in some model formulations. These are alternative parameterizations, not interchangeable prior specifications. The choice of scale alone does not guarantee good computation or a scientifically appropriate prior.

The change-of-variables formula assumes a continuous density and a differentiable one-to-one transformation on the support being considered. A point mass at zero variance is not transformed into an ordinary continuous density by multiplying by a Jacobian; its probability mass must be represented separately. Likewise, a transformation that is not one-to-one requires summing contributions from all preimages. A Jacobian corrects coordinates, but does not by itself resolve boundary behavior, prior propriety, or whether the prior reflects plausible heterogeneity.

Common Mistake

Common mistake: “The prior is $p_\tau(\tau)$, and I compute with $\eta=\log(\tau)$, so I can evaluate $p_\tau(e^\eta)$ as the density on the $\eta$ scale.” This omits the Jacobian and therefore changes the distribution. The density with respect to $d\eta$ is $p_\tau(e^\eta)e^\eta$. Check the integration measure and support whenever a parameter is transformed.

Check Your Understanding

Try each problem, then check the concise solutions below.

If $p_v(v)$ is a density for $v>0$, what is the induced density for $\tau=\sqrt v$?
A density $p_\tau(\tau)$ is given for $\tau>0$. Write the corresponding density for $\eta=\log\tau$.
If $\lambda=1/v$ and $p_\lambda(\lambda)=1/3$ on $0<\lambda<3$, what is the support and density of $v$?
For two variance components, what is the Jacobian determinant for transforming $(v_1,v_2)$ to $(\tau_1,\tau_2)$?
Why is a point mass at $v=0$ not handled by the continuous density transformation formula?

Solutions. (1) $p_\tau(\tau)=2\tau p_v(\tau^2)$, for $\tau>0$. (2) $p_\eta(\eta)=p_\tau(e^\eta)e^\eta$, for real $\eta$. (3) The support $\lambda\in(0,3)$ maps to $v>1/3$, and $p_v(v)=(1/3)v^{-2}$ there. (4) The determinant is $(2\tau_1)(2\tau_2)=4\tau_1\tau_2$. (5) A point mass is discrete probability, not a continuous density with respect to interval length; its probability must be carried as a separate mass.