Transformations For Constrained Posterior Parameters

Why Change the Sampling Scale?

Posterior computation often involves parameters whose allowed values do not fill the real line. A rate must be positive, and a probability must lie between zero and one. A sampling method designed to propose real-valued updates can therefore propose impossible values. One solution is to transform the constrained parameter to a real-valued parameter, evaluate the posterior density on that new scale, and sample there.

The transformation changes more than the name of the parameter. A probability density is defined relative to a scale: the density of $\theta$ with respect to $d\theta$ is not generally the density of a transformed variable $u$ with respect to $du$. The change-of-variables formula supplies the required Jacobian adjustment. Omitting it changes the target distribution.

Definition: Suppose a parameter $\theta$ has posterior density $\pi(\theta\mid y)$ with respect to $d\theta$, and $\theta=g(u)$ is a differentiable, one-to-one transformation. The posterior density on the $u$ scale is $$ \pi_u(u\mid y)=\pi(g(u)\mid y)\left|\frac{dg(u)}{du}\right|. $$ Thus its log density is $\log \pi(g(u)\mid y)+\log\left|dg(u)/du\right|$. For a vector transformation, the absolute derivative is replaced by the absolute determinant of the Jacobian matrix.

This formula applies to a density, not merely to a likelihood. The posterior density includes both likelihood and prior, so the Jacobian multiplies their product. Additive constants that do not depend on the sampling parameter may be omitted when evaluating a Metropolis acceptance ratio, but the parameter-dependent Jacobian term may not be omitted.

Two Useful Transformations

For a positive parameter $\theta$, choose a positive reference value $\theta_0$ in the same units and define $u=\log(\theta/\theta_0)$. Then $\theta=\theta_0\exp(u)$, which maps every real $u$ to a positive $\theta$. The derivative is $d\theta/du=\theta_0\exp(u)=\theta$, so the log-Jacobian is $\log\theta$.

For a probability $p$, define the logit $\eta=\log\{p/(1-p)\}$. Its inverse is $p=1/(1+\exp(-\eta))$. Since $dp/d\eta=p(1-p)$, the log-Jacobian is $\log p+\log(1-p)$. Both transformations are one-to-one between their stated open intervals and the real line.

\begin{aligned} \theta=\theta_0\exp(u) &: \quad \log\pi_u(u\mid y)=\log\pi(\theta_0\exp(u)\mid y)+\log\theta,\\ p=\frac{1}{1+\exp(-\eta)} &: \quad \log\pi_\eta(\eta\mid y)=\log\pi(p\mid y)+\log p+\log(1-p). \end{aligned}

The reference value $\theta_0$ can make $u$ more numerically convenient; it does not change the underlying posterior for $\theta$. In the probability transformation, a finite $\eta$ gives a probability strictly between zero and one. Exact boundary values $p=0$ or $p=1$ are not represented by finite $\eta$. These transforms are appropriate for continuous parameters with support in the interior, not for models assigning point probability to a boundary.

Deriving the Adjustment

The Jacobian follows by requiring the probability assigned to corresponding small intervals to agree. For a one-dimensional monotone transformation, an interval of width $du$ maps approximately to an interval of width $|d\theta/du|du$. Therefore

\pi_u(u\mid y)\,du =\pi(\theta\mid y)\,|d\theta| =\pi(g(u)\mid y)\left|\frac{dg(u)}{du}\right|du.

Dividing by $du$ gives the transformed density. This argument also explains a subtle point: the mode of a density can depend on the scale on which the density is written. A mode on the $u$ scale need not map to the mode on the $\theta$ scale. The posterior distribution itself is unchanged when transformed correctly; its density values and mode locations are scale-dependent.

Worked Example: A Positive Event Rate

Suppose a hypothetical trial observes $y=6$ events over $E=11$ patient-years, with an exponential event-time model whose event rate is $\lambda>0$. The likelihood kernel is $\lambda^y\exp(-E\lambda)$. Take a Gamma prior with shape $a=2$ and rate $b=3$, whose density kernel is $\lambda^{a-1}\exp(-b\lambda)$. The likelihood and prior are proper, and the resulting posterior is proper.

Multiplying the kernels gives a posterior density proportional to $\lambda^{y+a-1}\exp\{-(E+b)\lambda\}=\lambda^7\exp(-14\lambda)$. This is a Gamma posterior with shape $8$ and rate $14$. Set $\lambda=\lambda_0\exp(u)$, using $\lambda_0=1$ event per patient-year. The Jacobian is $\lambda$, so the transformed kernel is proportional to $\lambda^8\exp(-14\lambda)$, not $\lambda^7\exp(-14\lambda)$.

At the hypothetical value $\lambda=0.5$ events per patient-year, $u=\log(0.5)$. Omitting terms constant in $u$, the transformed log posterior is

8u-14\exp(u) =8\log(0.5)-14(0.5) \approx -5.5452-7.0000 =-12.5452.

The coefficient $8$, rather than $7$, records the added log-Jacobian. A sampler on the $u$ scale must use this transformed target. After sampling, return each draw to event rates per patient-year by calculating $\lambda=\exp(u)$ for this chosen reference value.

Probability Parameters and Trial Outcomes

For a binomial endpoint, suppose $Y\mid p\sim\operatorname{Binomial}(n,p)$, and $p\sim\operatorname{Beta}(a,b)$, with $a,b>0$. The likelihood is proportional to $p^y(1-p)^{n-y}$; the prior density is proportional to $p^{a-1}(1-p)^{b-1}$. Consequently the posterior density on the $p$ scale is proportional to $p^{a+y-1}(1-p)^{b+n-y-1}$. Applying the logit transformation adds one power of $p(1-p)$, through its Jacobian.

Worked Example: A Single Trial-Arm Response Probability

In a hypothetical arm, 7 of 10 participants have a binary response. Let the prior be $\operatorname{Beta}(2,2)$. The posterior density for the response probability $p$ is proportional to $p^8(1-p)^4$, or equivalently it is a $\operatorname{Beta}(9,5)$ density. The continuous beta prior and binomial likelihood satisfy the support and model assumptions for this update.

Write $p=1/(1+\exp(-\eta))$. The transformed log posterior, up to a constant, is

8\log p+4\log(1-p)+\log p+\log(1-p) =9\log p+5\log(1-p).

At the hypothetical value $p=0.6$, $\eta=\log(0.6/0.4)\approx0.4055$. Substitution gives

9\log(0.6)+5\log(0.4) \approx 9(-0.510826)+5(-0.916291) \approx -4.59743-4.58145 =-9.17888.

This is the transformed log density up to a constant. Without the Jacobian, the coefficient pair would incorrectly remain $8$ and $4$, targeting a different density for $\eta$. If draws are made on the logit scale, report probabilities by applying the inverse-logit to each draw.

A useful check is to compare the modes. The density on the $p$ scale is proportional to $p^8(1-p)^4$, whose interior mode is $8/(8+4)=2/3$. The transformed density on the $\eta$ scale is proportional to $p^9(1-p)^5$, whose mode corresponds to $p=9/(9+5)=9/14$. These differ because a density is relative to its scale. They do not indicate two different posterior distributions.

Worked Example: Two Arm Probabilities Together

Consider hypothetical independent response data: 7 responses among 20 participants in the treatment arm and 3 among 20 in the control arm. Assign each arm an independent $\operatorname{Beta}(1,1)$ prior. Conditional on the two probabilities, take the arm outcomes to be independent binomial observations. The posterior kernels are $p_T^7(1-p_T)^{13}$ and $p_C^3(1-p_C)^{17}$.

Transform both probabilities with logits $\eta_T,\eta_C$. The Jacobian matrix is diagonal, so its determinant is $p_T(1-p_T)p_C(1-p_C)$. The transformed log posterior, omitting constants, is

8\log p_T+14\log(1-p_T)+4\log p_C+18\log(1-p_C).

At hypothetical values $p_T=0.4$ and $p_C=0.2$, the two logits are $\log(0.4/0.6)\approx-0.4055$ and $\log(0.2/0.8)\approx-1.3863$. The transformed log kernel is

\begin{aligned} &8\log(0.4)+14\log(0.6)+4\log(0.2)+18\log(0.8)\\ &\approx -7.330326-7.151558-6.437752-4.016584\\ &\approx -24.9362. \end{aligned}

For each draw, transform both probabilities back before computing a risk difference $p_T-p_C$, risk ratio, or other clinical contrast. The transformation makes the sampling coordinates unconstrained; it does not itself change the clinical estimand.

Using Transformed Draws and Checking the Target

Posterior summaries should be calculated on the scale of the quantity being reported. For $\theta=\theta_0\exp(u)$, $\theta_0$ times the mean of $\exp(u)$ across draws estimates the posterior mean of $\theta$. In general, exponentiating the mean of $u$ does not give that posterior mean. For probabilities, apply the inverse-logit to each draw before averaging to estimate the posterior mean probability. Since both inverse maps are monotone, quantiles transform directly: transform the lower and upper quantiles of $u$ or $\eta$ to obtain the corresponding quantiles on the original scale.

For Metropolis-Hastings updates on a transformed scale, evaluate the transformed log posterior at the proposed and current values and use their difference in the log acceptance ratio, along with the proposal-density terms required by the proposal. A symmetric proposal on the transformed scale has cancelling proposal terms, but the Jacobian remains part of the target. The parameterization alone does not guarantee good exploration or convergence; chains still require appropriate initialization and diagnostic checks.

Approach	Strength	Limitation
Transform and sample on the real line	Every proposed value maps to a valid interior parameter value.	Requires the correct Jacobian and back-transformation; geometry can still be difficult.
Propose directly on the constrained scale	Target can be expressed in the parameter's original units.	Proposals may leave the support and need boundary-aware construction or rejection.
Use a model-specific conditional distribution when available	Can avoid a Metropolis update for that parameter.	Requires a tractable conditional distribution; not every model provides one.

Common warning signs include a transformed log density that does not include the derivative term, numerical overflow from directly evaluating $\exp(u)$ at extreme values, and reported summaries that remain on the transformed scale. Work with stable log-density expressions where possible. Also verify the model support: if a parameter has a point mass at zero as well as a continuous positive component, a log transformation alone does not represent the full posterior.

Common mistake: “The original posterior kernel is $p^8(1-p)^4$, so I can substitute the inverse-logit and use that as the density for $\eta$.” The correction is to multiply by $dp/d\eta=p(1-p)$. The resulting density is proportional to $p^9(1-p)^5$; omitting the Jacobian changes the distribution being sampled.

Check Your Understanding

Answer each question, then compare with the solutions below.

For $\theta=\theta_0\exp(u)$, what is the Jacobian factor, and what is its log?
A posterior density for $p$ is proportional to $p^2(1-p)^6$. What is its logit-scale density up to proportionality?
Why is the transformed-density mode not necessarily the transform of the original-density mode?
If draws $u$ correspond to $\theta=\theta_0\exp(u)$, how should the posterior mean of $\theta$ be estimated from those draws?

Solutions. (1) The derivative is $d\theta/du=\theta_0\exp(u)=\theta$, so the factor is $\theta$ and the log-Jacobian is $\log\theta$. (2) The logit Jacobian contributes $p(1-p)$, giving a transformed density proportional to $p^3(1-p)^7$, where $p=1/(1+\exp(-\eta))$. (3) Density values are measured relative to the scale element, and the Jacobian changes those values; therefore their maximizers need not correspond under transformation. (4) Calculate $\theta_0\exp(u)$ for every draw and average those values, equivalently multiplying $\theta_0$ by the sample mean of $\exp(u)$.