Normalization Under Reparameterization

Why Reparameterization Requires Care

A parameter can be expressed in more than one coordinate system. For example, a control response probability $p$ lies between 0 and 1, while its logit $\eta=\log\{p/(1-p)\}$ ranges over the real line. The statistical model has not changed, but a probability density is a density with respect to a particular coordinate. Its numerical values therefore change when the coordinate changes.

This distinction matters for a normalized power prior. The prior density must acquire a Jacobian under reparameterization, while the integral that normalizes it is unchanged if the prior and likelihood are both expressed consistently. Confusing these two facts can produce a density that does not integrate to one, or a different prior distribution than intended.

Definition: Let $\theta$ be a parameter with baseline prior density $\pi_0(\theta)$, and let $L_0(\theta)$ be the likelihood from historical data $D_0$, viewed as a function of $\theta$. For a fixed discount $a\in[0,1]$, define the powered likelihood to be $L_0(\theta)^a$ when $a>0$, and define it to be identically 1 when $a=0$, including at parameter values where $L_0(\theta)=0$. If $$ C(a)=\int \pi_0(\theta)L_0(\theta)^a\,d\theta $$ is finite and strictly positive, the normalized power-prior density is $$ \pi_a(\theta)=\frac{\pi_0(\theta)L_0(\theta)^a}{C(a)}. $$

The convention at $a=0$ avoids the undefined expression $0^0$. If the baseline prior is proper, meaning $\int\pi_0(\theta)\,d\theta=1$, this convention gives $C(0)=1$ and $\pi_a(\theta)=\pi_0(\theta)$ at $a=0$. For $a>0$, the definition requires a finite, positive integral; not every baseline prior and likelihood combination guarantees one. The likelihood may be represented up to a factor that does not depend on $\theta$: such a factor changes $C(a)$, but cancels between numerator and denominator of $\pi_a$.

The Change-of-Variables Result

Suppose $\phi=g(\theta)$ is a one-to-one differentiable transformation with differentiable inverse $\theta=h(\phi)$. In one dimension, assume the derivative of the inverse is nonzero in the region of interest. The change-of-variables formula says that a density transforms by multiplying by the absolute derivative of the inverse. In several dimensions, the corresponding factor is the absolute determinant of the inverse Jacobian matrix.

Theorem: Under the stated transformation, the normalized power-prior density with respect to $d\phi$ is $$ \pi_a^\phi(\phi)= \frac{\pi_0^\theta(h(\phi))L_0^\theta(h(\phi))^a}{C(a)} \left|\frac{d h(\phi)}{d\phi}\right|, $$ where the powered likelihood at $a=0$ uses the identically-one convention. The normalizing integral in the $\phi$ coordinate is the same number: $$ C_\phi(a)=\int \pi_0^\phi(\phi)L_0^\phi(\phi)^a\,d\phi=C_\theta(a). $$

Here $\pi_0^\phi(\phi)=\pi_0^\theta(h(\phi))|dh/d\phi|$, and $L_0^\phi(\phi)=L_0^\theta(h(\phi))$. A likelihood is a function of the parameter for the observed data; re-expressing the parameter does not add a Jacobian to the likelihood. The Jacobian belongs to the density with respect to the parameter coordinate.

Why the Normalizing Integral Is Invariant

Substitute the transformed baseline density and likelihood into the integral defining the normalizer. For $a>0$, the likelihood term is simply composed with the inverse map, so

\begin{aligned} C_\phi(a) &=\int \pi_0^\theta(h(\phi)) \left|\frac{dh(\phi)}{d\phi}\right| L_0^\theta(h(\phi))^a\,d\phi\\ &=\int \pi_0^\theta(\theta)L_0^\theta(\theta)^a\,d\theta\\ &=C_\theta(a). \end{aligned}

The second line is precisely the change-of-variables formula with $\theta=h(\phi)$. At $a=0$, the powered likelihood is identically 1 in either coordinate, so the same argument reduces to invariance of the baseline-prior integral. Thus the normalizer is a scalar integral over the parameter space, not a density value that must itself acquire a Jacobian. The density changes; its integral remains the same.

Worked Example: A Beta Power Prior on the Logit Scale

Suppose a historical control group has $x=7$ responses among $n=10$ patients. Let the response probability be $p$, take a uniform baseline prior $\operatorname{Beta}(1,1)$, and use the likelihood kernel $L_0(p)=p^7(1-p)^3$. Set $a=0.5$. The kernel omits the binomial coefficient, which is constant in $p$ and cancels from the normalized density.

The unnormalized power prior is $p^{3.5}(1-p)^{1.5}$. Its normalizer is

C(0.5)=\int_0^1p^{3.5}(1-p)^{1.5}\,dp =B(4.5,2.5)\approx0.02148.

Consequently $p$ has a $\operatorname{Beta}(4.5,2.5)$ normalized power-prior distribution. Now set $\eta=\log\{p/(1-p)\}$, so $p=e^\eta/(1+e^\eta)$. Since $dp/d\eta=p(1-p)$, the density with respect to $d\eta$ is

\pi_{0.5}^{\eta}(\eta) =\frac{p^{3.5}(1-p)^{1.5}}{B(4.5,2.5)}p(1-p) =\frac{p^{4.5}(1-p)^{2.5}}{B(4.5,2.5)}.

For a numerical check at $\eta=0$, $p=0.5$. The density on the $p$ scale is $0.5^{3.5+1.5}/0.02148\approx1.455$. Multiplying by $dp/d\eta=0.25$ gives $0.364$, which agrees with $0.5^{4.5+2.5}/0.02148\approx0.364$. These are densities with respect to different units, so their values need not match.

Worked Example: A Nonlinear Transformation With a Simple Normalizer

Let $\theta\in(0,1)$ have a uniform baseline prior, and suppose the historical likelihood kernel is $L_0(\theta)=\theta^2$, as for two successes in two Bernoulli observations. Choose $a=0.5$. Then $L_0(\theta)^a=\theta$, and

C(0.5)=\int_0^1\theta\,d\theta=\frac12, \qquad \pi_{0.5}^{\theta}(\theta)=2\theta.

Transform to $\phi=\theta^2$, a one-to-one map from $(0,1)$ to $(0,1)$. Its inverse is $\theta=\sqrt{\phi}$, with $d\theta/d\phi=1/(2\sqrt{\phi})$. The transformed prior density is therefore

\pi_{0.5}^{\phi}(\phi) =2\sqrt{\phi}\frac{1}{2\sqrt{\phi}}=1,\qquad 0<\phi<1.

It integrates to one. The normalizer calculated directly in the new coordinate also agrees: the transformed baseline density is $1/(2\sqrt{\phi})$, and the likelihood kernel is $\phi$, so $\int_0^1\{1/(2\sqrt{\phi})\}\phi^{0.5}\,d\phi=1/2$. This example shows why a nonconstant Jacobian can substantially alter the displayed density even though it leaves the normalizer unchanged.

Affine Transformation of a Continuous Parameter

Worked Example: A Discounted Normal Likelihood After Rescaling

Let $\theta$ have baseline prior $N(0,1)$, and let the historical likelihood kernel be $L_0(\theta)=\exp\{-(1-\theta)^2/2\}$, corresponding to a normal observation with value 1 and known variance 1, with parameter-independent factors omitted. Set $a=0.5$. The product of the baseline density and powered likelihood is proportional to $\exp\{-\theta^2/2-(0.5)(1-\theta)^2/2\}$. Completing the square gives a normalized power prior $N(1/3,2/3)$: its precision is $1+0.5=1.5$, and its mean is $0.5/1.5=1/3$.

For this likelihood kernel, direct integration gives

C(0.5)=\frac{1}{\sqrt{1+0.5}} \exp\left\{-\frac{0.5(1)^2}{2(1+0.5)}\right\} \approx0.6911.

Now define $\phi=2\theta+3$, so $\theta=(\phi-3)/2$ and $|d\theta/d\phi|=1/2$. The transformed normalized power prior is normal with mean $2(1/3)+3=3.667$ and variance $2^2(2/3)=2.667$. Its density is the density of $N(1/3,2/3)$ evaluated at $(\phi-3)/2$, multiplied by $1/2$. The normalizer remains approximately $0.6911$, since the same change of variables converts its integral back to the $\theta$ integral.

Assumptions, Checks, and Common Errors

The transformation result assumes a valid one-to-one change of coordinates over the parameter space, a baseline prior density with respect to the stated measure, and a normalizer that is finite and positive. If the parameter is vector-valued, replace the absolute derivative by the absolute determinant $|\det J_h(\phi)|$ of the inverse Jacobian. If a transformation is many-to-one, one must account for all inverse branches; the single-branch formula above does not apply as written.

A practical diagnostic is to integrate the transformed density over its transformed support. It must equal one. Also check that the Jacobian uses the inverse map when the original density is written as a function of the new coordinate. In the logit example, $dp/d\eta=p(1-p)$; using $d\eta/dp$ instead would multiply by the reciprocal and generally fail the integration check.

Omitting the Jacobian. The reasoning “the parameter is the same, so its density is unchanged” is wrong: a density is defined relative to a coordinate measure. Multiply by the absolute inverse derivative or determinant.
Changing only part of the construction. Transform the baseline prior consistently and evaluate the same likelihood at the inverse-mapped parameter. Mixing a prior density from one coordinate with a likelihood or integral from another can yield an incorrect normalizer.
Treating the normalizer as a density. It is an integral and is invariant under a valid change of variables. It does not get multiplied by the Jacobian as a standalone scalar.
Using $0^0=1$ without a definition. If the likelihood vanishes, ordinary exponentiation does not define its zero-th power there. Define the powered likelihood at $a=0$ to be identically 1; for a proper baseline prior this gives $C(0)=1$.

A common alternative is to work entirely in the original coordinate, normalize there, and transform the resulting density afterward. This is often the simplest computational check. Directly constructing the power prior in the new coordinate is equally valid when the transformed baseline density and likelihood are used consistently. Neither approach makes the numerical density invariant. Both produce the same probability distribution under the bijection. Reparameterization also does not repair an improper or incompatible construction: the required finiteness and positivity of $C(a)$ must still be checked.

Key takeaway: For a fixed discount, the normalized power-prior density transforms like any other density: evaluate it at the inverse-mapped parameter and multiply by the absolute inverse Jacobian. The normalizing integral is unchanged when the entire construction is transformed consistently.

Check Your Understanding

Answer each question, then compare with the solutions below.

For a one-to-one scalar map $\phi=g(\theta)$, which derivative appears in the transformed density, and why?
A proper baseline prior is used with $a=0$. What is $C(0)$, including at parameter values where the historical likelihood is zero?
A normalized density on $\theta$ is transformed to $\phi=3\theta$, with $\theta\in(0,1)$. What is the absolute Jacobian factor in the $\phi$-density?
Does the normalizing integral change under a one-to-one reparameterization? State the condition needed for the equality.

Solutions. (1) Use $|d\theta/d\phi|$, the derivative of the inverse map, because the density is being expressed per unit of $\phi$. (2) Define $L_0^0$ to be identically 1; then $C(0)=\int\pi_0(\theta)d\theta=1$, including wherever $L_0=0$. (3) The inverse is $\theta=\phi/3$, so the factor is $|d\theta/d\phi|=1/3$. (4) No: the value is invariant when the baseline prior and likelihood are both expressed in the new coordinate and the change-of-variables formula is applied. The integral must exist as a finite, positive normalizer for the normalized power prior to be defined.