Invariance of Profile Likelihood Under Reparameterization

Why Reparameterization Should Not Change the Inference

A scalar parameter may have more than one useful interpretation. In logistic regression, a coefficient $\beta$ is a change in log odds, while $\exp(\beta)$ is the corresponding odds ratio. A profile likelihood interval for $\beta$ should therefore give an interval for the odds ratio by transforming its values. The key fact is stronger than an approximation: for a one-to-one transformation, profiling and transforming the target give exactly the same likelihood-ratio inference.

This result concerns a change in the parameter used to describe the same model. It is not a change to the probability model for the data. In particular, a Jacobian factor used when transforming a random variable’s density does not enter the likelihood merely because a parameter has been re-expressed.

Definition and Invariance Result

Definition: Let $\theta\in\Theta$ be a model parameter, let $L(\theta)$ be its likelihood, and let $\tau:\Theta\to\Psi$ be a scalar target. For $\psi\in\Psi$, the profile likelihood for $\psi$ is $$ L_p(\psi)=\sup\{L(\theta):\theta\in\Theta,\ \tau(\theta)=\psi\}. $$ The supremum over an empty constraint set is defined to be zero. Write $L_{\max}=\sup_{\theta\in\Theta}L(\theta)$.

Now suppose $h:\Psi\to\Phi$ is a bijection, and use $\phi=h(\psi)$ as the target instead. The transformed target is $\tau_\phi(\theta)=h(\tau(\theta))$. A profile likelihood is defined by maximizing over parameters that give the specified target value; it does not integrate over nuisance parameters.

Theorem: For every $\phi\in\Phi$, $$ L_{p,\phi}(\phi)=L_p\bigl(h^{-1}(\phi)\bigr). $$ Consequently, if $0<L_{\max}<\infty$ and $0<c<\infty$, the likelihood-ratio confidence sets $$ C_\psi=\left\{\psi\in\Psi:2\log\frac{L_{\max}}{L_p(\psi)}\leq c\right\},\qquad C_\phi=\left\{\phi\in\Phi:2\log\frac{L_{\max}}{L_{p,\phi}(\phi)}\leq c\right\} $$ satisfy $C_\phi=h(C_\psi)=\{h(\psi):\psi\in C_\psi\}$. If a profile likelihood is zero, its likelihood-ratio statistic is taken to be $+\infty$.

Proof. Fix $\phi\in\Phi$, and set $\psi=h^{-1}(\phi)$, which exists and is unique because $h$ is a bijection. For any $\theta\in\Theta$, $$ \tau_\phi(\theta)=\phi \quad\Longleftrightarrow\quad h(\tau(\theta))=\phi \quad\Longleftrightarrow\quad \tau(\theta)=h^{-1}(\phi)=\psi. $$ Thus the two constrained parameter sets are identical. Taking the supremum of the same likelihood over the same set gives $L_{p,\phi}(\phi)=L_p(\psi)$. The global supremum $L_{\max}$ is unchanged because the underlying parameter space and likelihood have not changed. Substitution into the two likelihood-ratio statistics shows that the statistic at $\phi$ equals the statistic at $h^{-1}(\phi)$. Therefore $\phi\in C_\phi$ exactly when $h^{-1}(\phi)\in C_\psi$, which is equivalent to $\phi\in h(C_\psi)$. $\square$

The result requires neither a differentiable transformation nor an attained constrained maximum: it uses only equality of the constraint sets and their suprema. The cutoff $c$ can be a usual asymptotic cutoff or another chosen calibration. Whatever cutoff is used, the two sets correspond exactly when the same cutoff is used in both.

Worked Examples

Worked Example: Cubing a Gaussian Mean

Let $Y\sim N(\psi,0.25)$, with observed value $y=2$, and take the target to be $\psi$. This is a Gaussian model with known variance. Its log likelihood, up to a constant independent of $\psi$, is $\ell(\psi)=-(2-\psi)^2/(2(0.25))$. The maximum occurs at $\hat\psi=2$, so the likelihood-ratio statistic is $$ 2\{\ell(2)-\ell(\psi)\}=\frac{(2-\psi)^2}{0.25}=4(2-\psi)^2. $$

Using the cutoff $c=3.841$, the profile confidence set is obtained by solving $4(2-\psi)^2\leq3.841$. Hence $$ 2-\sqrt{3.841/4}\leq\psi\leq2+\sqrt{3.841/4}. $$ Set $d=\sqrt{3.841/4}=\sqrt{0.96025}\approx0.979923$. The unrounded endpoints are $2-d$ and $2+d$, approximately $1.020077$ and $2.979923$.

The map $h(\psi)=\psi^3$ is a bijection on the real line and is increasing. Its transformed confidence set is therefore $$ [(2-d)^3,(2+d)^3]\approx[1.061,26.462]. $$ For a calculation check, the endpoints can be evaluated as $8-12d+6d^2-d^3$ and $8+12d+6d^2+d^3$, respectively; using $d^2=0.96025$ gives approximately $1.061447$ and $26.461553$. Rounding the original endpoints first to $1.020$ and $2.980$ and then cubing gives a slightly different approximation: $2.980^3=26.463592$, which rounds to $26.464$. Endpoint calculations should use unrounded values when reporting the transformed set.

Worked Example: Exponentiating a Logistic Coefficient

Suppose two independent Bernoulli observations share a logistic intercept $\beta$, with one success and one failure. Write $p=\exp(\beta)/(1+\exp(\beta))$. Ignoring the constant combinatorial factor, the likelihood is $L(\beta)=p(1-p)$. It is maximized at $p=1/2$, equivalently $\hat\beta=0$, where $L_{\max}=1/4$.

Choose the likelihood-ratio cutoff $c=2\log(25/9)\approx2.0433$. The LR statistic at a value corresponding to $p$ is $$ 2\log\frac{1/4}{p(1-p)}. $$ The condition that this be at most $c$ is equivalent to $$ \frac{1/4}{p(1-p)}\leq\frac{25}{9}, \qquad p(1-p)\geq\frac{9}{100}=0.09. $$ Solving $p-p^2\geq0.09$, or $p^2-p+0.09\leq0$, gives roots $(1\pm\sqrt{1-0.36})/2=(1\pm0.8)/2$, so $p\in[0.1,0.9]$. The logit map then gives $$ \beta\in\left[\log\frac{0.1}{0.9},\log\frac{0.9}{0.1}\right]=[-\log9,\log9]. $$

Since $h(\beta)=\exp(\beta)$ is increasing and one-to-one, the corresponding set for the odds ratio is $$ \exp(C_\beta)=[\exp(-\log9),\exp(\log9)]=[1/9,9]. $$ Thus exponentiating the coefficient interval yields an odds-ratio interval without refitting or changing the likelihood calculation. The numerical cutoff here was selected to make the endpoints exact; the invariance result also holds for other cutoffs.

Worked Example: A Decreasing Reciprocal Transformation With a Nuisance Parameter

Let $Y\sim N(\psi,1)$ and $Z\sim N(\lambda,1)$ independently, with parameter space $\Theta=(0,\infty)\times\mathbb R$. Suppose the observations are $y=2$ and $z=-1$, and the target is $\psi$; $\lambda$ is a nuisance parameter. The joint log likelihood, up to a constant, is $$ \ell(\psi,\lambda)=-\frac{(2-\psi)^2}{2}-\frac{(-1-\lambda)^2}{2}. $$

At fixed $\psi$, the second term is maximized at $\hat\lambda=-1$. Thus $\ell_p(\psi)$, up to the same constant, is $-(2-\psi)^2/2$, with maximum at $\hat\psi=2$. The LR statistic is $(2-\psi)^2$. With cutoff $3.841$, the profile set is $[2-\sqrt{3.841},2+\sqrt{3.841}]$, approximately $[0.040,3.960]$, so it lies in the positive target space.

For $\phi=1/\psi$, the map is a bijection from $(0,\infty)$ to $(0,\infty)$, but it is decreasing. The transformed set therefore reverses the endpoint order: $$ C_\phi=\left[\frac{1}{2+\sqrt{3.841}},\frac{1}{2-\sqrt{3.841}}\right]\approx[0.253,24.9]. $$ The same result follows by profiling at fixed $\phi$: that constraint is $\psi=1/\phi$, with the same maximizing nuisance value $\lambda=-1$, so the profile LR statistic at $\phi$ is $(2-1/\phi)^2$. Its cutoff condition is exactly the image of the original condition. This example shows why a decreasing transformation changes endpoint order, not the underlying likelihood-ratio set.

What the Invariance Does—and Does Not—Say

The profile likelihood itself is a likelihood as a function of a target value. Under a bijective target transformation, its numerical value is carried along by composition with the inverse map: $L_{p,\phi}(\phi)=L_p(h^{-1}(\phi))$. The likelihood-ratio statistic and its confidence set follow the same correspondence. If a procedure has coverage for $\psi$, the transformed procedure has the same coverage for $\phi=h(\psi)$: the event $h(\psi)\in h(C_\psi)$ is equivalent to $\psi\in C_\psi$, because $h$ is one-to-one.

There is no Jacobian multiplying $L(\theta)$ here. A likelihood is the data density or mass function evaluated at the observed data, regarded as a function of the parameter. Rewriting that parameter as $\eta=r(\theta)$ gives $L_\eta(\eta)=L(r^{-1}(\eta))$; it does not transform the observed data or its density. A Jacobian would arise in a different operation, such as transforming a random variable’s density or forming a probability density over parameters.

One-to-one correspondence matters. If $h$ is many-to-one, the constraint $h(\tau(\theta))=\phi$ may combine several distinct values of the original target. Its profile likelihood is then the supremum across those values, not the value at a unique inverse. For example, if $h(\psi)=\psi^2$, fixing a positive $\phi$ allows both $\psi=\sqrt{\phi}$ and $\psi=-\sqrt{\phi}$, when both are in the target space. The resulting profile need not be the image of the original confidence set under the simple endpoint rule. The theorem’s bijection condition avoids this ambiguity.

Common Pitfalls

Adding a Jacobian to the likelihood. A parameter re-expression composes the likelihood with the inverse map; it does not add a density-transformation factor.
Reversing endpoints incorrectly. For a decreasing bijection, the image of $[a,b]$ is $[h(b),h(a)]$, not $[h(a),h(b)]$.
Transforming only the estimate. A transformed maximum-likelihood estimate does not by itself give a transformed profile interval. Transform the entire confidence set, or equivalently evaluate the reparameterized profile likelihood.
Assuming the result for a many-to-one map. Several original target values can satisfy the same transformed constraint, so the transformed profile may take the largest likelihood among multiple branches.
Rounding too early. Nonlinear transformations magnify endpoint rounding differences. Transform unrounded endpoints, then round the final values.

Key takeaway: If $h$ is a bijection of the scalar target space, its profile likelihood is the original profile likelihood composed with $h^{-1}$, and its likelihood-ratio confidence set is exactly $h(C_\psi)$. Use the actual image of the set, with endpoint order determined by whether $h$ is increasing or decreasing.

Check Your Understanding

Use the definition and results above to answer these questions.

In one sentence, why does the constrained parameter set stay the same when the target is changed from $\psi$ to $h(\psi)$ for a bijection $h$?
If a profile confidence interval is $[a,b]$ with $0<a<b$, what is its image under $h(\psi)=1/\psi$?
For the logistic example, why does exponentiating $[-\log9,\log9]$ give an odds-ratio interval rather than a probability interval?
Does a differentiable bijection require a Jacobian factor in the reparameterized likelihood? Explain briefly.
What changes in the profile constraint when $h(\psi)=\psi^2$ and both signs of $\psi$ are allowed?