Why Change the Parameterization?
A variance component may be reported as zero, a correlation as one, or a positive fixed effect as sitting at its lower bound. Such a result can reflect a genuine maximum of the likelihood at the edge of the allowed parameter space. It can also arise because the optimizer’s steps, stopping tolerances, or finite bounds behave poorly on the scale used for estimation. A reparameterization is a diagnostic: it changes the coordinates used to search while preserving the statistical model.
The central question is not whether an alternative parameterization makes the estimate look more interior. It is whether the same likelihood, model predictions, and boundary behavior are recovered after translating estimates back to the original parameter. If they are not, the discrepancy is evidence to investigate the optimization—not proof that the transformed estimate is correct.
For example, a variance \(v\) must satisfy \(v\geq 0\). For strictly positive values, the transformation \(\eta=\log(v)\), with inverse \(v=\exp(\eta)\), is one-to-one. But \(v=0\) has no finite value of \(\eta\): it corresponds to the limit \(\eta\to-\infty\). That difference matters. An unconstrained optimizer in \(\eta\) can represent every positive variance, but cannot attain exactly zero at a finite coordinate.
What Is Preserved, and What Is Not?
Let \(\ell(\theta)\) be the log likelihood and suppose \(\phi=g(\theta)\) is a one-to-one transformation. The likelihood in the new coordinate is \(\tilde{\ell}(\phi)=\ell(g^{-1}(\phi))\). Since this only relabels the same parameter values, the exact maximum and its likelihood value are preserved, provided the transformed parameter space is handled correctly. The estimate in the original coordinate is recovered by applying \(g^{-1}\).
This equality concerns an exact optimization over equivalent spaces. A computer performs finite steps, uses tolerances, and may impose finite bounds. Those practical details are not invariant to a change of coordinates. Consequently, different estimates from two parameterizations can reveal a numerical or optimization issue even though neither transformation changes the underlying likelihood.
For a differentiable transformation, the chain rule makes the scale dependence explicit:
A gradient that is numerically small in one coordinate need not be small in another, especially near a boundary where the derivative of the transformation or its inverse changes greatly. Thus, convergence should not be judged from a small step or gradient in a single parameterization alone. Compare objective values, transformed estimates, and the convergence diagnostics in the original scientific coordinates.
For a smooth transformation with nonzero derivative, the second derivative is \(\partial^2\tilde{\ell}/\partial\phi^2 =(\partial^2\ell/\partial\theta^2)(\partial\theta/\partial\phi)^2 +(\partial\ell/\partial\theta)(\partial^2\theta/\partial\phi^2)\). At an interior stationary point, the second term is zero. This explains why local curvature magnitudes can change with coordinates even when the optimum is unchanged. At a boundary, or away from a stationary point, the convenient interior argument does not establish equivalence of the numerical diagnostics.
Variance Components on the Log Scale
In pharmacometric models, a variance component \(v\), including an interindividual or residual variance, is often constrained to be nonnegative. A log-scale parameter \(\eta=\log(v)\) enforces positivity automatically when \(v=\exp(\eta)\). The transformation is helpful for diagnosing estimates that are small but positive; it does not make an exact zero an ordinary finite value.
A fair comparison requires using the same model, data, likelihood evaluation, and scientifically allowed parameter space. If the original fit allows \(v=0\), then the log-scale fit must be interpreted with the limiting value \(v\to0\), or with an explicitly stated small positive lower bound. Comparing a fit that permits zero with one that silently excludes a substantial interval near zero changes the problem.
Worked Example: A Small Variance and a Scale-Based Stopping Rule
Suppose a hypothetical model has a profile log likelihood for a variance \(v\), measured in squared concentration units, of \(\ell(v)=-(v-0.0004)^2/[2(0.0005)^2]\), for \(v\geq0\). Assume this profile is smooth and comes from the same fitted model throughout. The exact maximum is at \(v=0.0004\), not at zero.
At the maximum, \(\ell(0.0004)=0\). At the boundary, \(\ell(0)=-(0.0004)^2/[2(0.0005)^2]=-0.32\). Thus the boundary fit has a log likelihood 0.32 lower than the interior maximum. In this illustrative profile, the likelihood favors a small positive value, although the difference alone does not establish that the variance is precisely estimated.
Now suppose a linear-scale search uses an absolute parameter-step tolerance of 0.001 variance units and stops at \(v=0\). The reported value may be treated as converged simply because the proposed changes are smaller than the tolerance, despite the optimum lying at 0.0004. The derivative at zero is \(\ell'(0)=(0.0004)/(0.0005)^2=1600\) per variance unit, so zero is not an interior stationary point. That large derivative is a warning that a step-size stopping rule alone is inadequate.
On the log scale, the maximizing coordinate is \(\eta=\log(0.0004)\approx-7.824\). Converting back gives \(v=\exp(-7.824)\approx0.0004\). If the log-scale fit reaches this value and the linear-scale fit does not, the disagreement points to the search or stopping behavior in the linear coordinate. It does not show that the log transformation created new information; both fits must be checked against the same likelihood.
Boundary Behavior and Information in the Data
A transformation can expose the difference between a finite positive estimate and a likelihood that continues to favor the boundary. It cannot, by itself, decide whether a boundary estimate represents a scientifically negligible effect or a parameter that the data estimate poorly. A flat likelihood near the boundary may support a range of small values even when its maximizer is exactly zero.
Worked Example: A Boundary Maximum With Little Separation Nearby
Consider a hypothetical profile log likelihood for a nonnegative variance \(v\): \(\ell(v)=-\tfrac{1}{2}(v/0.02)^2\), with \(v\) measured in squared concentration units. Its derivative for \(v\geq0\) is \(\ell'(v)=-v/(0.02)^2\), so the maximum is at the boundary \(v=0\).
At \(v=0.01\), the log-likelihood difference from the maximum is \(\ell(0.01)-\ell(0)=-\tfrac{1}{2}(0.01/0.02)^2=-0.125\). At \(v=0.02\), it is \(-\tfrac{1}{2}(0.02/0.02)^2=-0.5\). Thus the profile distinguishes larger values more than tiny values, while values close to zero have similar likelihoods. The data in this example do not sharply distinguish zero from a small variance.
On the log scale, an optimizer with lower bound \(\eta=-10\) can only reach \(v=\exp(-10)\approx4.540\times10^{-5}\). Lowering the bound to \(\eta=-20\) permits \(v\approx2.061\times10^{-9}\). The respective log-likelihood differences from zero are approximately \(-\tfrac{1}{2}(4.540\times10^{-5}/0.02)^2=-2.576\times10^{-6}\) and \(-\tfrac{1}{2}(2.061\times10^{-9}/0.02)^2=-5.308\times10^{-15}\). The estimated positive variance shrinks substantially as the bound moves, while the likelihood barely changes. This is evidence of a boundary maximum with little local information about the precise tiny variance—not evidence that one finite lower bound is the true estimate.
Profile likelihood is a natural companion diagnostic: it shows how the likelihood changes as the parameter is fixed at different values while other parameters are re-optimized. Reparameterization asks a different, complementary question: does the optimizer locate and represent the same likelihood behavior when the coordinate changes? A profile that is nearly flat near zero and a transformed estimate that moves with the lower bound both warn against interpreting a displayed small positive number as precise.
Other Useful Transformations
The same logic applies to parameters with other constraints. A positive clearance \(CL\) can be represented as \(\gamma=\log(CL)\), with \(CL=\exp(\gamma)\). A correlation \(\rho\), constrained to \(-1<\rho<1\), can be represented as \(\zeta=\operatorname{atanh}(\rho)\), with \(\rho=\tanh(\zeta)\). The endpoints \(\rho=-1\) and \(\rho=1\) correspond to \(\zeta\to-\infty\) and \(\zeta\to+\infty\), respectively.
Worked Example: A Correlation at Its Upper Bound
Suppose a hypothetical correlation profile over \(-1\leq\rho\leq1\) is \(\ell(\rho)=-\tfrac{1}{2}[(\rho-1.02)/0.08]^2\). Its unconstrained peak is at 1.02, outside the allowed interval, so its constrained maximum is at \(\rho=1\). At that boundary, \(\ell(1)=-\tfrac{1}{2}(0.02/0.08)^2=-0.03125\). At \(\rho=0.99\), \(\ell(0.99)=-\tfrac{1}{2}(0.03/0.08)^2=-0.07031\), so the log-likelihood difference from the boundary is approximately \(-0.03906\).
The Fisher-z coordinates for these two correlations are \(\operatorname{atanh}(1)\to+\infty\) and \(\operatorname{atanh}(0.99)=\tfrac{1}{2}\log(1.99/0.01)\approx2.647\). A finite upper bound on \(\zeta\) therefore corresponds to a correlation strictly below one. If increasing that bound produces estimates ever closer to one with negligible likelihood change, the result is consistent with a boundary maximum and weak separation near it. It does not establish that the true correlation is exactly one, nor does the transformation repair an inadequately supported correlation model.
Transformations must respect the parameter’s actual constraints. For a correlation matrix, transforming each pairwise correlation separately does not automatically guarantee that the resulting matrix is positive definite. The optimizer must use a parameterization that preserves the model’s full constraints, or explicitly check them; otherwise it may be optimizing over invalid covariance structures.
A Practical Diagnostic Sequence
Record the scientific parameter, its units, its allowed range, and whether the boundary is included in the model. For a variance, distinguish an exact zero from a very small positive value.
Use a transformation appropriate to the constraint, such as \(\eta=\log(v)\) for positive values or \(\zeta=\operatorname{atanh}(\rho)\) for an interior correlation. State how boundary values map, including any infinite limits.
Use the same data, model, and likelihood. Convert the transformed estimate to the original units, then compare the objective-function values, fitted predictions, and estimates of other parameters.
If a transformed parameter requires a finite computational bound, change that bound and observe whether the original-scale estimate and likelihood stabilize. Check gradients and other convergence diagnostics rather than relying on a stopping message alone.
Use the profile likelihood and model context to determine whether nearby values have meaningfully worse likelihood. Report weak separation as uncertainty or weak identification, not as proof of an exact zero or endpoint.
Common Misinterpretations and Limitations
- “The transformed estimate is interior, so the boundary problem is solved.” A positive value on a log scale may merely be the finite approximation imposed by a lower bound. Translate it back and check whether it changes as the bound changes.
- “The transformed fit has a better likelihood, so it is a better model.” Equivalent parameterizations have the same exact likelihood optimum. A different reported value can reflect an optimization discrepancy; verify that both fits use the same likelihood and parameter space.
- “A zero variance proves there is no heterogeneity.” A boundary maximizer can occur when the likelihood is flat near zero. The fitted result alone does not distinguish a precisely negligible variance from limited information about small variances.
- “A small transformed-scale gradient proves convergence.” Gradients change under transformations. Check gradients in context, objective values, and the mapped estimate; no single coordinate’s gradient is a universal convergence certificate.
Reparameterization is most useful as a controlled perturbation of the optimization coordinates. If multiple equivalent fits agree in likelihood and map to the same scientific estimate, that agreement is reassuring but does not guarantee strong identification. If they disagree, diagnose the optimization before interpreting the boundary estimate. The transformation cannot replace a suitable model, informative data, or a careful examination of the likelihood.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- For a variance \(v\geq0\), what value of \(\eta=\log(v)\) corresponds to \(v=0\)? Why does this matter in a finite optimization?
- A linear-scale fit reports \(v=0\), while a log-scale fit reports \(v=0.0002\). Name two checks needed before concluding that the boundary result was a scale artifact.
- A variance estimate becomes smaller each time the lower bound on \(\eta=\log(v)\) is decreased, but the likelihood changes negligibly. What does this pattern suggest, and what does it not establish?
- Why can a small gradient in a transformed coordinate not be interpreted as a coordinate-independent convergence check?
- What additional constraint must be considered when reparameterizing correlations that form a correlation matrix?
Solutions. (1) Zero corresponds to \(\eta\to-\infty\), not a finite value, so a finite optimizer can approach but not attain zero through this transformation. (2) Confirm the model, data, likelihood, and allowed parameter space are the same; then compare likelihood values, mapped estimates, gradients, and sensitivity to bounds or stopping tolerances. (3) It suggests that the optimizer is following a boundary with little likelihood separation among tiny variances. It does not prove an exact zero variance or identify the true variance precisely. (4) The chain rule multiplies the gradient by the derivative of the coordinate transformation, so its numerical magnitude depends on the coordinate. (5) The full correlation matrix must remain positive definite; individually valid pairwise correlations need not satisfy that joint constraint.