Why Simulate on a Transformed Scale?
Many NLME parameters have natural constraints. A clearance, volume, or rate constant must be positive; a fraction such as bioavailability must lie between zero and one. Yet the usual approximate distribution for an estimated parameter is often normal, and a normal distribution assigns some probability to values outside any finite interval. Directly simulating on the natural scale can therefore produce impossible parameter values.
A transformation offers a direct remedy: work on an unbounded scale where an approximate normal distribution is suitable, then apply the inverse transformation. This tutorial concerns parameter uncertainty in draws around fitted estimates. These draws are distinct from random effects for new individuals and residual error in simulated observations.
For a positive parameter \(\theta>0\), the log transformation is \(\phi=\log(\theta)\), with inverse \(\theta=\exp(\phi)\). For a fraction \(0<\theta<1\), the logit transformation is \(\phi=\log\{\theta/(1-\theta)\}\), with inverse \(\theta=1/\{1+\exp(-\phi)\}\). These transformations permit any finite transformed draw while enforcing the parameter's bounds after inversion.
Model and Assumptions for the Draw
Suppose the fitted transformed-scale estimate is \(\hat{\boldsymbol{\phi}}\), with estimated covariance matrix \(\widehat{\Sigma}_{\phi}\). A common simulation approximation is to draw \(\boldsymbol{\phi}^{(b)}\sim N(\hat{\boldsymbol{\phi}},\widehat{\Sigma}_{\phi})\), then transform each component back to the natural scale. Here \(b\) indexes a simulation draw. The covariance matrix must refer to the transformed estimates and must include the covariances among parameters being drawn.
This approach relies on the transformed estimator being adequately approximated by a multivariate normal distribution in the region carrying most of its probability. It also assumes the supplied covariance matrix is an appropriate estimate of uncertainty for that estimator and parameterization. The transformation guarantees permissible values, but it does not guarantee that the normal approximation is accurate, that the fitted model is adequate, or that every simulated parameter combination is scientifically plausible.
If the fitted estimates and covariance are already reported on the log or logit scale, use them on that scale directly. Do not transform them a second time. If instead the covariance is available on the natural scale, a local approximation can convert it. For a vector transformation \(\boldsymbol{\phi}=g(\boldsymbol{\theta})\), define the Jacobian \(J\) as the matrix of first derivatives of the transformed components with respect to the natural-scale components. The delta method gives \(\widehat{\Sigma}_{\phi}\approx J\widehat{\Sigma}_{\theta}J^\mathsf{T}\), evaluated at the fitted natural-scale estimate.
Why the Covariance Transforms This Way
A first-order Taylor expansion explains the delta-method conversion. Near the estimate \(\hat{\boldsymbol{\theta}}\), the transformed vector is approximately its value at the estimate plus a linear change:
If the natural-scale estimate has covariance \(\widehat{\Sigma}_{\theta}\), the covariance of the linear term is obtained by multiplying the covariance on the left by \(J\) and on the right by \(J^\mathsf{T}\). Thus \(\widehat{\Sigma}_{\phi}\approx J\widehat{\Sigma}_{\theta}J^\mathsf{T}\). For a single positive parameter, \(d\log(\theta)/d\theta=1/\theta\), so its transformed variance is approximately its natural-scale variance divided by \(\hat{\theta}^{\,2}\). For a fraction, \(d\operatorname{logit}(\theta)/d\theta=1/\{\theta(1-\theta)\}\). These are local approximations; for substantial uncertainty or strong nonlinearity, a natural-scale covariance may not be well represented by a normal distribution after transformation.
Worked Example: Drawing an Uncertain Positive Clearance
Suppose a hypothetical fitted model estimates the typical clearance as \(5.00\) L/h, with a standard error of \(0.12\) on the log-clearance scale. For one simulated uncertainty draw, let the standard-normal draw be \(z=1.25\). The fitted log estimate is \(\log(5.00)=1.6094\), and the log-scale draw is
Transforming back gives \(\theta^{(b)}=\exp(1.7594)=5.809\) L/h (approximately). The simulated clearance is positive, as required. This is one draw in the distribution of uncertainty about the fitted typical clearance; it is not a random-effect draw for a particular subject.
The inverse transformation also affects summaries. If a log-scale random variable has mean \(\mu_\phi\) and variance \(\sigma_\phi^2\), its exponent has median \(\exp(\mu_\phi)\) and mean \(\exp(\mu_\phi+\sigma_\phi^2/2)\). Thus exponentiating the log-scale mean gives the median on the natural scale, not generally its mean. For the example's uncertainty distribution, \(\exp(1.6094)=5.00\) L/h is the median, while the implied mean is approximately \(5.00\exp(0.12^2/2)=5.036\) L/h.
Bounded Parameters and Interpreting Intervals
The logit scale works similarly for parameters strictly between zero and one. A symmetric interval on the logit scale generally becomes asymmetric after inverse transformation. This is expected: the transformation compresses values toward zero and one while keeping every finite draw within the interval.
Worked Example: Transforming a Logit-Scale Interval
For a hypothetical bounded parameter such as a bioavailability fraction, suppose the fitted logit estimate is \(0.9445\), with standard error \(0.5952\). A normal-approximation 95% interval on the logit scale is approximately \(0.9445\pm1.96(0.5952)\), or \([-0.222,\ 2.111]\) after rounding. Apply the inverse logit to each endpoint:
The transformed interval is approximately \([0.4447,\ 0.8920]\). It is asymmetric around the inverse-logit estimate, which is about \(0.7200\). Under the stated normal approximation, the interval procedure has approximately 95% coverage in repeated sampling; it does not mean there is a 95% probability that the fixed parameter lies in this particular interval. The inverse transformation is monotone increasing, so transforming both endpoints preserves their order.
Drawing Several Parameters Together
When parameters are correlated, draw them jointly on the scale for which the covariance was estimated. Then apply each parameter's inverse transformation to its corresponding component. Independent draws from marginal distributions discard dependence and can change the distribution of model predictions. Moreover, correlations need not remain numerically the same after nonlinear transformations, so the covariance used for the draw must match the scale on which the joint normal approximation is specified.
Worked Example: A Correlated Clearance and Fraction Draw
Consider hypothetical transformed estimates \(\hat{\phi}_1=\log(5.00)=1.6094\) for clearance and \(\hat{\phi}_2=\operatorname{logit}(0.70)=0.8473\) for a fraction. Suppose their transformed-scale standard errors are \(0.10\) and \(0.30\), respectively, with correlation \(-0.40\). To construct one joint draw, let \(z_1=0.50\) and \(e_2=-1.00\) be independent standard-normal values, and set \(z_2=\rho z_1+\sqrt{1-\rho^2}e_2\). This construction gives standard-normal draws with the specified correlation:
The transformed parameter draws are \(\phi_1^{(b)}=1.6094+(0.10)(0.50)=1.6594\) and \(\phi_2^{(b)}=0.8473+(0.30)(-1.1165)=0.5124\). After inversion, clearance is \(\exp(1.6594)=5.256\) L/h, and the fraction is \(1/\{1+\exp(-0.5124)\}\approx0.6254\). Both values satisfy their constraints, and their joint draw reflects the specified negative correlation on the transformed scale.
Diagnostics, Alternatives, and Limitations
A valid-looking draw is not sufficient evidence that the simulation is reliable. Examine the transformed-scale distribution and the natural-scale results. Useful checks include confirming that every draw is finite and in range; comparing simulated transformed means, standard deviations, and correlations with their targets; and inspecting natural-scale quantiles, skewness, and extreme values. In NLME applications, also check whether the resulting parameter combinations lead to numerically stable predictions and plausible model behavior.
Warning signs include a covariance matrix that is not positive definite, extremely large natural-scale draws, estimates close to a boundary with a poor normal approximation, or intervals so wide that a small part of the transformed normal distribution drives results. A transformation enforces mathematical constraints, not scientific plausibility: for example, an extremely large positive clearance remains positive but may still be implausible. Do not silently discard draws that cause difficulty; investigate the cause, since deletion changes the simulated distribution.
| Approach | What it does | Main concern |
|---|---|---|
| Normal draws on the transformed scale | Draws an unconstrained vector, then maps values into the allowable parameter space | Depends on a suitable transformed-scale normal approximation and covariance |
| Normal draws on the natural scale | Uses a normal approximation directly for the parameter values | Can generate impossible negative or out-of-range values |
| Truncated natural-scale normal draws | Rejects or truncates normal draws outside the allowable range | Changes the distribution; it is not equivalent to an untruncated normal approximation |
The transformed-normal approach is a practical approximation, not a universal solution. It can be weak when the fitted estimate is near a boundary, uncertainty is large, or the transformed likelihood remains markedly skewed. A covariance estimated on the natural scale may also be insufficient for a faithful transformed-scale approximation when curvature is substantial. In such cases, assess whether the chosen approximation is adequate before relying on its predictions or summaries.
Common Mistakes
- Transforming only the point estimate. A valid fitted estimate does not prevent unconstrained normal draws from leaving the parameter space. Draw on the transformed scale and invert each draw.
- Using the wrong covariance scale. A natural-scale covariance is not automatically a transformed-scale covariance. Use the covariance reported for the transformed estimates or convert it with the Jacobian as a local approximation.
- Transforming an interval but calling it symmetric. Inverse transformations generally make intervals asymmetric on the natural scale. Report the transformed endpoints, not a symmetric interval around the natural-scale estimate.
- Assuming the inverse transform solves every problem. It guarantees parameter bounds, not a good normal approximation, plausible parameter combinations, or a well-fitting model.
- Dropping correlation. Independent marginal draws do not represent a correlated joint uncertainty distribution. Preserve the covariance on its stated scale before applying the component-wise inverse transformations.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A positive parameter has a fitted log estimate of \(1.2\). What is its natural-scale fitted value, and what is the natural-scale value for a log-scale draw of \(1.5\)?
- Why does a logit-scale normal draw produce a value strictly between zero and one after inverse transformation?
- A natural-scale positive estimate is \(4.0\), with standard error \(0.4\). Using the scalar delta method, approximate the standard error of its log transformation.
- When parameters are correlated on the transformed scale, why should they be drawn jointly before inversion?
Solutions. (1) The fitted value is \(\exp(1.2)=3.320\); the draw is \(\exp(1.5)=4.482\). (2) For every finite \(x\), \(0<1/\{1+\exp(-x)\}<1\), so the inverse logit lies strictly inside the allowable interval. (3) The log-scale standard error is approximately \(0.4/4.0=0.1000\), since the derivative of \(\log(\theta)\) at \(\theta=4.0\) is \(1/4.0\). (4) A joint draw preserves the specified dependence; independent marginal draws would instead impose zero transformed-scale correlation and could alter the distribution of model predictions.