Log-Scale Optimization For Positive Parameters

Why Optimize on a Log Scale?

Many actuarial likelihood models have parameters that must be positive: an exponential rate, a Pareto shape, a Weibull scale, or a Gamma rate. A numerical optimizer that searches directly over such a parameter can propose zero or a negative value. Those proposals may make the likelihood undefined, even though the parameter is only meant to vary over positive values.

A direct remedy is to optimize over an unrestricted real parameter instead. For a positive parameter \(\lambda\), set \(\theta=\ln\lambda\), so that \(\lambda=e^\theta\). Every finite real \(\theta\) gives a valid positive \(\lambda\). After optimization, convert the result back with \(\hat{\lambda}=e^{\hat{\theta}}\). This technique changes the coordinates used by the optimizer, not the probability model.

Definition: For a parameter \(\lambda>0\), its log-scale parameterization is \(\theta=\ln\lambda\), with inverse transformation \(\lambda=e^\theta\). The transformation is one-to-one between \((0,\infty)\) and the real line. For several positive parameters, apply it separately: \(\theta_j=\ln\lambda_j\) and \(\lambda_j=e^{\theta_j}\).

This is a numerical optimization device. The data likelihood remains the likelihood of the original observations under the original model, evaluated at \(\lambda=e^\theta\). In maximum likelihood estimation, do not multiply that likelihood by a Jacobian for this parameter substitution. A Jacobian is used when transforming a probability density over a random variable; here the parameter is being re-expressed for optimization.

The Transformed Score

Let \(\ell(\lambda)\) be the log-likelihood as a function of a positive parameter, and define \(\tilde{\ell}(\theta)=\ell(e^\theta)\). The chain rule gives the derivative that a log-scale optimizer needs.

\frac{d\tilde{\ell}}{d\theta} = \frac{d\ell}{d\lambda}\frac{d\lambda}{d\theta} = \lambda\frac{d\ell}{d\lambda}, \qquad \lambda=e^\theta.

Because \(\lambda>0\), the transformed score is zero exactly when the original score is zero. Thus, any interior stationary estimate is represented in either parameterization. The second derivative also follows from the chain rule:

\frac{d^2\tilde{\ell}}{d\theta^2} = \lambda\frac{d\ell}{d\lambda} + \lambda^2\frac{d^2\ell}{d\lambda^2}.

The extra first-derivative term means that curvature away from a stationary point need not match the curvature in the original parameter. At a stationary point, the first term is zero. This distinction matters when interpreting a Newton step: the optimizer must use derivatives with respect to the parameter it is actually optimizing.

Key result: A finite log-scale estimate \(\hat{\theta}\) always maps to a positive estimate \(\hat{\lambda}=e^{\hat{\theta}}\). The transformation enforces positivity, but it does not guarantee that an optimum exists at a finite value, that the likelihood is identifiable, or that a numerical routine will find the desired solution.

Worked Examples

Worked Example: Exponential Severity Rate

Suppose three independent, complete loss observations are \(2\), \(3\), and \(4\) units. Assume they are exact observations from an exponential severity distribution with rate \(\lambda>0\) per unit. Find the maximum likelihood estimate using the log-rate parameter \(\theta=\ln\lambda\). Show numerical Newton steps starting from \(\lambda_0=0.5\).

Method and likelihood. For an exponential distribution, \(f(x\mid\lambda)=\lambda e^{-\lambda x}\) for \(x\geq0\). The observations are independent and complete, so, omitting no parameter-dependent terms,

\ell(\lambda)=3\ln\lambda-\lambda(2+3+4) =3\ln\lambda-9\lambda.

Put \(\lambda=e^\theta\). Then \(\tilde{\ell}(\theta)=3\theta-9e^\theta\), with transformed score \(g(\theta)=3-9e^\theta\) and second derivative \(h(\theta)=-9e^\theta\). Newton’s update is \(\theta_{\text{new}}=\theta-g(\theta)/h(\theta)\).

First iteration. The starting value is \(\theta_0=\ln(0.5)=-0.6931\). At this point, \(g(\theta_0)=3-9(0.5)=-1.5\) and \(h(\theta_0)=-9(0.5)=-4.5\). Therefore,

\theta_1=-0.6931-\frac{-1.5}{-4.5}=-1.0264, \qquad \lambda_1=e^{-1.0264}\approx0.3583.

Second iteration. Using \(\lambda_1\approx0.3583\), \(g(\theta_1)=3-9(0.3583)\approx-0.2247\), and \(h(\theta_1)\approx-3.2247\). Thus \(\theta_2=-1.0264-(-0.2247/-3.2247)\approx-1.0961\), giving \(\lambda_2\approx e^{-1.0961}=0.3343\).

Stationary estimate and check. Setting the transformed score to zero gives \(3-9e^\theta=0\), so \(e^{\hat{\theta}}=1/3\). Hence \(\hat{\theta}=\ln(1/3)\approx-1.0986\), and \(\hat{\lambda}=e^{\hat{\theta}}=1/3=0.3333\) per unit. Substitution gives \(g(\hat{\theta})=3-9(1/3)=0\), confirming the score equation. Newton’s iterates are approaching this value.

Common mistake avoided. Differentiating \(3\theta-9e^\theta\) as if \(e^\theta\) were \(\theta\) would give the wrong score. The derivative of \(e^\theta\) is \(e^\theta\), so the transformed score is \(3-9e^\theta\), not a score copied unchanged from the original scale.

Worked Example: Weibull Scale with Known Shape

Assume four independent, complete observations \(1,2,2,1\) come from a Weibull severity distribution with known shape \(k=2\) and unknown scale \(\eta>0\). The density is \(f(x\mid\eta)=\frac{k}{\eta}(x/\eta)^{k-1}\exp[-(x/\eta)^k]\), for \(x>0\). Estimate \(\eta\) by optimizing over \(\phi=\ln\eta\).

Method and likelihood. For \(k=2\), the log-likelihood, omitting terms independent of \(\eta\), is \(\ell(\eta)=-nk\ln\eta-\sum x_j^k/\eta^k\). Here \(n=4\) and \(\sum x_j^2=1+4+4+1=10\), so with \(\eta=e^\phi\),

\tilde{\ell}(\phi)=-8\phi-10e^{-2\phi}.

The transformed score is \(-8+20e^{-2\phi}\). Set it equal to zero: \(20e^{-2\hat{\phi}}=8\), so \(e^{-2\hat{\phi}}=0.4\). Therefore \(\hat{\phi}=-\frac12\ln(0.4)=\frac12\ln(2.5)\approx0.4581\), and

\hat{\eta}=e^{\hat{\phi}}=\sqrt{2.5}\approx1.5811.

Check. At this estimate, \(e^{-2\hat{\phi}}=1/2.5=0.4\), so the transformed score is \(-8+20(0.4)=0\). The scale estimate is positive, as required, and is in the same units as the observations.

Common mistake avoided. The Weibull scale term in the log-likelihood is \(-nk\ln\eta\), not \(-n\ln\eta\). Omitting the shape factor \(k=2\) changes the score and produces an incorrect estimate.

Worked Example: Pareto Shape on the Log Scale

A hypothetical portfolio has three independent, complete losses of \(100\), \(200\), and \(400\) dollars. Assume a Pareto Type I model with fixed minimum \(x_m=100\) dollars and unknown shape \(\alpha>0\), with density \(f(x\mid\alpha)=\alpha x_m^\alpha x^{-(\alpha+1)}\) for \(x\geq x_m\). Estimate the shape using \(\zeta=\ln\alpha\), and report both the transformed and original-scale estimates.

Method and likelihood. The parameter-dependent part of the log-likelihood can be written \(n\ln\alpha-\alpha T\), where \(T=\sum_{j=1}^n\ln(x_j/x_m)\). The ratios are \(1,2,4\), so

T=\ln1+\ln2+\ln4=0+0.693147+1.386294=2.079441.

With \(\alpha=e^\zeta\), the transformed log-likelihood, up to an additive constant, is \(\tilde{\ell}(\zeta)=3\zeta-2.079441e^\zeta\). Its score is \(3-2.079441e^\zeta\). Setting the score to zero gives \(e^{\hat{\zeta}}=3/2.079441\approx1.4427\), so \(\hat{\zeta}=\ln(1.4427)\approx0.3665\) and \(\hat{\alpha}=e^{\hat{\zeta}}\approx1.4427\).

Check. Substituting the estimate into the score gives \(3-2.079441(1.4427)\approx0\); the small discrepancy is rounding. The estimate is a positive, dimensionless shape parameter.

Common mistake avoided. The Pareto likelihood uses the log ratios \(\ln(x_j/x_m)\), not the raw losses. Using \(\ln(100)+\ln(200)+\ln(400)\) in place of \(T\) would ignore the specified threshold and give a different, incorrect estimate.

Practical Guidance for Numerical Optimization

A log transformation is especially useful when a numerical method evaluates the likelihood repeatedly and needs to stay within the parameter space. A typical implementation stores \(\theta\), computes \(\lambda=e^\theta\), evaluates the original log-likelihood at \(\lambda\), and returns derivatives with respect to \(\theta\). For a vector of positive parameters, transform each component and apply the chain rule to each corresponding derivative.

Identify the constraint.
Confirm that the model requires the parameter to be strictly positive, and state its meaning and units.

Transform the parameter.
Set \(\theta=\ln\lambda\), write \(\lambda=e^\theta\), and substitute into the original log-likelihood.

Optimize the transformed objective.
Use the score and, when needed, the Hessian with respect to \(\theta\). Do not mix derivatives from the two parameter scales.

Translate and report.
Convert the result with \(\hat{\lambda}=e^{\hat{\theta}}\), then report the original parameter, its units if it has units, and suitable rounding.

The log transformation maps values near zero to very negative values and very large positive parameters to large values of \(\theta\). Consequently, an estimate on the boundary \(\lambda=0\), if relevant to a particular model, cannot be represented by any finite \(\theta\); the optimizer may instead keep moving toward increasingly negative values. A log transformation also does not enforce constraints such as a parameter lying between zero and one. That interval requires a different transformation, and the chosen parameterization must match the model’s actual constraints.

Common Exam Traps

Forgetting the chain rule. If \(\theta=\ln\lambda\), then \(d\lambda/d\theta=\lambda\). The transformed score is \(\lambda\) times the original score.
Adding a Jacobian to the likelihood. Reparameterizing an unknown parameter for maximum likelihood does not change the data likelihood by a Jacobian factor.
Reporting the transformed estimate as the model parameter. If the optimizer returns \(\hat{\theta}\), the positive parameter estimate is \(e^{\hat{\theta}}\), not \(\hat{\theta}\).
Dropping parameter-dependent terms or model factors incorrectly. Terms constant in the parameter may be omitted, but terms such as \(nk\ln\eta\) are not constant when \(\eta\) is estimated.
Assuming positivity proves a successful optimization. The transformation keeps finite iterates positive; it does not by itself establish that the likelihood has a finite optimum or that the numerical search found the relevant solution.

Key takeaway: For a strictly positive parameter, optimizing in its logarithm gives an unconstrained real search while preserving the likelihood’s stationary points. Apply the chain rule, optimize the correctly transformed log-likelihood, and exponentiate the final result to recover the parameter in its original units.

Check Your Understanding

For each problem, identify the log-scale parameterization and return the estimate on the original scale.

Three complete independent observations \(1,2,3\) follow an exponential distribution with unknown rate \(\lambda>0\). Find the maximum likelihood estimates of \(\theta=\ln\lambda\) and \(\lambda\). (Answer: \(\hat{\theta}=\ln(0.5)\approx-0.6931\), \(\hat{\lambda}=0.5000\) per unit.)
Three complete observations \(1,1,3\) follow a Weibull distribution with known shape \(k=2\) and unknown scale \(\eta>0\). Find \(\hat{\eta}\) and \(\hat{\phi}\), where \(\phi=\ln\eta\). (Answer: \(\hat{\eta}=\sqrt{11/3}\approx1.9149\), \(\hat{\phi}=\frac12\ln(11/3)\approx0.6496\).)
Three complete losses \(50,100,200\) follow a Pareto Type I model with fixed minimum \(x_m=50\) and unknown shape \(\alpha>0\). Find \(\hat{\alpha}\) and \(\hat{\zeta}\), where \(\zeta=\ln\alpha\). (Answer: \(\hat{\alpha}=3/\ln8\approx1.4427\), \(\hat{\zeta}\approx0.3665\).)
Four complete observations \(1,2,3,2\) follow a Gamma distribution with known shape \(k=2\) and unknown rate \(\beta>0\); its density is proportional to \(\beta^k x^{k-1}e^{-\beta x}\). Find the rate estimate and its log. (Answer: \(\hat{\beta}=nk/\sum x_j=8/8=1\) per unit, and \(\ln\hat{\beta}=0\).)
A numerical optimizer reports \(\hat{\theta}=-0.4\) for a model whose scale parameter is \(\eta=e^\theta\). Convert this result to the original scale, to four decimal places. (Answer: \(\hat{\eta}=e^{-0.4}\approx0.6703\).)

What You'll Learn