Lognormal Severity Maximum Likelihood

Why Logarithms Simplify Lognormal Estimation

A lognormal severity model is useful when losses are positive and their logarithms are plausibly described by a normal distribution. For complete observations, the key estimation step is to transform each loss using the natural logarithm, estimate the mean and standard deviation on that log scale, and then interpret those estimates in the original loss units.

The transformation does not mean that the observed losses themselves are normally distributed. It is the logarithms of the losses that follow a normal distribution. This distinction affects both the likelihood and the meaning of the fitted parameters. The estimation method below is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.

Definition: A positive loss \(X\) has a lognormal distribution with parameters \(\mu\) and \(\sigma\), written \(X\sim\operatorname{Lognormal}(\mu,\sigma^2)\), if \(\ln X\sim N(\mu,\sigma^2)\), where \(\sigma>0\). Thus, \(\mu\) is the mean of \(\ln X\), and \(\sigma\) is the standard deviation of \(\ln X\). These are not, in general, the mean and standard deviation of \(X\).

Suppose \(x_1,\ldots,x_n\) are independent, complete observations of losses from this model. Complete means that each recorded value is the actual loss, rather than an observation altered by a deductible, limit, truncation, or censoring. Each observation must be strictly positive because the logarithm is not defined at zero.

The Likelihood and Its Estimates

The density of \(X\) follows from the normal density of \(\ln X\). The transformation from the log scale back to the loss scale contributes a Jacobian factor \(1/x\):

f_X(x;\mu,\sigma) = \frac{1}{x\sigma\sqrt{2\pi}} \exp\left(-\frac{(\ln x-\mu)^2}{2\sigma^2}\right), \qquad x>0,\ \sigma>0.

For independent observations, the likelihood is the product of these densities. Taking its logarithm gives a sum of terms, including \(-\sum_i\ln x_i\). That term is essential to the correct likelihood on the loss scale, even though it does not affect the estimates of \(\mu\) and \(\sigma\).

\ell(\mu,\sigma) = -\sum_{i=1}^n\ln x_i -n\ln\sigma -\frac{n}{2}\ln(2\pi) -\frac{1}{2\sigma^2}\sum_{i=1}^n(\ln x_i-\mu)^2.

The parameters can be estimated by maximizing this log-likelihood. Differentiating with respect to \(\mu\) and setting the score to zero gives the average of the observed logarithms. Substituting that value into the score for \(\sigma\) gives the maximum likelihood estimate of the log-scale variance.

Theorem: If the complete positive observations are not all identical, the lognormal maximum likelihood estimates are

\hat{\mu} = \frac{1}{n}\sum_{i=1}^n\ln x_i, \qquad \hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n(\ln x_i-\hat{\mu})^2, \qquad \hat{\sigma} = \sqrt{\hat{\sigma}^2}.

The divisor in the estimate of \(\sigma^2\) is \(n\), not \(n-1\). Maximum likelihood estimates maximize the likelihood; they are not required to be unbiased estimates. The familiar sample variance with divisor \(n-1\) is a different estimator.

Why the Formulas Give a Maximum

Write \(z_i=\ln x_i\) and \(Q(\mu)=\sum_i(z_i-\mu)^2\). The terms in the log-likelihood that depend on \(\mu\) and \(\sigma\) are \(-n\ln\sigma-Q(\mu)/(2\sigma^2)\). The score for \(\mu\) is

\frac{\partial\ell}{\partial\mu} = \frac{\sum_{i=1}^n(z_i-\mu)}{\sigma^2}.

Since \(\sigma^2>0\), this score is zero exactly when \(\mu=\bar z\), the mean of the log observations. The second derivative with respect to \(\mu\) is \(-n/\sigma^2<0\), so this is the unique maximum in \(\mu\) for fixed \(\sigma\).

At \(\mu=\bar z\), let \(Q=\sum_i(z_i-\bar z)^2\). The score for \(\sigma\) is \(-n/\sigma+Q/\sigma^3\). Setting it to zero yields \(\sigma^2=Q/n\). If \(Q>0\), the profile log-likelihood has second derivative \(-2n/\sigma^2<0\) at this solution, and it tends downward as \(\sigma\) approaches zero or infinity. Thus the stationary point is the maximum. If all observations are identical, then \(Q=0\): the likelihood increases without bound as \(\sigma\) approaches zero, so there is no finite maximum likelihood estimate with \(\sigma>0\).

Worked Estimation Examples

Worked Example: Three Losses on a Geometric Scale

A hypothetical portfolio has three independent, complete losses of \(100\), \(100e\), and \(100e^2\) dollars. Assume a lognormal severity model. Estimate \(\mu\) and \(\sigma\), and check the calculation.

Method. Take natural logarithms, calculate their mean, and then calculate their squared deviations using divisor \(n\).

Log observations. They are \(\ln 100\), \(\ln 100+1\), and \(\ln 100+2\). Their mean is \[ \hat{\mu} = \frac{(\ln 100)+(\ln 100+1)+(\ln 100+2)}{3} = \ln 100+1. \]

Log-scale variance and standard deviation. The deviations from the mean are \(-1,0,1\), so \[ \hat{\sigma}^2 = \frac{(-1)^2+0^2+1^2}{3} = \frac{2}{3}, \qquad \hat{\sigma} = \sqrt{\frac{2}{3}} \approx 0.8165. \]

Check. The fitted log mean is the middle of the three log observations, and the squared deviations sum to \(2\). Dividing by \(3\), rather than \(2\), gives the maximum likelihood estimate \(2/3\). The estimate is finite because the observations are not all identical.

Common mistake avoided. Using the sample standard deviation would give \(\sqrt{2/2}=1\). That is not the lognormal maximum likelihood estimate; the correct estimate uses divisor \(n=3\).

Worked Example: Unevenly Spaced Log Losses

A hypothetical insurer observes four independent, complete losses of \(50e^{-1}\), \(50\), \(50e\), and \(50e^2\) dollars. Under a lognormal model, estimate the two parameters and state the fitted median loss.

Method. The log observations are \(\ln 50-1\), \(\ln 50\), \(\ln 50+1\), and \(\ln 50+2\). Their offsets from \(\ln 50\) are \(-1,0,1,2\), which have mean \(1/2\). Therefore, \[ \hat{\mu}=\ln 50+\frac{1}{2}. \]

Subtracting the mean offset \(1/2\) gives deviations \(-3/2,-1/2,1/2,3/2\). The sum of their squares is \[ \frac{9}{4}+\frac{1}{4}+\frac{1}{4}+\frac{9}{4}=5. \] Thus \[ \hat{\sigma}^2=\frac{5}{4}=1.25, \qquad \hat{\sigma}=\sqrt{1.25}\approx 1.118. \]

Interpretation and check. For a lognormal distribution, the median is \(\exp(\mu)\), since the median of the normally distributed logarithm is \(\mu\). The fitted median is therefore \(\exp(\hat{\mu})=50e^{1/2}\) dollars. The mean log offset is \(1/2\), and the four deviations sum to zero, as required for deviations from their mean.

Common mistake avoided. The fitted median is not \(\hat{\mu}\) dollars. The estimate \(\hat{\mu}\) is on the log scale; exponentiating it returns the median in dollars.

Worked Example: Changing Loss Units

A hypothetical analyst records three complete losses as \(2e^{-1}\), \(2\), and \(2e\), with amounts expressed in thousands of dollars. Estimate the lognormal parameters first in thousands of dollars and then in dollars. Explain which estimate changes.

Estimation in thousands. The log observations are \(\ln 2-1\), \(\ln 2\), and \(\ln 2+1\). Hence \[ \hat{\mu}_{\text{thousands}}=\ln 2, \qquad \hat{\sigma}_{\text{thousands}}^2 = \frac{(-1)^2+0^2+1^2}{3} = \frac{2}{3}, \qquad \hat{\sigma}_{\text{thousands}}=\sqrt{\frac{2}{3}}\approx 0.8165. \]

Convert the observations to dollars. The losses are \(2000e^{-1}\), \(2000\), and \(2000e\). Their log observations are \(\ln 2000-1\), \(\ln 2000\), and \(\ln 2000+1\), so \[ \hat{\mu}_{\text{dollars}}=\ln 2000=\ln 2+\ln 1000, \qquad \hat{\sigma}_{\text{dollars}}=\sqrt{\frac{2}{3}}\approx 0.8165. \]

Check. Multiplying every loss by \(1000\) adds \(\ln 1000\) to every log observation. It shifts their mean by \(\ln 1000\) but leaves each deviation from the mean unchanged. Thus the estimate of \(\mu\) depends on the unit of measurement, while the estimate of \(\sigma\) does not.

Common mistake avoided. A change from thousands of dollars to dollars changes the numerical log-scale location. It does not multiply \(\hat{\sigma}\) by \(1000\), because \(\sigma\) describes variation in log losses.

Interpretation, Conditions, and Exam Traps

The fitted parameters describe the distribution on the log scale. In particular, \(\exp(\hat{\mu})\) is the fitted median loss, while the fitted arithmetic mean loss is \(\exp(\hat{\mu}+\hat{\sigma}^2/2)\). These quantities answer different questions: a lognormal distribution is typically right-skewed, so its mean exceeds its median when \(\sigma>0\). Do not report \(\hat{\mu}\) itself as a dollar amount or treat \(\hat{\sigma}\) as the standard deviation of the losses.

The likelihood formulas here require independent observations from one lognormal model and actual, strictly positive loss amounts. If losses are recorded after a deductible or limit, or if observations are truncated or censored, their likelihood contributions differ; applying the complete-data formulas without accounting for that observation process is not justified.

Using \(n-1\). The maximum likelihood estimate of \(\sigma^2\) uses the average squared deviation of the log observations, with divisor \(n\).
Forgetting the logarithm. Both parameter estimates are calculated from \(\ln x_i\), not directly from the losses.
Dropping the density transformation. The loss-scale density includes \(1/x\). Although this term does not change the estimates of \(\mu\) and \(\sigma\), omitting it gives an incorrect loss-scale likelihood.
Ignoring the all-equal case. If every observed loss is identical, the likelihood has no finite maximum with \(\sigma>0\). The usual formulas then produce the boundary value \(\hat{\sigma}=0\), which is not an admissible lognormal parameter.

A useful final check is to confirm that the estimated log mean lies between the smallest and largest log observations, that the deviations from it sum to zero, and that the estimate of log-scale variance is nonnegative. If all observations are distinct from at least one other observation, the variance estimate is positive and the finite-parameter formula applies.

Check Your Understanding

For each hypothetical data set, assume independent, complete positive losses from one lognormal model. Calculate the requested estimates on the stated scale.

Three losses are \(20\), \(20e\), and \(20e^2\) dollars. Find \(\hat{\mu}\) and \(\hat{\sigma}\). (Answer: \(\hat{\mu}=\ln 20+1\), \(\hat{\sigma}=\sqrt{2/3}\approx 0.8165\).)
Four losses are \(8e^{-1}\), \(8\), \(8e\), and \(8e^2\) dollars. Find \(\hat{\mu}\) and \(\hat{\sigma}^2\). (Answer: \(\hat{\mu}=\ln 8+0.5\), \(\hat{\sigma}^2=1.25\).)
Three losses are \(30e^{-2}\), \(30\), and \(30e^2\) dollars. Find the log-scale estimates and check whether the data support a finite estimate with \(\sigma>0\). (Answer: \(\hat{\mu}=\ln 30\), \(\hat{\sigma}^2=8/3\), \(\hat{\sigma}=\sqrt{8/3}\approx 1.633\); the estimate is finite because the losses are not all identical.)
Four losses, in thousands of dollars, are \(4e^{-1}\), \(4e\), \(4e^2\), and \(4e^2\). Find \(\hat{\mu}\) and \(\hat{\sigma}^2\). (Answer: \(\hat{\mu}=\ln 4+1\), \(\hat{\sigma}^2=1.5\).)
Four losses are \(10\), \(10e\), \(10e^2\), and \(10e^3\) dollars. Find \(\hat{\mu}\), \(\hat{\sigma}\), and the fitted median loss. (Answer: \(\hat{\mu}=\ln 10+1.5\), \(\hat{\sigma}=\sqrt{1.25}\approx 1.118\), fitted median \(=10e^{1.5}\) dollars.)

What You'll Learn