Transformation-Based Density Estimation For Positive Losses

Why Transform Positive Losses?

Loss amounts are positive, and their distributions may be strongly right-skewed. A standard kernel density estimator places a smooth kernel around each observation. When applied directly near zero, a symmetric kernel can put estimated density on negative losses; near the boundary, it can also distort the estimated shape. A transformation can instead map positive losses to a scale on which ordinary smoothing is more suitable.

The transformation does not eliminate the need to interpret the result on the original loss scale. A density is measured per unit of its variable, and changing variables changes density values. The required adjustment is the Jacobian: omit it, and the result is a density for the transformed variable, not for the original loss.

Definition: Let $X_1,\ldots,X_n$ be independent, identically distributed, exact observations of positive losses, and let $Z=g(X)$, where $g$ is differentiable and one-to-one on the positive loss range. If $\hat f_Z$ is a density estimate for the transformed observations, the corresponding estimate on the original loss scale is $$ \hat f_X(x)=\hat f_Z(g(x))\,|g'(x)|,\qquad x>0. $$ For a kernel $K$ and bandwidth $h>0$, the transformed-scale estimator is $$ \hat f_Z(z)=\frac{1}{nh}\sum_{j=1}^{n}K\!\left(\frac{z-g(X_j)}{h}\right). $$

Throughout the worked examples, use the standard normal kernel $K(u)=\phi(u)$, where $\phi(u)=(2\pi)^{-1/2}\exp(-u^2/2)$. The observations are hypothetical, fully observed losses; no truncation, censoring, or parametric distribution is assumed. A bandwidth is always stated in transformed-scale units. The estimator is an empirical smoothing method, not an assertion that the losses follow a normal distribution.

Change of Variables and the Jacobian

The adjustment follows from preserving probability. For a differentiable, strictly increasing transformation $z=g(x)$, the probability in a small interval around $x$ must equal the probability in the corresponding transformed interval: $$ f_X(x)\,dx=f_Z(g(x))\,dg. $$ Since $dg=g'(x)\,dx$, division by $dx$ gives $f_X(x)=f_Z(g(x))g'(x)$. For a decreasing transformation, the interval length changes at rate $|g'(x)|$, so the general one-to-one formula uses the absolute derivative.

A common choice for positive losses is $g(x)=\ln x$. It maps $x>0$ to the full real line, and $g'(x)=1/x$. Thus $$ \hat f_X(x)=\frac{\hat f_Z(\ln x)}{x}. $$ Another possible choice is $g(x)=\sqrt{x}$, for which $g'(x)=1/(2\sqrt{x})$. The Jacobian depends on the transformation: it is not automatically $1/x$. That factor belongs specifically to the logarithmic transformation.

Worked Example: A Log-Scale Estimate at the Geometric Center

A hypothetical portfolio has three exact loss observations $1$, $e$, and $e^2$ loss units. Estimate the loss density at $x=e$ using $Z=\ln X$, a standard normal kernel, and transformed-scale bandwidth $h=1$. No parametric loss distribution is imposed.

Method and transformed observations. Transform the three values: $z_1=\ln 1=0$, $z_2=\ln e=1$, and $z_3=\ln(e^2)=2$. The query point is $z=\ln e=1$, with $n=3$.

Evaluate the transformed estimate. The standardized differences are $1$, $0$, and $-1$. The standard normal kernel values are $\phi(1)=\phi(-1)=0.2420$ and $\phi(0)=0.3989$, to four decimal places. Therefore, $$ \hat f_Z(1)=\frac{0.2420+0.3989+0.2420}{3(1)} =\frac{0.8829}{3} =0.2943. $$

Return to the loss scale. For $g(x)=\ln x$, the Jacobian at $x=e$ is $1/e$. Hence $$ \hat f_X(e)=\frac{0.2943}{e} =\frac{0.2943}{2.7183} \approx 0.1083. $$

The estimate is approximately $0.1083$ per loss unit. As a check on the method, the transformed estimate is divided by the query loss, not multiplied by it. The common mistake this example avoids is reporting $0.2943$ as the density of $X$; that is the estimate for $Z=\ln X$, on a different scale.

Choosing and Applying a Transformation

The logarithm is especially useful when multiplicative differences are more meaningful than equal dollar differences. On the log scale, losses $x$ and $2x$ are separated by the same distance, $\ln 2$, regardless of the value of $x$. A square-root transformation is another monotone option, but it does not map the positive half-line to the whole real line. With either transformation, the estimate at $x$ uses both the transformed query $g(x)$ and the derivative at that same $x$.

The transformation and bandwidth should be treated as a pair. If the transformed variable is rescaled, its numerical bandwidth must be interpreted in the rescaled units. A bandwidth that is appropriate for logged dollars is not automatically appropriate for square-root dollars. The kernel contributions are averaged and divided by $h$ on the transformed scale; only after that calculation is the Jacobian applied.

Worked Example: Using a Square-Root Transformation

For a hypothetical severity sample, the exact losses are $1$, $4$, and $9$ units. Estimate the density at $x=4$ using $Z=\sqrt{X}$, a standard normal kernel, and $h=1$ on the square-root scale. Assume no parametric distribution.

Transform the observations and query. The transformed sample is $1,2,3$, and $z=\sqrt{4}=2$. The standardized differences $z-z_j$ are $1,0,-1$.

Calculate the transformed density estimate. $$ \hat f_Z(2)=\frac{\phi(1)+\phi(0)+\phi(-1)}{3(1)} =\frac{0.2420+0.3989+0.2420}{3} =0.2943. $$

Apply the correct Jacobian. Since $g'(x)=1/(2\sqrt{x})$, at $x=4$ the Jacobian is $1/(2\cdot2)=1/4$. Therefore, $$ \hat f_X(4)=0.2943\left(\frac14\right)=0.0736. $$

The estimated density is $0.0736$ per original loss unit. The result is checked by using $g'(4)$, not the inverse transformation's derivative $dx/dz$. Using $dx/dz=2\sqrt{x}$ would multiply by $4$ rather than $1/4$, giving the wrong density. The method avoids that inverse-Jacobian trap.

Scale, Units, and Interpretation

A density value is not a probability. Probabilities are obtained by integrating a density over an interval. For a small interval of width $\Delta x$ near $x$, the estimate $\hat f_X(x)\Delta x$ can approximate the probability in that interval, but the density alone need not be less than one. Its units are inverse units of loss: for losses measured in dollars, the density is per dollar.

The Jacobian also explains how density estimates respond to a change of measurement units. For example, expressing a loss in thousands rather than single currency units changes the numerical density, even though the underlying probabilities do not change. Always identify the unit of the query and report the density on that same scale.

Worked Example: Log Smoothing with Losses Reported in Dollars

A hypothetical sample contains exact losses of $\$500$, $\$2{,}000$, and $\$8{,}000$. Estimate the density at $\$2{,}000$ using $Z=\ln(X/2000)$, a standard normal kernel, and bandwidth $h=\ln 2$ on the transformed scale. The losses are treated as exact observations, with no fitted parametric distribution.

Transform the data. The transformed observations are $\ln(0.25)=-\ln 4$, $0$, and $\ln 4$. Since $\ln 4=2\ln 2$, dividing their differences from the query $z=0$ by $h=\ln 2$ gives standardized differences $2,0,-2$. Notice that these are not $-1,0,1$.

Calculate the transformed estimate. Using $\phi(2)=\phi(-2)=0.0540$ and $\phi(0)=0.3989$, and $h=\ln 2=0.6931$, $$ \hat f_Z(0)=\frac{0.0540+0.3989+0.0540}{3(0.6931)} =\frac{0.5069}{2.0793} \approx 0.2438. $$

Apply the Jacobian. The transformation is $g(x)=\ln(x/2000)$, and $g'(x)=1/x$. At $x=2000$, $$ \hat f_X(2000)=\frac{0.2438}{2000} \approx 0.0001219. $$

The estimated density is approximately $0.0001219$ per dollar. As a check, the constant $2000$ inside the logarithm does not change the derivative $1/x$. A frequent error is to treat the transformed observations as one bandwidth apart: they are two bandwidths from the query, so the kernel contributions must use $\phi(2)$, not $\phi(1)$.

Practical Checks and Limitations

Transformation-based smoothing is useful when the original support or skewness makes direct smoothing unattractive, but it does not guarantee a good estimate. The bandwidth still controls the bias-variance tradeoff: a small bandwidth can produce a jagged estimate, while a large bandwidth can obscure features. The earlier material on kernel density estimation and bandwidth selection applies on the scale where smoothing is performed.

Check the target. Evaluate the transformed estimator at $g(x)$, not at $x$ itself.
Check the scale. Use a bandwidth measured in units of $g(X)$.
Check the derivative. Multiply by $|g'(x)|$ to report the density of $X$.
Check the support and data. A logarithm requires strictly positive values. Exact-observation formulas do not account for censoring or truncation.

For a one-to-one differentiable transformation, the Jacobian makes the transformed density estimate correspond to the original scale. It does not restore information lost through censoring, correct sample-selection effects, or make an unsuitable bandwidth appropriate. Those require separate treatment.

Key takeaway: Smooth the transformed observations using a bandwidth on the transformed scale, evaluate at $g(x)$, and multiply by $|g'(x)|$. For logarithmic smoothing, the final factor is $1/x$; for other transformations, derive the Jacobian from that transformation.

Check Your Understanding

For each question, use a standard normal kernel and the stated transformation. Give density estimates to four decimal places unless a different scale is requested.

Exact losses are $1,e,e^2$. Use $Z=\ln X$, $h=1$, and estimate at $x=e$. (Answer: $0.1083$ per loss unit)
Exact losses are $1,4,9$. Use $Z=\sqrt X$, $h=1$, and estimate at $x=9$. (Answer: $0.0386$ per loss unit)
Exact losses are $\$100,\$200,\$400$. Use $Z=\ln(X/200)$, $h=\ln 2$, and estimate at $\$200$. (Answer: approximately $0.0021$ per dollar)
Exact losses are $1,2,3$. Use $Z=X^2$, $h=3$, and estimate at $x=2$. The transformed observations are $1,4,9$. (Answer: approximately $0.3291$ per loss unit)

What You'll Learn