Daniels' Saddlepoint Approximation for Densities

Why Approximate a Density Directly?

A normal approximation can describe the central shape of a sampling distribution, but its accuracy may deteriorate in the tails or when the distribution is notably skewed. Daniels’ saddlepoint approximation uses the cumulant-generating function (CGF) to adapt its exponential decay and local scale to the point where the density is being evaluated. It is especially useful when a density, rather than a tail probability, is the quantity of interest.

The basic construction is for an independent, identically distributed sample \(X_1,\ldots,X_n\) with a continuous distribution. Write \(S_n=\sum_{i=1}^n X_i\) and \(\bar X_n=S_n/n\). The approximation applies at a value \(x\) in the interior of the range of possible tilted means. At such a point, a saddlepoint \(t\) selects an exponential tilt under which the mean of each observation is exactly \(x\).

CGF, Saddlepoint, and Rate Function

Definition: The cumulant-generating function of \(X_1\) is \(K(t)=\log E[e^{tX_1}]\), for \(t\) in the interior \(D\) of the set where this expectation is finite. Where derivatives exist, \(K'(t)\) is the mean and \(K''(t)\) is the variance under exponential tilting by \(t\). For a target \(x\), a saddlepoint is a solution \(t=t(x)\in D\) of \(K'(t)=x\). The associated rate function at \(x\) is \(I(x)=t(x)x-K(t(x))\).

For a nondegenerate distribution, \(K''(t)>0\) wherever the tilted distribution has a finite, positive variance. Thus \(K'\) is strictly increasing there, so there is at most one solution to the saddlepoint equation. A solution exists when \(x\) is in the range of \(K'\) on the CGF’s domain. The variance \(K''(t(x))\) determines the local scale of the approximation, while \(I(x)\) determines its exponential decay.

For a sum, the CGF is \(nK(t)\), because the observations are independent. Its saddlepoint equation at \(s=nx\) is \(nK'(t)=s\), which is the same equation \(K'(t)=x\). This is why it is convenient to describe the approximation using the per-observation CGF.

Daniels’ Density Approximation

Theorem: Suppose \(X_1,\ldots,X_n\) are independent and identically distributed with a nondegenerate absolutely continuous distribution. Let \(K(t)\) be finite on an open interval \(D\), with \(K''(t)>0\). Fix \(x\) for which there is a solution \(t(x)\in D\) to \(K'(t)=x\). Assume the exponentially tilted distribution at \(t(x)\) satisfies a local central limit theorem for densities at its mean; a sufficient regularity condition is that its characteristic function, raised to some positive integer power, is integrable. Then, as \(n\) tends to infinity,

f_{S_n}(nx) \sim \frac{\exp\{-nI(x)\}}{\sqrt{2\pi nK''(t(x))}}.

Equivalently, for the sample mean,

f_{\bar X_n}(x) \sim \sqrt{\frac{n}{2\pi K''(t(x))}}\, \exp\{-nI(x)\}.

Here \(\sim\) means that the ratio of the exact density to the displayed approximation tends to \(1\) at the fixed point \(x\).

The local limit condition is important: a weak central limit theorem alone does not guarantee convergence of densities at a point. For practical use, the formula is typically most informative where the saddlepoint exists and the tilted density is sufficiently regular. Stronger smoothness conditions can yield sharper relative-error statements, but the leading approximation above does not require stating such a rate.

Derivation by Exponential Tilting

The density form follows by changing to a probability distribution under which \(x\) is the mean, then applying a local normal approximation at that mean. This derivation also shows why both \(I(x)\) and \(K''(t(x))\) appear.

Define the tilted distribution.
For \(t=t(x)\), its one-observation density is \(q_t(y)=e^{ty-K(t)}f(y)\). It integrates to one by the definition of \(K(t)\), and has mean \(K'(t)=x\) and variance \(K''(t)\).

Relate the sum densities.
For a sample with sum \(s\), the product of the original densities equals the product of the tilted densities times \(\exp\{nK(t)-ts\}\). Integrating over the sample values with sum \(s\) gives \(f_{S_n}(s)=\exp\{nK(t)-ts\}q_{t,n}(s)\), where \(q_{t,n}\) is the tilted sum density.

Approximate the tilted density at its mean.
Under the tilted distribution, \(S_n\) has mean \(nx\) and variance \(nK''(t)\). The local central limit theorem gives \(q_{t,n}(nx)\sim [2\pi nK''(t)]^{-1/2}\).

Substitute \(s=nx\) in the density identity. Since \(nK(t)-tnx=-n[tx-K(t)]=-nI(x)\), the sum-density approximation is

f_{S_n}(nx) \sim e^{-nI(x)}\frac{1}{\sqrt{2\pi nK''(t(x))}}.

To obtain the density of the sample mean, use the change of variable \(S_n=n\bar X_n\). Its Jacobian is \(n\), so \(f_{\bar X_n}(x)=n f_{S_n}(nx)\). Multiplying the sum approximation by \(n\) gives the stated sample-mean formula. This completes the derivation. \(\square\)

Worked Applications

Worked Example: A Normal Sample Gives the Exact Density

Suppose \(X_i\sim N(\mu,\sigma^2)\), with \(\sigma^2>0\). Find the saddlepoint approximation to the density of \(\bar X_n\) at \(x\), and check it against the exact distribution.

The CGF is \(K(t)=\mu t+\sigma^2t^2/2\), so \(K'(t)=\mu+\sigma^2t\) and \(K''(t)=\sigma^2\). Solving \(K'(t)=x\) gives \(t(x)=(x-\mu)/\sigma^2\). Then

I(x)=t(x)x-K(t(x)) =\frac{(x-\mu)^2}{2\sigma^2}.

Daniels’ sample-mean formula is therefore

f_{\bar X_n}(x)\approx \sqrt{\frac{n}{2\pi\sigma^2}}\, \exp\left\{-\frac{n(x-\mu)^2}{2\sigma^2}\right\}.

But \(\bar X_n\sim N(\mu,\sigma^2/n)\), whose exact density is precisely the expression on the right. In this special model the saddlepoint approximation is exact, providing a check on the exponent and the sample-mean scale factor.

Worked Example: An Exponential Sample Mean

Let \(X_i\) be exponential with rate \(2\), and take \(n=5\). Approximate the density of \(\bar X_5\) at \(x=0.4\), then compare it with the exact density.

For an exponential variable with rate \(2\), \(K(t)=-\log(1-t/2)\) for \(t<2\). Thus \(K'(t)=1/(2-t)\) and \(K''(t)=1/(2-t)^2\). At \(x=0.4\), the saddlepoint solves \(1/(2-t)=0.4\), giving \(t= -0.5\). In particular, \(K''(t)=x^2=0.16\). The rate function is

I(0.4)=t x-K(t)=2x-1-\log(2x) =0.8-1-\log(0.8) \approx 0.0231436.

Therefore,

f_{\bar X_5}(0.4)\approx \sqrt{\frac{5}{2\pi(0.16)}}\,e^{-5(0.0231436)} \approx 2.2302(0.8907) \approx 1.986.

For comparison, \(S_5\) has a gamma distribution with shape \(5\) and rate \(2\). The exact sample-mean density is

f_{\bar X_5}(0.4) =5\,\frac{2^5(5\cdot0.4)^4e^{-2(5\cdot0.4)}}{\Gamma(5)} =5\,\frac{32\cdot16\cdot e^{-4}}{24} \approx 1.9537.

The approximation is close, while not exact at this finite sample size. The calculation also illustrates how a saddlepoint outside zero handles a value below the exponential mean of \(0.5\).

Worked Example: A Gamma Sample Mean

Suppose \(X_i\sim\operatorname{Gamma}(\alpha=2,\text{ rate } \beta=3)\), with \(n=4\). Approximate the density of \(\bar X_4\) at \(x=0.5\), and check it using the exact gamma density.

For a gamma variable with shape \(\alpha\) and rate \(\beta\), \(K(t)=-\alpha\log(1-t/\beta)\) for \(t<\beta\). Hence \(K'(t)=\alpha/(\beta-t)\) and \(K''(t)=\alpha/(\beta-t)^2\). The saddlepoint satisfies \(\beta-t=\alpha/x\), so \(t=3-2/0.5=-1\). At this root \(K''(t)=x^2/\alpha=0.125\), and

I(0.5)=\beta x-\alpha-\alpha\log\left(\frac{\beta x}{\alpha}\right) =1.5-2-2\log(0.75) \approx 0.0753641.

The sample-mean approximation is

f_{\bar X_4}(0.5)\approx \sqrt{\frac{4}{2\pi(0.125)}}\,e^{-4(0.0753641)} \approx 2.2568(0.7397) \approx 1.669.

The sum \(S_4\) has a gamma distribution with shape \(8\) and rate \(3\). Since \(\bar X_4=S_4/4\), its exact density at \(0.5\) is

f_{\bar X_4}(0.5) =4\,\frac{3^8(4\cdot0.5)^7e^{-3(4\cdot0.5)}}{\Gamma(8)} =4\,\frac{6561\cdot128\cdot e^{-6}}{5040} \approx 1.652.

The saddlepoint value is again close to the exact value. In both gamma examples the CGF domain contains the computed root, and \(K''(t)>0\), as required.

Scaling, Interpretation, and a Common Pitfall

The approximation separates the density into an exponential factor and a local scale factor. The term \(e^{-nI(x)}\) reflects how far the target mean is from the model’s ordinary mean, while \(K''(t(x))\) is the variance under the tilt that moves the mean to \(x\). The same construction therefore responds to the shape of the distribution at the target, rather than using one fixed normal curve everywhere.

A frequent error is to use the sum-density formula as though it were already the sample-mean density. The two densities are not numerically the same: because \(S_n=n\bar X_n\), the change-of-variable formula requires a factor of \(n\). Omitting it gives a sample-mean density too small by a factor of \(n\). Conversely, multiplying the sum-density approximation by \(n\) twice makes it too large by that factor.

Another important distinction is that this tutorial concerns densities for continuous distributions. A density value is not a probability at a point. For lattice variables, such as a binomial sum, point probabilities require a lattice saddlepoint approximation rather than interpreting a continuous density formula as a point mass.

Key takeaway: For a continuous iid sample, solve \(K'(t)=x\), compute \(I(x)=tx-K(t)\) and \(K''(t)\), then use \(\exp\{-nI(x)\}/\sqrt{2\pi nK''(t)}\) for the density of the sum at \(nx\). Multiply by \(n\) to approximate the density of the sample mean at \(x\).

Check Your Understanding

Use the definitions and scaling argument above to check each step.

For a nondegenerate distribution with \(K''(t)>0\), why can the saddlepoint equation have at most one solution?
If \(K(t)=\mu t+\sigma^2t^2/2\), solve the saddlepoint equation at a general \(x\) and find \(I(x)\).
Explain why the tilted sum density at \(nx\) has approximate scale \(1/\sqrt{2\pi nK''(t(x))}\).
State the change-of-variable factor that converts a density approximation for \(S_n\) into one for \(\bar X_n\).
Why should Daniels’ continuous density formula not be read directly as an approximation to a binomial point probability?