Saddlepoint Approximations for Exponential-Family Sums

Why Exponential Families Make Saddlepoints Explicit

For a sum of observations, the saddlepoint is the exponential tilt that makes the observed value typical under a changed probability law. In a regular exponential family, both the cumulant-generating function and the tilt have a direct expression in terms of the log-partition function. That structure often turns the saddlepoint equation into an explicit calculation.

This tutorial applies the saddlepoint equation and exponential tilting to sums from one-parameter exponential families. The resulting density and lattice-tail formulas are approximations, not exact identities. Their usefulness comes from evaluating a local Gaussian approximation under a tilted law, while retaining the exponential factor that captures the tail's scale.

Assume the log-partition function is finite and twice differentiable on \(\mathcal H\), with \(A''(\eta)>0\). The parameter shift \(\eta+t\) must remain in \(\mathcal H\). These conditions give \(K(t)=n\{A(\eta+t)-A(\eta)\}\), \(K'(t)=nA'(\eta+t)\), and \(K''(t)=nA''(\eta+t)\). Thus the saddlepoint for an observed sum \(s\) solves \(K'(\hat t)=s\). The second derivative is positive, so \(K'\) is strictly increasing: whenever a solution exists, it is unique.

Existence and Meaning of the Saddlepoint

The possible values of \(A'(\eta+t)\) range over the mean-parameter space of \(T(Y_i)\). Therefore a finite saddlepoint exists when \(s/n\) lies in the interior of that space. At the solution, the tilted family has parameter \(\eta+\hat t\), and its expected sum is exactly \(s\). This is the central convenience: the observed value is the mean under the tilted distribution.

The uniqueness claim follows because \(K''(t)=nA''(\eta+t)>0\), so \(K'\) is strictly increasing. The specified interior condition ensures that \(s/n\) is attained as \(A'(\eta+\hat t)\), giving existence as well. To see the density formula, let \(P_{\hat t}\) denote the law obtained by tilting the distribution of \(S\) by \(\hat t\). Its density is $$ f_{\hat t}(x)=\exp\{\hat t x-K(\hat t)\}f_S(x). $$ Rearranging at \(x=s\) gives the exact identity $$ f_S(s)=\exp\{K(\hat t)-\hat t s\}f_{\hat t}(s). $$ Under \(P_{\hat t}\), the mean of \(S\) is \(K'(\hat t)=s\), and its variance is \(K''(\hat t)\). A local central limit approximation at this mean gives \(f_{\hat t}(s)\approx 1/\sqrt{2\pi K''(\hat t)}\), yielding the displayed formula. The approximation requires the local normal approximation to be valid; it is not implied merely by the algebraic tilting identity. \(\square\)

For an integer-valued sum, a density is not the target. The corresponding leading lattice approximation to a point probability with span 1 is \(P(S=k)\approx \exp\{K(\hat t)-\hat t k\}/\sqrt{2\pi K''(\hat t)}\). For an upper tail well above the mean, summing nearby lattice masses gives a useful geometric-tail version. In the following formula, \(t>0\) solves \(K'(t)=k\); the approximation is most useful when \(t\) is bounded away from zero and the tilted local approximation is accurate.

Indeed, exponential tilting gives the exact identity $$ P(S\geq k)=\exp\{K(t)-tk\}\sum_{j=0}^{\infty}e^{-tj}P_t(S=k+j). $$ When \(t>0\), the weights \(e^{-tj}\) decrease geometrically. For the terms that materially contribute, approximate the tilted point probabilities by their value at the tilted mean \(k\), namely \(1/\sqrt{2\pi K''(t)}\). Summing \(\sum_{j\geq0}e^{-tj}=1/(1-e^{-t})\) gives the formula. Near the mean, \(t\) approaches zero and this geometric factor becomes unstable; a near-mean tail method such as the Lugannani–Rice approximation is more appropriate.

Worked Examples

Common Pitfall and Practical Interpretation

A common mistake is to solve the saddlepoint equation correctly and then treat the resulting approximation as an exact probability, or to use the upper-tail geometric factor when the target is near the mean. The tilt equation only identifies a distribution under which the observed value is the mean. The local Gaussian approximation and, for tails, the geometric summation are additional approximations with their own conditions.

Another important check is the domain: a proposed saddlepoint must keep \(\eta+t\) inside the natural-parameter space. For a Poisson sum, every finite real \(t\) is allowed. For the exponential sum above, \(t<1\) is required. The root, the domain, and the sign of \(t\) should all be checked before interpreting a tail approximation.

Exponential-family structure matters because the same log-partition function supplies the cumulants, the tilted parameter, and the curvature term \(K''(t)\). The exponent \(K(t)-ts\) captures the rarity of the observed sum, while the square-root term describes local spread under the tilt. This is why the method can remain informative in tails where a central normal approximation may be poor.