Why the Multivariate Saddlepoint Is a Vector
For a vector-valued quantity, a density approximation must account for joint behavior: not only the curvature in each coordinate but also the dependence between coordinates. The multivariate saddlepoint method does this through the Hessian matrix of the cumulant-generating function. Its determinant supplies the joint curvature factor, while the saddlepoint itself adjusts the approximation to the target point.
Let \(X\) be a random vector in \(\mathbb{R}^d\), and write \(t^\top X\) for the usual inner product. When the expectation is finite, its cumulant-generating function (CGF) is \(K(t)=\log E[\exp(t^\top X)]\). For a sum of independent random vectors, the CGF of the sum is the sum of the individual CGFs. The construction below applies to that sum directly.
The gradient equation matches the mean under an exponential tilt to the target. The Hessian is not just a collection of coordinatewise variances: its off-diagonal entries record tilted covariances. Consequently, the determinant in the density formula is the determinant of the full Hessian.
Where the Equations Come From
Assume \(S\) has a density \(f\). For any \(t\) where \(K(t)\) is finite, define the exponentially tilted density $$ f_t(s)=\exp\{t^\top s-K(t)\}f(s). $$ This integrates to one by the definition of \(K\). Differentiating the CGF, when the required moments exist, gives $$ \nabla K(t)=E_t[S], \qquad \nabla^2K(t)=\operatorname{Cov}_t(S), $$ where \(E_t\) and \(\operatorname{Cov}_t\) refer to the tilted distribution. Thus the saddlepoint equation selects the tilt under which the target \(x\) is the mean. If the Hessian is positive definite throughout a convex domain, this equation has at most one solution there: for distinct \(t_1,t_2\), the inner product of \(t_1-t_2\) with \(\nabla K(t_1)-\nabla K(t_2)\) is the integral of a strictly positive quadratic form along the segment between them. Existence is not automatic; the target must be in the image of the gradient on the CGF's domain.
The stated local limit condition holds under suitable regularity conditions; it is important because a central limit theorem for probabilities alone does not establish a density approximation at a point. Under the tilt, the exact relationship between the original and tilted densities is $$ f_{S_n}(s)=\exp\{n\kappa(t)-t^\top s\}f_{S_n,t}(s). $$ At \(s=ny\), substitute the local limit approximation into this identity. The exponential factor becomes \(\exp\{n\kappa(t)-nt^\top y\}\), and the local limit supplies the determinant and power of \(2\pi n\). This proves the stated leading approximation. \(\square\)
Equivalently, for a general sum with CGF \(K\), the tilted mean and covariance are \(\nabla K(t)\) and \(\nabla^2K(t)\). Evaluating the tilted density near its mean by a multivariate normal local approximation yields the formula in the definition. The approximation is asymptotic, not an identity in general.
Worked Examples
Worked Example: A Bivariate Normal Sum
Let \(S\sim N(m,\Sigma)\), where \(m=(1,-1)^\top\) and $$ \Sigma=\begin{pmatrix}2&1\\1&2\end{pmatrix}. $$ Approximate its density at \(x=(2,1)^\top\).
The CGF is \(K(t)=t^\top m+\tfrac12t^\top\Sigma t\). Hence \(\nabla K(t)=m+\Sigma t\). The saddlepoint equation gives \(\Sigma\hat t=x-m=(1,2)^\top\). Since $$ \Sigma^{-1}=\frac13\begin{pmatrix}2&-1\\-1&2\end{pmatrix}, \qquad \hat t=\Sigma^{-1}(1,2)^\top=(0,1)^\top, $$ the saddlepoint is \((0,1)^\top\). The Hessian is \(\Sigma\), whose determinant is \(2\cdot2-1\cdot1=3\). Also, \(K(\hat t)-\hat t^\top x=-1\). Therefore $$ \hat f_S(x)=\frac{e^{-1}}{2\pi\sqrt{3}}. $$ For a multivariate normal distribution this formula is exact: the saddlepoint exponent is \(-\tfrac12(x-m)^\top\Sigma^{-1}(x-m)\), and the Hessian is the covariance matrix.
Worked Example: A Bivariate Sum of Exponentials
For each \(i=1,\ldots,10\), let \(Y_i=(U_i,V_i)^\top\), where all \(U_i\) and \(V_i\) are independent exponential random variables with rate 1. Set \(S=\sum_{i=1}^{10}Y_i\), and approximate its density at \(x=(12,8)^\top\).
For \(t_1,t_2<1\), the CGF is $$ K(t_1,t_2)=-10\log(1-t_1)-10\log(1-t_2). $$ Its gradient is \((10/(1-t_1),10/(1-t_2))^\top\). Setting this equal to \((12,8)^\top\) gives $$ \hat t_1=1-\frac{10}{12}=\frac16, \qquad \hat t_2=1-\frac{10}{8}=-\frac14. $$ The Hessian is diagonal, with entries \(10/(1-t_1)^2\) and \(10/(1-t_2)^2\). At the saddlepoint these are \(14.4\) and \(6.4\), so \(\det H=14.4(6.4)=92.16\) and \(\sqrt{\det H}=9.6\). Further, $$ K(\hat t)=10\log(12/10)+10\log(8/10)=10\log(0.96), \qquad \hat t^\top x=2-2=0. $$ Thus $$ \hat f_S(12,8)=\frac{0.96^{10}}{2\pi(9.6)}\approx 0.01102. $$ The exact density is the product of two Gamma\((10,1)\) densities: $$ f_S(12,8)=\frac{12^9 8^9e^{-20}}{(9!)^2}\approx 0.01084. $$ The values are close here; the calculation also shows why the determinant uses both coordinate curvatures.
Worked Example: A Sum and Its Sample Mean
Suppose \(S_n\) is a sum of \(n\) independent vectors with CGF \(\kappa\) per vector, and consider the sample mean \(\bar Y=S_n/n\) at \(y\). The saddlepoint solves \(\nabla\kappa(t)=y\). The corresponding approximation for the density of the sum at \(ny\) is $$ \hat f_{S_n}(ny)= \frac{\exp\{n[\kappa(t)-t^\top y]\}} {(2\pi n)^{d/2}\{\det\nabla^2\kappa(t)\}^{1/2}}. $$ Since the transformation from \(S_n\) to \(\bar Y\) has Jacobian \(n^d\), its density approximation is $$ \hat f_{\bar Y}(y)=n^d\hat f_{S_n}(ny) = \frac{n^{d/2}\exp\{n[\kappa(t)-t^\top y]\}} {(2\pi)^{d/2}\{\det\nabla^2\kappa(t)\}^{1/2}}. $$ The exponent and saddlepoint are unchanged, but the density scale is not. In dimension two, the factor converting the sum density to the mean density is \(n^2\).
Interpretation and Common Pitfalls
The exponent \(K(\hat t)-\hat t^\top x\) measures the exponential change associated with tilting the distribution so its mean reaches \(x\). The Hessian determinant adjusts for the local volume around that point under the tilted distribution. When coordinates are dependent, the off-diagonal Hessian entries affect this volume through the determinant.
- Replacing the Hessian determinant by a product of marginal variances. This discards dependence. Use \(\det H(\hat t)\), including all off-diagonal entries; only a diagonal Hessian reduces to a product of diagonal terms.
- Using the mean equation at the wrong scale. For a sum \(S_n\) at \(ny\), solve \(\nabla\kappa(t)=y\), or equivalently \(\nabla K_n(t)=ny\). Do not set the per-observation gradient equal to \(ny\).
- Treating the approximation as exact. Apart from special cases such as the normal example, the formula is a leading approximation whose accuracy depends on regularity and sample size.
- Assuming every target has a saddlepoint. A target outside the gradient image, or at a boundary where the Hessian degenerates, may have no valid interior saddlepoint.
The multivariate method is useful when a joint density is needed in several dimensions and ordinary normal approximations do not capture the target's location well. The gradient selects the relevant exponential tilt, and the Hessian determinant retains the local joint geometry.
Check Your Understanding
Answer each question by referring to the CGF and the target vector.
- For a \(d\)-dimensional random vector, what equation defines the saddlepoint for a density approximation at \(x\)?
- Why does the multivariate density formula involve the determinant of the full Hessian rather than only its diagonal entries?
- In the exponential example, verify that both saddlepoint coordinates lie in the CGF's domain.
- What local approximation condition is used to turn the exact exponential-tilting identity into a leading density approximation?
- How does the density approximation change when the target is for a sample mean rather than its corresponding sum?