Exponential Tilting Derivation of the Saddlepoint Density

Why Exponential Tilting Helps

A density can be very small at a point far from the mean. A direct normal approximation may then be poor: the central limit theorem describes behavior near the mean, while the target lies in a tail. Exponential tilting changes the probability law so that the target becomes the mean. Under that tilted law, the density at the target can be approximated locally by a normal density; an exact change-of-measure identity then converts the result back to the original law.

This derivation produces the leading saddlepoint density approximation. It applies to densities, under suitable local regularity conditions. It is not a formula for a lattice point probability or a tail probability.

Exponential Tilting and the Saddlepoint

Let $X_1,\ldots,X_n$ be independent and identically distributed real-valued random variables. Assume their cumulant generating function $K(t)=\log E[e^{tX_1}]$ is finite on an open interval containing $0$. At any $t$ in the interior of this interval, the exponentially tilted distribution is defined by weighting the original law by $e^{tX_1}$, then normalizing.

Definition: The tilted law $P_t$ is given, for every measurable set $A$, by $$ P_t(X_1\in A)=E\!\left[\mathbf{1}_{\{X_1\in A\}}e^{tX_1-K(t)}\right]. $$ When $K$ is twice differentiable, its mean and variance under this law are $K'(t)$ and $K''(t)$, respectively. For a target value $x$, an interior solution $t=t(x)$ of $$ K'(t)=x $$ is called the saddlepoint associated with $x$. The corresponding rate value is $I(x)=t(x)x-K(t(x))$.

The moment generating function is finite in a neighborhood of any such interior $t$, so differentiation under the expectation gives $K'(t)=E_t[X_1]$ and $K''(t)=\operatorname{Var}_t(X_1)$. Here $E_t$ denotes expectation under $P_t$. If $X_1$ is nondegenerate, this variance is positive; consequently $K'$ is strictly increasing on the interval, and there is at most one saddlepoint for a given $x$.

Worked Example: Normal Observations

Let $X_i\sim N(0,1)$, and consider the density of the sample mean $\bar X_n$ at $x=1.5$. The cumulant generating function is $K(t)=t^2/2$, so $K'(t)=t$. The saddlepoint is therefore $t=1.5$.

At this value, $K''(t)=1$ and $I(x)=tx-K(t)=1.5(1.5)-(1.5)^2/2=1.125$. The leading approximation for the sample-mean density is $$ \widehat f_{\bar X_n}(1.5) =\sqrt{\frac{n}{2\pi}}\exp(-1.125n). $$ In this case the approximation is exact: $\bar X_n\sim N(0,1/n)$, whose density at $1.5$ is precisely the displayed expression. The example checks both the exponential term and the sample-mean prefactor.

The Exact Change of Measure

Write $S_n=\sum_{i=1}^n X_i$. Independence makes the tilted law of the full sample a product law. Its likelihood ratio relative to the original sample law is $e^{tS_n-nK(t)}$. This identity gives an exact relationship between the two densities, not an approximation.

Theorem: Suppose $S_n$ has a density $f_{S_n}$ under the original law and a density $f_{t,S_n}$ under the tilted product law. For any interior $t$, versions of these densities can be chosen so that, almost everywhere in $s$, $$ f_{S_n}(s)=e^{-ts+nK(t)}f_{t,S_n}(s). $$

Proof. For any measurable set $B$ of sample vectors, the tilted product law satisfies $$ P_t((X_1,\ldots,X_n)\in B) =E\!\left[\mathbf{1}_{\{(X_1,\ldots,X_n)\in B\}}e^{tS_n-nK(t)}\right]. $$ Equivalently, the original product law has Radon–Nikodym derivative $e^{-tS_n+nK(t)}$ with respect to the tilted product law. Apply this change of measure to the event $S_n\in A$. The resulting density of $S_n$ under the original law is the tilted density multiplied at $s$ by $e^{-ts+nK(t)}$, giving the stated equality almost everywhere. $\square$

At the saddlepoint $t=t(x)$, the tilted mean of each observation is $x$, so the tilted mean of $S_n$ is $nx$. The density value needed in the exact identity is therefore at the center of the tilted sum. This is the key change: a potentially remote density evaluation under the original law becomes a central one under the tilted law.

From a Local Normal Approximation to the Density Formula

A central limit theorem alone gives convergence in distribution, not convergence of densities at a point. To justify the step used here, impose a local limit condition on the tilted law. One convenient sufficient set of assumptions at the chosen $t$ is the following: under $P_t$, $X_1$ has a finite, positive variance $v=K''(t)$; the characteristic function of $X_1-x$ is integrable; and its absolute value is strictly less than one at every nonzero argument, with supremum strictly less than one outside every neighborhood of zero. These conditions give a density local limit at the center of the tilted sum.

Local density limit: Under the stated conditions, if $g_n$ is the density of $S_n-nx$ under the tilted product law, then $$ \sqrt{n}\,g_n(0)\longrightarrow \frac{1}{\sqrt{2\pi v}}. $$ Since $g_n(0)=f_{t,S_n}(nx)$, this is the local normal approximation needed for the derivation.

For completeness, the Fourier argument at the center shows why the local conclusion follows from these conditions. Let $\psi(u)=E_t[e^{iu(X_1-x)}]$. It has $\psi(0)=1$, mean zero, and variance $v$, so $\psi(u)=1-vu^2/2+o(u^2)$ as $u\to0$. Fourier inversion for the sum, followed by the change of variable $z=u\sqrt n$, gives $$ \sqrt n\,g_n(0)=\frac{1}{2\pi}\int_{\mathbb R}\psi(z/\sqrt n)^n\,dz. $$ For each fixed $z$, the integrand tends to $e^{-vz^2/2}$. Near zero, the finite positive variance implies that for some $c,\delta>0$, $|\psi(u)|\le e^{-cu^2}$ whenever $|u|\le\delta$. Thus, on $|z|\le\delta\sqrt n$, the integrands are bounded in absolute value by $e^{-cz^2}$, an integrable function. Dominated convergence on this expanding region, with its indicator included, gives the limiting integral there. On the complementary region, integrability of $\psi$ and the assumed bound $\sup_{|u|\ge\delta}|\psi(u)|=\rho<1$ give $$ \sqrt n\int_{|u|>\delta}|\psi(u)|^n\,du \le \sqrt n\,\rho^{n-1}\int_{\mathbb R}|\psi(u)|\,du \longrightarrow 0. $$ The remaining Gaussian integral is $\int_{\mathbb R}e^{-vz^2/2}\,dz=\sqrt{2\pi/v}$. Substitution yields the local density limit. $\square$

Now take $s=nx$ and $t=t(x)$ in the exact density identity. Using $I(x)=tx-K(t)$ and the local density limit gives the leading approximation for the sum:

Theorem: Under the saddlepoint and local density conditions above, $$ f_{S_n}(nx)\ \approx\ \frac{\exp[-nI(x)]}{\sqrt{2\pi nK''(t)}}. $$ Equivalently, since the density transformation from $S_n$ to $\bar X_n=S_n/n$ contributes a factor $n$, $$ f_{\bar X_n}(x)\ \approx\ \sqrt{\frac{n}{2\pi K''(t)}}\,\exp[-nI(x)]. $$

The exponential factor records the cost of moving from the original law to the tilted one; the square-root factor is the tilted density’s local normal scale. The approximation is asymptotic as $n$ grows with a fixed interior target $x$. It is not, by itself, a guarantee of high accuracy for every finite sample size.

Worked Example: Exponential Observations

Let the $X_i$ have the exponential distribution with rate $1$, and evaluate the sample-mean density at $x=1.5$. For $t<1$, $K(t)=-\log(1-t)$, so $K'(t)=1/(1-t)$. Solving $K'(t)=1.5$ gives $t=1/3$.

The tilted variance is $K''(t)=1/(1-t)^2=9/4$. The rate value is $$ I(1.5)=\frac13(1.5)+\log(1-\tfrac13) =\frac12+\log(\tfrac23) \approx 0.094535. $$ Thus, for $n=10$, the leading saddlepoint approximation is $$ f_{\bar X_{10}}(1.5)\ \approx\ \sqrt{\frac{10}{(9/2)\pi}}\, \exp\!\left[-10\left\{\frac12+\log(\tfrac23)\right\}\right]. $$ The saddlepoint is inside the domain $t<1$, and the tilted variance is positive. The formula uses the variance under the tilted law, not the original exponential variance.

As a check on what is being approximated, the exact sample-mean density is $10^{10}(1.5)^9e^{-15}/9!$. The saddlepoint expression is its leading large-$n$ form, not an identity for $n=10$.

Worked Example: A Two-Normal Mixture

Suppose each $X_i$ has the mixture distribution that assigns probability $1/2$ to $N(-1,1)$ and probability $1/2$ to $N(1,1)$. Its moment generating function is $e^{t^2/2}\cosh(t)$, hence $K(t)=t^2/2+\log\cosh(t)$.

Choose $t=\log 3$. Since $\tanh(\log 3)=4/5$, the associated target is $x=K'(t)=\log 3+4/5$. Also, $\cosh(\log 3)=5/3$ and $K''(t)=1+\operatorname{sech}^2(t)=1+9/25=34/25$. Therefore $$ I(x)=tx-K(t) =\frac12(\log 3)^2+\frac45\log 3-\log(\tfrac53). $$ For $n=5$, the leading sample-mean density approximation at this target is $$ f_{\bar X_5}(x)\ \approx\ \sqrt{\frac{5}{2\pi(34/25)}}\, \exp\!\left[-5\left\{\frac12(\log 3)^2+\frac45\log 3-\log(\tfrac53)\right\}\right]. $$ This example illustrates the general calculation when the cumulant generating function is not purely quadratic. The tilt makes the target the mean, while its curvature $K''(t)$ determines the local spread.

Common Pitfalls and Scope

Using the original variance in the prefactor. The local normal approximation is under $P_t$, so its variance is $K''(t)$, not generally $K''(0)$.
Confusing the sum and sample-mean densities. The density of $\bar X_n$ includes an additional factor $n$ relative to the density of $S_n$ evaluated at $nx$.
Replacing a local limit with an ordinary central limit theorem. Convergence in distribution does not, by itself, establish convergence of density values at a point. A density local limit or conditions that imply one are required.
Applying the density formula to lattice observations. A discrete distribution has point probabilities rather than a density, so the local density derivation does not apply. Lattice point probabilities require a different approximation.
Using a nonexistent saddlepoint. The target must correspond to an interior solution of $K'(t)=x$, with a finite, positive tilted variance for this derivation.

The importance of tilting is that it separates a rare-event exponent from a local fluctuation scale. That structure is useful for density approximation away from the mean, while the assumptions identify the boundary of the argument: the local normal step concerns a continuous density at the tilted center, not a discrete mass or an integrated tail.

Check Your Understanding

Use the definitions and derivation above to answer the following questions.

For a target $x$, what equation determines the saddlepoint, and what is the mean of one observation under the resulting tilted law?
Explain why the change-of-measure identity is exact even though the final saddlepoint density formula is an approximation.
For the exponential model in the worked example, verify that $t=1/3$ gives tilted mean $1.5$ and tilted variance $9/4$.
Why does the approximation for the density of $\bar X_n$ have a factor $n$ relative to the density of $S_n$ at $nx$?
Why is an ordinary central limit theorem alone insufficient to justify replacing the tilted density at its mean by a normal density value?