Why Bound an Exponential Expectation?
An expectation describes an average, but applying a nonlinear function before averaging can change the result. For an exponential transform, convexity determines the direction of that change: the average of \(e^{tX}\) cannot fall below the exponential of the average of \(tX\). This gives a general lower bound that can be evaluated using only the mean of \(X\), even when the full distribution is complicated.
The parameter \(t\) may be positive, negative, or zero. The sign matters in applications, but it does not change the direction of the inequality. The essential conditions are that \(X\) have a finite mean and that the exponential expectation be finite if a finite-valued bound is wanted.
The finite-mean condition ensures that \(E[X]\) is a real number, so the right side is well defined. The exponential expectation is nonnegative and may be treated as an extended expectation if it is not finite. In the examples below, it is finite.
Deriving the Bound and Its Equality Cases
The exponential function is strictly convex because its second derivative is \(e^u>0\) for every real \(u\). A convenient consequence of its tangent line at zero is the pointwise inequality \(e^u\ge 1+u\), with equality only at \(u=0\). To verify it directly, define \(g(u)=e^u-1-u\). Its derivative is \(g'(u)=e^u-1\), which is negative for \(u<0\) and positive for \(u>0\); hence \(g\) has its unique minimum, zero, at \(u=0\).
Proof. Since \(E[|X|]<\infty\), the centered variable \(X-\mu\) is integrable and \(E[X-\mu]=0\). Apply \(e^u\ge 1+u\) pointwise with \(u=t(X-\mu)\):
The right side is integrable, and its expectation is \(1+tE[X-\mu]=1\). Taking expectations of the pointwise inequality therefore gives \(E[e^{t(X-\mu)}]\ge 1\), including when the expectation on the left is infinite. Because \(e^{tX}=e^{t\mu}e^{t(X-\mu)}\), multiplying by the positive constant \(e^{t\mu}\) gives \(E[e^{tX}]\ge e^{t\mu}\).
For the equality statement, suppose first that \(t\ne0\) and the exponential expectation is finite. The gap \(e^{t(X-\mu)}-1-t(X-\mu)\) is nonnegative, and its expectation is zero exactly when the expectation bound is equality. A nonnegative random variable with expectation zero is zero almost surely. The pointwise gap is zero only when \(t(X-\mu)=0\), so equality requires \(X=\mu\) almost surely. Conversely, if \(X=\mu\) almost surely, both sides equal \(e^{t\mu}\). If \(t=0\), both sides equal one regardless of \(X\). \(\square\)
This centered form is often useful: the exponential of the centered variable has expectation at least one. It also makes clear why the result works for negative \(t\). The pointwise inequality \(e^u\ge1+u\) holds for every real \(u\), not only for nonnegative \(u\).
Worked Applications
Worked Example: A Normal Random Variable
Let \(X\sim N(1,4)\), so \(E[X]=1\) and \(\operatorname{Var}(X)=4\). Take \(t=-\tfrac12\). Find the exponential expectation and compare it with the Jensen bound.
For \(X\sim N(\mu,\sigma^2)\), the density has exponent \(-(x-\mu)^2/(2\sigma^2)\). Completing the square gives $$ tx-\frac{(x-\mu)^2}{2\sigma^2} = -\frac{(x-(\mu+\sigma^2t))^2}{2\sigma^2} +\mu t+\frac{\sigma^2t^2}{2}. $$ The shifted normal density integrates to one, so \(E[e^{tX}]=e^{\mu t+\sigma^2t^2/2}\). With \(\mu=1\), \(\sigma^2=4\), and \(t=-\tfrac12\), $$ E[e^{-X/2}] = \exp\left(-\frac12+\frac{4}{2}\cdot\frac14\right) = e^0=1. $$
The Jensen bound is $$ e^{tE[X]}=e^{-1/2}\approx 0.6065. $$ Thus \(1\ge 0.6065\), as required. The inequality is strict because the normal variable is not constant almost surely and \(t\ne0\). This calculation also shows that a negative value of \(t\) does not reverse the bound.
Worked Example: A Bernoulli Random Variable
Let \(X\sim\operatorname{Bernoulli}(0.3)\), so \(P(X=1)=0.3\), \(P(X=0)=0.7\), and \(E[X]=0.3\). Take \(t=\ln 2\). Calculate both sides.
Since \(X\) takes only the values zero and one, $$ E[e^{tX}] = 0.7e^0+0.3e^t = 0.7+0.3(2) = 1.3. $$ The Jensen bound is $$ e^{tE[X]} = e^{0.3\ln 2} = 2^{0.3} \approx 1.2311. $$ Hence \(1.3\ge 1.2311\). The inequality is strict: both Bernoulli outcomes have positive probability, so \(X\) is not almost surely equal to its mean \(0.3\).
The bound uses the mean alone; it does not reproduce the exact exponential expectation. Here the distribution’s two possible values allow that expectation to be calculated directly, and the calculation confirms that it exceeds the mean-based lower bound.
Worked Example: A Poisson Random Variable
Let \(X\sim\operatorname{Poisson}(2)\), so \(P(X=k)=e^{-2}2^k/k!\) for integers \(k\ge0\), and \(E[X]=2\). Take \(t=\ln(1.5)\). Compute the expectation from the probability mass function and compare it with the bound.
Using \(e^{tk}=(e^t)^k\) and the power series for the exponential, $$ \begin{aligned} E[e^{tX}] &=\sum_{k=0}^{\infty}e^{tk}e^{-2}\frac{2^k}{k!}\\ &=e^{-2}\sum_{k=0}^{\infty}\frac{(2e^t)^k}{k!}\\ &=e^{-2}e^{2e^t} =\exp\{2(e^t-1)\}. \end{aligned} $$ Since \(e^t=1.5\), this is \(\exp\{2(1.5-1)\}=e\approx2.7183\).
The Jensen bound is $$ e^{tE[X]} = e^{2\ln(1.5)} = (1.5)^2 = 2.25. $$ Thus \(e\approx2.7183\ge2.25\). Equality does not hold because a Poisson random variable with positive rate is not constant almost surely.
What the Bound Does—and Does Not—Say
The inequality supplies a lower bound using \(E[X]\) and \(t\). If the exact distribution is available, as in the examples, the exponential expectation may be calculated and compared with the bound. If only the mean is known, the result guarantees that the expectation cannot be smaller than \(e^{tE[X]}\), provided \(X\) is integrable.
The direction matters. This is not an upper bound on \(E[e^{tX}]\). For instance, if \(X\sim N(0,1)\) and \(t=1\), then the bound says \(E[e^X]\ge1\), while the exact expectation is \(e^{1/2}\). The inequality does not give a finite upper bound from the mean alone. In particular, using this lower bound as though it were an upper bound in a tail-probability calculation would reverse the logic.
The mean-based bound is exact when \(t=0\) or, for nonzero \(t\), when \(X\) is constant almost surely. For a nonconstant variable and a nonzero parameter, strict convexity makes the inequality strict. That distinction is useful when checking a proposed calculation: an equality claim for a nonconstant variable with \(t\ne0\) signals an error.
Common Mistakes
- Reversing the inequality. Convexity gives \(E[e^{tX}]\ge e^{tE[X]}\), not the reverse. The Bernoulli calculation above checks the direction numerically.
- Treating the statement as pointwise. In general, \(e^{tX}\ge e^{tE[X]}\) does not hold for every outcome. The theorem compares expectations, not each realized value with a fixed number.
- Restricting \(t\) to positive values. The result holds for all real \(t\). The centered pointwise inequality is valid whether \(t(X-E[X])\) is positive or negative.
- Using the mean without checking its existence. The stated finite-valued bound requires \(E[|X|]<\infty\), so that \(E[X]\) is well defined as a finite number.
- Assuming the bound is tight. Except when \(t=0\) or \(X\) is constant almost surely, the inequality is strict. The lower bound need not equal the actual exponential expectation.
Check Your Understanding
Use the exponential Jensen bound and its equality conditions to answer these questions.
- If \(E[X]=-2\) and \(t=1\), what lower bound does the inequality give for \(E[e^X]\)?
- Does the bound apply when \(t<0\)? Briefly explain why.
- For \(t\ne0\), what condition on \(X\) makes the inequality an equality?
- If \(X\sim\operatorname{Bernoulli}(0.5)\) and \(t=\ln 3\), calculate \(E[e^{tX}]\) and the Jensen lower bound.
- Does \(E[e^{tX}]\ge e^{tE[X]}\) provide an upper bound on \(E[e^{tX}]\)? Explain.