Why MGF Coefficients Encode Moments
An MGF packages the raw moments of a random variable into a function of a real parameter. When the MGF is finite on an open interval around zero, its Taylor series does more than formally resemble a moment series: it actually converges to the MGF there, and its coefficients are the raw moments divided by factorials. The condition near zero matters. Having moments of every order, by itself, does not ensure that the moment series converges or represents an MGF.
The key issue is whether the exponential series can be averaged term by term. We will establish a condition that guarantees this exchange, work through examples, and then see explicitly what can go wrong when only the moments are known to exist.
The Taylor Expansion Theorem
The exact condition needed here is finiteness of the MGF on some open interval containing zero. Equivalently, there is an \(a>0\) such that \(M_X(t)\) is finite for every \(t\) with \(|t|<a\). The proof uses finiteness at both positive and negative arguments to control the absolute value of \(X\).
Proof. First, for any \(c\) with \(0<c<a\), both \(M_X(c)\) and \(M_X(-c)\) are finite. For every real \(x\), \(e^{c|x|}\le e^{cx}+e^{-cx}\): when \(x\ge0\), the first term on the right is \(e^{c|x|}\), and when \(x<0\), the second term is. Hence $$ E[e^{c|X|}]\le M_X(c)+M_X(-c)<\infty. $$ For each integer \(k\ge0\) and \(y\ge0\), the exponential series gives \(e^{cy}\ge (cy)^k/k!\). Therefore $$ |X|^k\le \frac{k!}{c^k}e^{c|X|}, $$ so \(E[|X|^k]<\infty\). This proves that all the raw moments in the claimed series exist.
Now fix \(t\) with \(|t|<a\), and choose \(c\) such that \(|t|<c<a\). Pointwise, the exponential series is absolutely convergent, and $$ \sum_{k=0}^{\infty}\left|\frac{(tX)^k}{k!}\right|=e^{|t||X|}\le e^{c|X|}. $$ The upper bound has finite expectation by the preceding argument. By Tonelli’s theorem applied to the nonnegative series, $$ \sum_{k=0}^{\infty}E\left[\left|\frac{(tX)^k}{k!}\right|\right] =E[e^{|t||X|}]<\infty. $$ Thus the series can be integrated term by term, giving $$ E[e^{tX}] =E\left[\sum_{k=0}^{\infty}\frac{(tX)^k}{k!}\right] =\sum_{k=0}^{\infty}\frac{E[X^k]}{k!}t^k. $$ This establishes the expansion throughout \((-a,a)\). A power series can be differentiated term by term inside its interval of convergence; differentiating \(k\) times and evaluating at zero identifies its \(k\)th derivative there as \(E[X^k]\). \(\square\)
The choice of \(c\) in the proof is useful: it is strictly larger than \(|t|\), so the absolute exponential \(e^{|t||X|}\) is bounded by an exponential already known to be integrable. This is what justifies the exchange, rather than merely the fact that the pointwise Taylor series exists.
Worked Example: Recovering Bernoulli Moments From Its MGF
Let \(X\) be Bernoulli with \(P(X=1)=p\) and \(P(X=0)=1-p\), where \(0\le p\le1\). Its MGF is finite for every real \(t\): $$ M_X(t)=E[e^{tX}]=(1-p)e^0+pe^t=1-p+pe^t. $$ Using the exponential series, $$ M_X(t)=1-p+p\sum_{k=0}^{\infty}\frac{t^k}{k!} =1+p\sum_{k=1}^{\infty}\frac{t^k}{k!}. $$ The constant coefficient is \(1=E[X^0]\). For every \(k\ge1\), \(X^k=X\), since \(X\) is either zero or one, so \(E[X^k]=p\). The coefficient of \(t^k\) is therefore \(p/k!=E[X^k]/k!\), as claimed. For example, \(M_X'(0)=p\) and \(M_X''(0)=p\), matching \(E[X]=E[X^2]=p\).
Worked Example: Reading Moments From a Uniform MGF
Let \(X\) have the uniform distribution on \([-1,1]\), with density \(f(x)=1/2\) on that interval. Since \(X\) is bounded, \(e^{tX}\le e^{|t|}\), so its MGF is finite for every real \(t\). Direct integration gives, for \(t\ne0\), $$ M_X(t)=\frac12\int_{-1}^{1}e^{tx}\,dx =\frac{e^t-e^{-t}}{2t}. $$ At \(t=0\), the integral gives \(M_X(0)=1\).
Expand \(e^{tx}\) and integrate term by term. This is justified, for fixed \(t\), by the bound \(\sum_{k=0}^{\infty}|(tx)^k/k!|=e^{|tx|}\le e^{|t|}\) on \([-1,1]\). Odd powers integrate to zero by symmetry. For \(j\ge0\), $$ E[X^{2j}]=\frac12\int_{-1}^{1}x^{2j}\,dx =\frac{1}{2j+1}. $$ Consequently, $$ M_X(t)=\sum_{j=0}^{\infty}\frac{E[X^{2j}]}{(2j)!}t^{2j} =\sum_{j=0}^{\infty}\frac{t^{2j}}{(2j+1)!}. $$ For instance, the \(t^2\) coefficient is \(1/3!=1/6\), so \(E[X^2]=2!(1/6)=1/3\). The \(t^4\) coefficient is \(1/5!=1/120\), so \(E[X^4]=4!(1/120)=1/5\). The zero coefficients on odd powers correspond to the zero odd raw moments.
Why a Formal Moment Series May Fail
The theorem gives a sufficient condition for a genuine Taylor expansion: an MGF finite on a neighborhood of zero. The implication should not be reversed casually. Moments may all exist even when the MGF is not finite on any such neighborhood. In that case, the moment series can have no nonzero interval of convergence.
Worked Example: All Moments Exist, but the Moment Series Diverges
Consider a random variable \(X\) with density $$ f(x)=\frac12 e^{-\sqrt{x}},\qquad x>0. $$ To check that this is a probability density, substitute \(u=\sqrt{x}\), so \(x=u^2\) and \(dx=2u\,du\): $$ \int_0^\infty \frac12 e^{-\sqrt{x}}\,dx =\int_0^\infty u e^{-u}\,du=1. $$ For every integer \(k\ge0\), the same substitution gives $$ E[X^k]=\int_0^\infty x^k\frac12 e^{-\sqrt{x}}\,dx =\int_0^\infty u^{2k+1}e^{-u}\,du =(2k+1)!. $$ The last equality follows from repeated integration by parts for the factorial integral. Thus every raw moment is finite.
For \(t>0\), however, $$ M_X(t)=\int_0^\infty u e^{tu^2-u}\,du=\infty, $$ because \(tu^2-u\) grows without bound quadratically, making the integrand nonintegrable in the tail. So the MGF is not finite on any open interval containing zero.
The formal moment series has terms \((2k+1)!t^k/k!\). For fixed \(t\ne0\), the ratio of successive term magnitudes is $$ \frac{(2k+3)!|t|^{k+1}/(k+1)!}{(2k+1)!|t|^k/k!} =2(2k+3)|t|\longrightarrow\infty. $$ In particular, the terms do not tend to zero, so the series diverges for every \(t\ne0\). All the moments exist, but they do not form a convergent MGF Taylor expansion.
A Reliable Way to Use MGF Series
When using an MGF expansion to recover moments, distinguish three statements: the MGF is finite near zero; the formal series made from moments converges; and the series equals the MGF. The theorem establishes all three together under the neighborhood-finiteness condition. Moment existence alone establishes only that individual coefficients can be written down.
Verify that \(M_X(t)\) is finite on an open interval containing zero. Finiteness only at zero, or only on one side of zero, is not enough for the theorem.
Under that condition, the coefficient of \(t^k\) in the Taylor series is \(E[X^k]/k!\). Multiply the coefficient by \(k!\) to recover the raw moment.
If the domain condition has not been checked, a list of finite moments supplies candidate coefficients, not a justified expansion of the MGF.
A common mistake is to expand \(e^{tX}\) pointwise and immediately take expectations term by term, on the grounds that every moment is finite. Pointwise convergence does not itself justify exchanging an infinite sum and expectation. The correction is to establish an integrable bound for the absolute series, as in the theorem’s proof. Finiteness of the MGF on both sides of zero supplies that bound; without it, the formal series may diverge, as the last example demonstrates.
Check Your Understanding
Use the domain condition and coefficient rule to answer these questions.
- For a random variable whose MGF is finite on \((-b,b)\), what is the coefficient of \(t^4\) in its Taylor expansion at zero?
- Why does finiteness of \(M_X(c)\) and \(M_X(-c)\), for some \(c>0\), imply \(E[e^{c|X|}]<\infty\)?
- If the coefficient of \(t^3\) in a valid MGF expansion is \(-2/3\), what is \(E[X^3]\)?
- Why is the existence of every raw moment not, by itself, enough to justify expanding the MGF term by term?
- For the density in the last worked example, what is the ratio of successive moment-series term magnitudes, and what does its limit imply?