Moments And Cumulants From Moment Generating Functions

From a Transform to Numerical Summaries

A moment generating function is useful not only because it encodes a distribution, but also because its derivatives at zero give numerical summaries of that distribution. The logarithm of the MGF gives a second set of summaries, the cumulants. The first two cumulants are the mean and variance; the third measures the signed asymmetry captured by the third central moment. These formulas are simple to state, but their validity depends on the MGF being finite in an interval around zero.

Let \(X\) be a real-valued random variable on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Its moment generating function is \(M_X(t)=\mathbb{E}[e^{tX}]\) at real \(t\) for which this expectation is finite. We assume throughout that there is some \(a>0\) such that \(M_X(t)<\infty\) for every \(t\in(-a,a)\). This local finiteness condition is stronger than merely having a few finite moments; it ensures that differentiation under the expectation is justified.

For real \(t\), \(e^{tX}>0\) almost surely, so whenever \(M_X(t)\) is finite it is positive. Thus its logarithm is defined. The assumption of finiteness on an open interval around zero also implies that all absolute moments exist: for any fixed integer \(r\ge0\), the function \(|x|^r\) is bounded by a constant times \(e^{a|x|}\), and \(\mathbb{E}[e^{a|X|}]\) is finite if the endpoints \(a\) and \(-a\) are chosen inside the interval of finiteness.

Derivatives of the MGF

Proof. Choose numbers \(0<c<a\) so that \([-a,a]\) lies inside the interval on which the MGF is finite. Then \(\mathbb{E}[e^{aX}]+\mathbb{E}[e^{-aX}]<\infty\), and \(e^{a|X|}\le e^{aX}+e^{-aX}\), so \(\mathbb{E}[e^{a|X|}]<\infty\). For each fixed nonnegative integer \(r\), there is a finite constant \(C_r\) such that \(|x|^r e^{c|x|}\le C_r e^{a|x|}\) for all real \(x\): this follows because \(|x|^r e^{-(a-c)|x|}\) is bounded. Thus \(|X|^r e^{c|X|}\) is integrable. Differentiating \(e^{tX}\) \(r\) times with respect to \(t\) gives \(X^r e^{tX}\). On any smaller interval around zero, these derivatives are dominated by an integrable bound of the form just established. The dominated convergence theorem therefore permits differentiation under the expectation, giving \(M_X^{(r)}(t)=\mathbb{E}[X^r e^{tX}]\) there. At \(t=0\), this becomes \(M_X^{(r)}(0)=\mathbb{E}[X^r]\). \(\square\)

In particular, \(M_X(0)=1\), \(M_X'(0)=\mathbb{E}[X]\), and \(M_X''(0)=\mathbb{E}[X^2]\). The second derivative is the second raw moment, not the variance. To obtain the variance, subtract the square of the mean.

Cumulants from the Logarithm

Proof. Since \(K_X(t)=\log M_X(t)\) and \(M_X(t)>0\), ordinary differentiation gives $$ K_X'(t)=\frac{M_X'(t)}{M_X(t)},\qquad K_X''(t)=\frac{M_X''(t)}{M_X(t)}-\left(\frac{M_X'(t)}{M_X(t)}\right)^2. $$ Differentiating the second expression gives $$ K_X'''(t)=\frac{M_X'''(t)}{M_X(t)} -3\frac{M_X''(t)M_X'(t)}{M_X(t)^2} +2\left(\frac{M_X'(t)}{M_X(t)}\right)^3. $$ At zero, \(M_X(0)=1\), and the moment theorem gives \(M_X^{(r)}(0)=m_r\). Hence \(\kappa_1=m_1\), \(\kappa_2=m_2-m_1^2\), and \(\kappa_3=m_3-3m_1m_2+2m_1^3\). By the definition of variance, \(m_2-m_1^2=\operatorname{Var}(X)\). To identify the third expression, expand the centered cube and take expectations: $$ \mathbb{E}[(X-m_1)^3] =\mathbb{E}[X^3-3m_1X^2+3m_1^2X-m_1^3] =m_3-3m_1m_2+3m_1^3-m_1^3. $$ This equals \(m_3-3m_1m_2+2m_1^3=\kappa_3\), as required. \(\square\)

This derivation also explains the distinction between raw moments and cumulants. The derivatives of \(M_X\) give raw moments directly; the derivatives of \(\log M_X\) adjust for lower-order contributions. For the third cumulant, those corrections are \(-3m_1m_2+2m_1^3\). In particular, \(\kappa_3\) is not generally the raw third moment \(m_3\).

Worked Calculations

Why the Logarithm Matters—and a Common Error

The cumulant generating function separates the contribution of the mean from higher-order variation. Its first derivative at zero gives the mean, its second derivative gives variance, and its third derivative gives the third central moment. This makes cumulants useful when a problem asks for variability or asymmetry rather than just raw powers. A zero third cumulant means the third central moment is zero; it does not, by itself, imply that the entire distribution is symmetric.

Common mistake: treating the third raw moment as the third cumulant. In the last example, one might incorrectly write \(\kappa_3=m_3=\tfrac{29}{6}\). The correction is to differentiate \(\log M_X(t)\), or use the proved formula: $$ \kappa_3=m_3-3m_1m_2+2m_1^3 =\tfrac{29}{6}-\tfrac{55}{12}+\tfrac{125}{108} =\tfrac{38}{27}. $$ The direct centered-cube calculation also gives \(\tfrac{38}{27}\), so the raw third moment \(\tfrac{29}{6}\) cannot be substituted for the third cumulant here. A related check is that \(\kappa_2\) is \(m_2-m_1^2\), not \(m_2\); only when the mean is zero do the second raw moment and variance agree.

The local finiteness assumption is important: it justifies differentiating the expectation and guarantees the moments used in these identities exist. If an MGF is not finite on any open interval around zero, these derivative formulas cannot be invoked on the basis of an MGF in the stated way. One should check the MGF's domain before differentiating, rather than treating formal derivatives as expectations without a finiteness condition.