Cumulants as Derivatives of the Log MGF

From an MGF to Cumulants

An MGF packages moments into a function of $t$. Cumulants use a closely related function: the logarithm of the MGF. Taking the logarithm changes the derivatives, so cumulants are not simply the moments in another notation. Their definition is compact, but it depends on being able to differentiate the MGF near zero. This tutorial makes that condition precise and proves the differentiation rule that supports the definition.

Definition: Let $X$ be a real-valued random variable with MGF $M_X(t)=E[e^{tX}]$. If $M_X(t)$ is finite on an open interval containing zero, its cumulant-generating function is $$ K_X(t)=\log M_X(t) $$ on that interval. For each integer $r\geq 1$, the cumulant of order $r$ is $$ \kappa_r(X)=K_X^{(r)}(0). $$

The logarithm is well defined wherever the MGF is finite: $e^{tX}>0$, so $M_X(t)>0$. The relevant domain condition is finiteness on some open interval around zero, not just finiteness at zero. Since $M_X(0)=E[1]=1$ for every random variable, finiteness at that single point says nothing about whether derivatives can be taken there.

The definition asks for derivatives of $K_X$, so it matters that the MGF is smooth in a neighborhood of zero. The following result gives more: every derivative of the MGF is an exponentially weighted moment.

The MGF Derivative Rule

Theorem: Suppose there is an $a>0$ such that $M_X(t)=E[e^{tX}]<\infty$ for every $t\in(-a,a)$. Then, for every integer $r\geq 0$ and every $t\in(-a,a)$, $$ M_X^{(r)}(t)=E[X^r e^{tX}]. $$ In particular, all absolute moments $E[|X|^r]$ are finite, and $M_X$ has derivatives of every order throughout $(-a,a)$.

Proof. First, the neighborhood condition implies the required moments are integrable. Fix $b\in(0,a)$. For each integer $r\geq 0$, there is a finite constant $C$ such that $|x|^r\leq C e^{b|x|}$ for all real $x$. Therefore $$ E[|X|^r]\leq C E[e^{b|X|}] \leq C\bigl(E[e^{bX}]+E[e^{-bX}]\bigr)<\infty. $$ The last two expectations are finite by the hypothesis.

We prove the derivative formula by induction. The case $r=0$ is the definition of $M_X(t)$. Suppose that $M_X^{(r)}(t)=E[X^r e^{tX}]$ on the interval. Fix an interior point $t$. Choose $\delta>0$ small enough that $t-2\delta$ and $t+2\delta$ both belong to $(-a,a)$. For $|h|<\delta$, the difference quotient of $x^r e^{t x}$ as $t$ changes to $t+h$ is bounded in absolute value by $$ |x|^{r+1}e^{t x+\delta|x|}. $$ By increasing a finite constant if necessary, this is bounded by $C e^{(t+2\delta)x}$ when $x\geq 0$, and by $C e^{(t-2\delta)x}$ when $x<0$. These bounds are integrable because the MGF is finite at both endpoints. The dominated convergence theorem therefore allows the limit of the difference quotient to pass through the expectation, giving $$ \frac{d}{dt}E[X^r e^{tX}]=E[X^{r+1}e^{tX}]. $$ The same kind of local domination gives continuity of these derivatives. This proves the induction step at every interior point, and hence the formula for all $r$. $\square$

Since $M_X(t)>0$ and $M_X$ has derivatives of every order, $K_X(t)=\log M_X(t)$ also has derivatives of every order on the interval, by the ordinary chain and product rules. Thus each $\kappa_r(X)$ in the definition exists as a finite number. The theorem does not say that $K_X^{(r)}(t)=E[X^r e^{tX}]$: that expectation is the derivative of $M_X$, not of its logarithm.

Worked Examples

Worked Example: Applying the Derivative Rule to a Finite-Support Law

Let $X$ take the values $-1,0,2$ with probabilities $0.2,0.5,0.3$, respectively. Find its MGF and check the derivative rule at order five.

MGF. Since the distribution has finite support, the expectation is a finite sum: $$ M_X(t)=0.2e^{-t}+0.5+0.3e^{2t}. $$ This is finite for every real $t$, so in particular it is finite on an open interval around zero.

Differentiating five times gives $$ M_X^{(5)}(t)=0.2(-1)^5e^{-t}+0.3(2^5)e^{2t} =-0.2e^{-t}+9.6e^{2t}. $$ At zero, $M_X^{(5)}(0)=-0.2+9.6=9.4$. Directly from the distribution, $$ E[X^5]=0.2(-1)^5+0.5(0)^5+0.3(2)^5 =-0.2+0+9.6=9.4. $$ The two calculations agree, as the derivative rule requires. The cumulant of order five is instead $K_X^{(5)}(0)$, where $K_X(t)=\log(0.2e^{-t}+0.5+0.3e^{2t})$; it is a derivative of the log MGF, not the fifth MGF derivative just computed.

Worked Example: A Symmetry Check on an Odd-Order Cumulant

Let $X$ take values $-1$ and $1$, each with probability $1/2$. Use the definition to determine its cumulant of order five without expanding the log MGF.

The MGF is $$ M_X(t)=\tfrac12 e^{-t}+\tfrac12 e^t=\cosh t, $$ which is finite for all real $t$. Thus $K_X(t)=\log(\cosh t)$ exists near zero. Because $\cosh(-t)=\cosh(t)$, the function $K_X$ is even: $K_X(-t)=K_X(t)$.

Differentiating this identity five times gives $-K_X^{(5)}(-t)=K_X^{(5)}(t)$. Setting $t=0$ yields $-K_X^{(5)}(0)=K_X^{(5)}(0)$, so $K_X^{(5)}(0)=0$. By definition, $\kappa_5(X)=0$. The conclusion follows from the symmetry of the log MGF and does not require confusing its derivatives with those of the MGF.

Worked Example: A Sixth Cumulant from a Continuous MGF

Let $X$ have the Laplace density $f(x)=\tfrac12 e^{-|x|}$ for real $x$. Find its cumulant of order six.

For $|t|<1$, split the defining integral at zero: $$ M_X(t)=\tfrac12\int_0^\infty e^{-(1-t)x}\,dx +\tfrac12\int_{-\infty}^0 e^{(1+t)x}\,dx =\tfrac12\left(\frac{1}{1-t}+\frac{1}{1+t}\right) =\frac{1}{1-t^2}. $$ The integrals converge because $1-t>0$ and $1+t>0$, precisely when $|t|<1$. Therefore $$ K_X(t)=-\log(1-t^2). $$ For $|t|<1$, the power series $-\log(1-u)=\sum_{j=1}^{\infty}u^j/j$, applied with $u=t^2$, gives $$ K_X(t)=t^2+\frac{t^4}{2}+\frac{t^6}{3}+\frac{t^8}{4}+\cdots. $$ The coefficient of $t^6$ is $1/3$. Because $K_X$ is smooth near zero, its Taylor coefficient at degree six is $K_X^{(6)}(0)/6!$. Hence $$ \kappa_6(X)=K_X^{(6)}(0)=6!\left(\frac13\right)=\frac{720}{3}=240. $$

What the Definition Depends On

The neighborhood condition is what justifies the derivative rule. It supplies an integrable bound when a difference quotient is moved inside an expectation. Without an appropriate bound, the formal calculation “differentiate $e^{tX}$ and then take expectations” is not, by itself, a proof. In this course, finiteness of the MGF on an open interval around zero is a convenient sufficient condition that guarantees derivatives of every order.

This condition is sufficient, not necessary for every individual derivative. A particular derivative may exist under weaker assumptions. The definition used here, however, is based on an MGF that exists in a neighborhood of zero, so all the cumulants can be obtained from the same smooth cumulant-generating function.

Common Mistakes

Using finiteness at zero alone. Since $M_X(0)=1$ for every random variable, that fact cannot establish differentiability. Require finiteness on an open interval around zero.
Calling MGF derivatives cumulants. The theorem gives $M_X^{(r)}(t)=E[X^r e^{tX}]$. Cumulants are $K_X^{(r)}(0)$, with $K_X=\log M_X$.
Differentiating inside an expectation without justification. The exchange requires a valid domination argument (or another appropriate differentiation theorem). The open-interval hypothesis supplies the needed local integrable bound.
Assuming the cumulant-generating function is defined wherever the MGF is finite at one point. The definition requires a neighborhood of zero so that derivatives at zero are available.

Key takeaway: If the MGF is finite on an open interval containing zero, then it has derivatives of every order there, with $M_X^{(r)}(t)=E[X^r e^{tX}]$. Cumulants are defined by differentiating the log MGF at zero: $\kappa_r(X)=K_X^{(r)}(0)$.

Check Your Understanding

Try these questions using the definitions and conditions above.

What domain condition on $M_X(t)$ guarantees that the cumulants of every positive integer order are defined by derivatives at zero?
For $X$ taking values $-2$ and $1$ with probabilities $1/3$ and $2/3$, write $M_X(t)$ and $M_X^{(3)}(t)$.
Explain why $M_X^{(r)}(0)$ is not, in general, the cumulant of order $r$.
If $K_X$ is even and has five derivatives at zero, what can be concluded about $K_X^{(5)}(0)$?
In the Laplace example, why is the MGF finite for $|t|<1$, but not obtained by the same convergent integral calculation for $t\geq 1$ or $t\leq -1$?