Deriving the First Four Cumulants

Why These Four Cumulants Have Familiar Forms

Cumulants are defined through derivatives of the log moment-generating function, but their statistical meaning is easiest to see after writing those derivatives in terms of moments. The first four reveal a useful pattern: the first cumulant is the mean, the second is the variance, and the third is the third central moment. The fourth is different: it is the fourth central moment with three copies of the squared variance removed.

That subtraction is not an arbitrary adjustment. It appears when the logarithm of the MGF is expanded, and it distinguishes the fourth cumulant from the fourth central moment. The derivation below makes the coefficients explicit and checks them in several distributions.

Definition: Suppose the MGF \(M_X(t)=E[e^{tX}]\) is finite on an open interval containing \(0\). The raw moment of order \(r\) is \(m_r=E[X^r]\), and the cumulant-generating function is \(K_X(t)=\log M_X(t)\). The cumulant of order \(r\) is \(\kappa_r=K_X^{(r)}(0)\). Write \(\mu=m_1=E[X]\), and define the central moment of order \(r\) by \(c_r=E[(X-\mu)^r]\).

The MGF condition ensures that the derivatives needed here exist and that \(M_X(t)\) has a Taylor expansion through order four at \(0\). Also, \(M_X(0)=1\), so \(M_X(t)>0\) for \(t\) sufficiently close to \(0\); its logarithm is therefore defined there. The raw-moment expansion of the MGF is

M_X(t)=1+m_1t+\frac{m_2}{2}t^2+\frac{m_3}{6}t^3+\frac{m_4}{24}t^4+o(t^4).

Here \(o(t^4)\) denotes a remainder which, divided by \(t^4\), tends to zero as \(t\) tends to \(0\). The expansion follows from differentiating the MGF at zero: \(M_X^{(r)}(0)=E[X^r]=m_r\) for \(r\leq 4\). These moment identities hold under the stated MGF condition.

Deriving the Raw-Moment Formulas

Set \(u=M_X(t)-1\), so \(u\) tends to zero with \(t\). The Taylor expansion \(\log(1+u)=u-u^2/2+u^3/3-u^4/4+o(u^4)\) can now be applied. Substituting the MGF expansion and collecting powers of \(t\) gives the coefficients of \(K_X(t)\). Since the coefficient of \(t^r\) in \(K_X(t)\) is \(\kappa_r/r!\), multiplying each coefficient by \(r!\) yields the cumulant.

Theorem: If \(M_X(t)\) is finite on an open interval containing zero, then the first four cumulants are

\begin{aligned} \kappa_1&=m_1,\\ \kappa_2&=m_2-m_1^2,\\ \kappa_3&=m_3-3m_1m_2+2m_1^3,\\ \kappa_4&=m_4-4m_1m_3-3m_2^2+12m_1^2m_2-6m_1^4. \end{aligned}

For clarity, the fourth-order coefficient includes contributions from all four terms in the logarithm expansion. Writing \(a_r=m_r/r!\), the terms of order \(t^4\) are \(a_4\) from \(u\), \(-a_2^2/2-a_1a_3\) from \(-u^2/2\), \(a_1^2a_2\) from \(u^3/3\), and \(-a_1^4/4\) from \(-u^4/4\). Multiplying their sum by \(24\) gives the stated formula for \(\kappa_4\).

Worked Example: Expanding the Log MGF Through Fourth Order

Let the raw moments of \(X\) be \(m_1=2\), \(m_2=5\), \(m_3=14\), and \(m_4=43\). Using the formulas, calculate the first four cumulants. The values are chosen to be consistent with a distribution: for example, they are the raw moments of a normal random variable with mean 2 and variance 1.

First cumulant. \(\kappa_1=m_1=2\).

Second cumulant. \(\kappa_2=m_2-m_1^2=5-2^2=1\).

Third cumulant. \(\kappa_3=m_3-3m_1m_2+2m_1^3=14-3(2)(5)+2(2^3)=14-30+16=0\).

Fourth cumulant. \(\kappa_4=m_4-4m_1m_3-3m_2^2+12m_1^2m_2-6m_1^4\). Substitution gives \(43-4(2)(14)-3(5^2)+12(2^2)(5)-6(2^4)=43-112-75+240-96=0\).

Thus the cumulants are \(2,1,0,0\). A translation changes the first cumulant but not the higher ones; the central-moment interpretation proved below explains why the last three values are \(1,0,0\) in this example.

Why the Cumulants Equal Central-Moment Quantities

Centering a random variable has a particularly simple effect on its cumulant-generating function. Let \(Z=X-\mu\), where \(\mu=E[X]\). Then \(M_Z(t)=e^{-\mu t}M_X(t)\), so taking logarithms subtracts only the linear term \(\mu t\). Consequently, all cumulants of order at least two are unchanged by centering. For \(Z\), the first raw moment is zero, while its raw moments of orders two through four are exactly the central moments of \(X\).

Theorem: Under the MGF condition above, the first four cumulants satisfy

\kappa_1=\mu,\qquad \kappa_2=c_2=\operatorname{Var}(X),\qquad \kappa_3=c_3,\qquad \kappa_4=c_4-3c_2^2.

Proof. For \(Z=X-\mu\), the MGF identity \(M_Z(t)=e^{-\mu t}M_X(t)\) gives \(K_Z(t)=K_X(t)-\mu t\). Differentiating at zero, \(K_Z'(0)=K_X'(0)-\mu=0\), and \(K_Z^{(r)}(0)=K_X^{(r)}(0)\) for each \(r\geq2\), because the derivatives of \(\mu t\) of those orders are zero.

The raw moments of \(Z\) are \(E[Z]=0\), \(E[Z^2]=c_2\), \(E[Z^3]=c_3\), and \(E[Z^4]=c_4\). Apply the raw-moment formulas to \(Z\). Its second cumulant is \(c_2-0^2=c_2\), and its third is \(c_3-3(0)c_2+2(0)^3=c_3\). Its fourth is \(c_4-4(0)c_3-3c_2^2+12(0)^2c_2-6(0)^4=c_4-3c_2^2\). Since the cumulants of \(Z\) of order two and higher equal those of \(X\), these are the corresponding cumulants of \(X\). Finally, \(K_X'(0)=E[X]=\mu\), and \(c_2=\operatorname{Var}(X)\). \(\square\)

Worked Example: Bernoulli Cumulants and Central Moments

Let \(X\sim\operatorname{Bernoulli}(p)\), with \(0<p<1\). Since \(X\) takes values zero and one, \(X^r=X\) for every positive integer \(r\), so \(m_1=m_2=m_3=m_4=p\).

Substituting these raw moments gives \(\kappa_1=p\), \(\kappa_2=p-p^2=p(1-p)\), and \(\kappa_3=p-3p^2+2p^3=p(1-p)(1-2p)\). For the fourth cumulant, \[ \kappa_4=p-4p^2-3p^2+12p^3-6p^4 =p-7p^2+12p^3-6p^4 =p(1-p)\{1-6p(1-p)\}. \]

The central-moment formulas agree. Write \(q=1-p\). The centered variable \(X-p\) equals \(q\) with probability \(p\) and \(-p\) with probability \(q\). Thus \(c_2=pq^2+qp^2=pq\), and \(c_3=pq^3-qp^3=pq(q-p)=pq(1-2p)\). Also \(c_4=pq^4+qp^4=pq(q^3+p^3)=pq(1-3pq)\), since \(p+q=1\). Therefore \(c_4-3c_2^2=pq(1-3pq)-3p^2q^2=pq(1-6pq)\), the same \(\kappa_4\).

Worked Example: A Symmetric Two-Point Distribution

Let \(X\) take the values \(-1\) and \(1\), each with probability \(1/2\). Its MGF is \(M_X(t)=(e^{-t}+e^t)/2=\cosh(t)\), which is finite for every real \(t\). Symmetry gives \(m_1=m_3=0\), while \(m_2=m_4=1\).

The raw-moment formulas give \(\kappa_1=0\), \(\kappa_2=1\), and \(\kappa_3=0\). For the fourth, \(\kappa_4=m_4-3m_2^2=1-3=-2\).

Here \(X\) is already centered, so \(c_4=1\) and \(c_2=1\). Hence \(c_4-3c_2^2=1-3=-2\), as required. In particular, the fourth cumulant can be negative even though the fourth central moment itself is nonnegative.

Worked Example: Exponential Distribution

Let \(X\) have an exponential distribution with rate \(2\), so its density is \(f(x)=2e^{-2x}\) for \(x\geq0\). Its MGF is \(M_X(t)=2/(2-t)\) for \(t<2\), which is finite on an open interval containing zero. The raw moments are \(m_r=r!/2^r\); in particular, \(m_1=1/2\), \(m_2=1/2\), \(m_3=3/4\), and \(m_4=3/2\).

The formulas yield \(\kappa_1=1/2\) and \(\kappa_2=1/2-(1/2)^2=1/4\). For the third cumulant, \(\kappa_3=3/4-3(1/2)(1/2)+2(1/2)^3=3/4-3/4+1/4=1/4\).

For the fourth cumulant, \[ \kappa_4=\frac32-4\left(\frac12\right)\left(\frac34\right) -3\left(\frac12\right)^2 +12\left(\frac12\right)^2\left(\frac12\right) -6\left(\frac12\right)^4 =\frac38. \] The central moments check the result: \(c_2=1/4\), \(c_3=1/4\), and \(c_4=m_4-4m_1m_3+6m_1^2m_2-3m_1^4=3/2-3/2+3/4-3/16=9/16\). Therefore \(c_4-3c_2^2=9/16-3/16=6/16=3/8\).

Common Pitfall: Treating the Fourth Cumulant as a Fourth Moment

A frequent error is to identify \(\kappa_4\) with \(E[(X-E[X])^4]\). The correct identity is \(\kappa_4=c_4-3c_2^2\). For the symmetric two-point example, the fourth central moment is \(1\), but the fourth cumulant is \(-2\). The distinction matters whenever a calculation uses cumulants: the fourth cumulant measures what remains after removing the contribution associated with variance alone.

A second useful check is to keep raw and central moments distinct during algebra. The raw formula for \(\kappa_4\) contains \(m_4\), \(m_1m_3\), \(m_2^2\), \(m_1^2m_2\), and \(m_1^4\); it is not obtained by replacing each raw moment with a central one. Centering first is a valid alternative, but then the first raw moment of the centered variable is zero and the fourth-cumulant formula reduces to \(c_4-3c_2^2\).

Key takeaway: With an MGF finite near zero, expand \(\log M_X(t)\) through fourth order to obtain the raw-moment formulas. Equivalently, center \(X\): the first cumulant is the mean, the next two are the second and third central moments, and the fourth cumulant is the fourth central moment minus three times the squared variance.

Check Your Understanding

Use the definitions and derivations above to answer the following.

For a random variable with raw moments \(m_1=1\), \(m_2=3\), \(m_3=7\), and \(m_4=25\), calculate \(\kappa_1,\kappa_2,\kappa_3,\kappa_4\).
Explain why centering \(X\) changes its first cumulant but leaves its second, third, and fourth cumulants unchanged.
If \(c_2=2\) and \(c_4=15\), what is \(\kappa_4\)? Why is it not equal to \(c_4\)?
For a Bernoulli random variable with \(p=1/2\), calculate the third and fourth cumulants and compare them with the corresponding central-moment expressions.
Why does the existence of an MGF on an open interval around zero justify using its derivatives to obtain the first four raw moments?