Cumulants of Bernoulli and Binomial Laws

Why Bernoulli and Binomial Cumulants Matter

Bernoulli trials are the basic model for repeated binary outcomes: success or failure, response or nonresponse, defective or acceptable. A binomial random variable counts successes across a fixed number of independent trials with the same success probability. Their cumulants summarize the location, variability, asymmetry, and fourth-order shape of these distributions in a form that is especially simple to combine across trials.

The key calculation is the logarithm of the moment-generating function. Its derivatives at zero give the cumulants. For a Bernoulli variable, those derivatives can be obtained directly from a short derivative recursion; for a binomial variable, independence makes the log MGF a multiple of the Bernoulli log MGF.

Bernoulli Cumulant-Generating Function

Definition: A random variable \(X\) has a Bernoulli distribution with success probability \(p\), written \(X\sim\operatorname{Bernoulli}(p)\), if \(P(X=1)=p\) and \(P(X=0)=1-p\), where \(0\leq p\leq 1\). Its moment-generating function is \(M_X(t)=E[e^{tX}]\), and its cumulant-generating function is \(K_X(t)=\log M_X(t)\) wherever the MGF is finite and positive. The \(r\)th cumulant is \(\kappa_r(X)=K_X^{(r)}(0)\), when that derivative exists.

For a Bernoulli variable, the MGF is a finite sum, so it is finite for every real \(t\):

M_X(t)=(1-p)e^{0t}+pe^t=1-p+pe^t, \qquad K_X(t)=\log(1-p+pe^t).

The quantity inside the logarithm is positive for every real \(t\), including when \(p=0\) or \(p=1\). Thus the CGF is defined on the entire real line. For the derivative calculation, first suppose \(0<p<1\), and define

r(t)=\frac{pe^t}{1-p+pe^t}.

This auxiliary function is between zero and one. It satisfies \(r(0)=p\), and differentiating it gives \(r'(t)=r(t)\{1-r(t)\}\). Since \(K_X'(t)=r(t)\), successive derivatives of \(K_X\) can be found by differentiating expressions in \(r(t)\).

Deriving the First Four Bernoulli Cumulants

The first derivative is \(K_X'(t)=r(t)\). Differentiating once more gives \(K_X''(t)=r(t)\{1-r(t)\}\). For the third derivative, apply the product rule and the chain rule:

\begin{aligned} K_X'''(t) &=r'(t)\{1-r(t)\}-r(t)r'(t)\\ &=r(t)\{1-r(t)\}\{1-2r(t)\}. \end{aligned}

Differentiate this expression to obtain the fourth derivative. Writing \(r=r(t)\) within the calculation,

\begin{aligned} K_X''''(t) &=r'(1-r)(1-2r)-r r'(1-2r)-2r(1-r)r'\\ &=r'(1-2r)^2-2r(1-r)r'\\ &=r(1-r)\left\{(1-2r)^2-2r(1-r)\right\}\\ &=r(1-r)\{1-6r+6r^2\}\\ &=r(1-r)\left\{1-6r(1-r)\right\}. \end{aligned}

Evaluating at \(t=0\), where \(r(0)=p\), gives the formulas below. If \(p=0\) or \(p=1\), \(X\) is constant and \(K_X(t)\) is respectively \(0\) or \(t\), so its cumulants of order two and higher are zero. The same formulas give zero for those higher cumulants at both endpoints.

Theorem: If \(X\sim\operatorname{Bernoulli}(p)\), then its first four cumulants are

\kappa_1(X)=p,\qquad \kappa_2(X)=p(1-p),\qquad \kappa_3(X)=p(1-p)(1-2p),\qquad \kappa_4(X)=p(1-p)\{1-6p(1-p)\}.

The formulas hold for every \(0\leq p\leq 1\).

The first cumulant is the mean, and the second is the variance. The third cumulant equals the third central moment; its sign describes the direction of asymmetry when the variance is positive. The fourth cumulant is not the fourth central moment: it is the fourth central moment minus three times the variance squared. It can be negative without any contradiction.

Worked Example: A Bernoulli Variable With Success Probability 0.2

Let \(X\sim\operatorname{Bernoulli}(0.2)\). Here \(p=0.2\), \(q=1-p=0.8\), and \(pq=0.16\). Substitution gives

\begin{aligned} \kappa_1(X)&=p=0.2,\\ \kappa_2(X)&=pq=(0.2)(0.8)=0.16,\\ \kappa_3(X)&=pq(1-2p)=0.16(1-0.4)=0.096,\\ \kappa_4(X)&=pq(1-6pq)=0.16\{1-6(0.16)\}\\ &=0.16(0.04)=0.0064. \end{aligned}

The positive third cumulant indicates right-skewness: most of the mass is at 0, with the tail extending toward 1. This is also confirmed directly by the third central moment:

$$ E[(X-0.2)^3]=(0.8)(-0.2)^3+(0.2)(0.8)^3 =-0.0064+0.1024=0.096. $$

From Bernoulli Trials to a Binomial Count

Definition: A random variable \(Y\) has a binomial distribution with parameters \(n\) and \(p\), written \(Y\sim\operatorname{Binomial}(n,p)\), if it counts successes in \(n\) independent Bernoulli trials, each with success probability \(p\). Here \(n\) is a nonnegative integer and \(0\leq p\leq 1\).

Represent the count as \(Y=X_1+\cdots+X_n\), where the \(X_i\) are independent and each has distribution \(\operatorname{Bernoulli}(p)\). Independence gives a product for the MGF:

M_Y(t)=E\left[e^{t\sum_{i=1}^{n}X_i}\right] =E\left[\prod_{i=1}^{n}e^{tX_i}\right] =\prod_{i=1}^{n}E[e^{tX_i}] =(1-p+pe^t)^n.

Taking logarithms yields \(K_Y(t)=n\log(1-p+pe^t)=nK_X(t)\). Differentiating this identity shows that each of the first four binomial cumulants is \(n\) times the corresponding Bernoulli cumulant.

Theorem: If \(Y\sim\operatorname{Binomial}(n,p)\), then

\kappa_1(Y)=np,\qquad \kappa_2(Y)=np(1-p),\qquad \kappa_3(Y)=np(1-p)(1-2p),\qquad \kappa_4(Y)=np(1-p)\{1-6p(1-p)\}.

These formulas also cover \(p=0\), \(p=1\), and \(n=0\), where the count is constant and all cumulants of order at least two vanish.

This binomial result follows directly from the independent-trial representation and the product rule for MGFs; it does not require calculating the binomial probability mass function term by term. The linear factor \(n\) is what makes cumulants convenient for describing counts built from repeated independent trials.

Worked Example: Cumulants of a Binomial Count

Suppose \(Y\sim\operatorname{Binomial}(10,0.3)\). Then \(q=0.7\) and \(pq=(0.3)(0.7)=0.21\). Using the binomial formulas,

\begin{aligned} \kappa_1(Y)&=10(0.3)=3,\\ \kappa_2(Y)&=10(0.21)=2.1,\\ \kappa_3(Y)&=10(0.21)(1-0.6)=0.84,\\ \kappa_4(Y)&=10(0.21)\{1-6(0.21)\}\\ &=2.1(1-1.26)=2.1(-0.26)=-0.546. \end{aligned}

Thus the expected count is 3 and its variance is 2.1. The positive third cumulant indicates right-skewness, while the negative fourth cumulant is valid: a fourth cumulant is not required to be nonnegative. As a check, each result is ten times its Bernoulli counterpart at \(p=0.3\).

Worked Example: A Fair Binomial Count

Let \(Y\sim\operatorname{Binomial}(8,0.5)\). Since \(p=q=0.5\), the distribution is symmetric about 4. The formulas give

\begin{aligned} \kappa_1(Y)&=8(0.5)=4,\\ \kappa_2(Y)&=8(0.25)=2,\\ \kappa_3(Y)&=8(0.25)(1-1)=0,\\ \kappa_4(Y)&=8(0.25)\{1-6(0.25)\} =2(1-1.5)=-1. \end{aligned}

The zero third cumulant agrees with symmetry: the centered distribution has no directional asymmetry. The fourth cumulant is negative, even though the fourth central moment is positive. For this binomial count, the fourth central moment is \(\kappa_4+3\kappa_2^2=-1+3(2^2)=11\), which is positive as it must be.

Interpreting the Formulas and Avoiding Common Errors

The factor \(p(1-p)\) appears in every cumulant of order two or higher. It vanishes at the degenerate endpoints \(p=0\) and \(p=1\), where there is no randomness. For \(0<p<1\), the sign of \(\kappa_3\) is the sign of \(1-2p\): it is positive when successes are less common than failures, zero at \(p=1/2\), and negative when successes are more common. For a binomial count, multiplying by \(n\) changes the magnitude but not that sign.

The factor \(1-6p(1-p)\) controls the sign of the fourth cumulant. It can be negative; for example, it is \(-1/2\) at \(p=1/2\). This does not mean a negative fourth central moment. The distinction is important because \(\kappa_4\) subtracts \(3\operatorname{Var}(X)^2\) from the fourth central moment.

Mistake: assigning skewness from the most likely value. A Bernoulli variable with \(p=0.2\) has most of its mass at 0, but its positive third cumulant does not indicate a tail toward 0. The centered value at 1 is \(0.8\), and its contribution to the third central moment outweighs the negative contribution at 0; the tail extends toward 1.
Mistake: treating \(\kappa_4\) as a fourth moment. The fourth cumulant can be negative. Use the relation \(\kappa_4=E[(X-E[X])^4]-3\operatorname{Var}(X)^2\) to distinguish it from the nonnegative fourth central moment.
Mistake: forgetting the factor \(n\) for a binomial count. Each cumulant shown here is \(n\) times the Bernoulli cumulant, because the binomial CGF is \(n\) times the one-trial CGF.

Key takeaway: A Bernoulli variable has CGF \(\log(1-p+pe^t)\), and its first four cumulants are \(p\), \(p(1-p)\), \(p(1-p)(1-2p)\), and \(p(1-p)\{1-6p(1-p)\}\). For a binomial count, multiply each of these by the number of trials.

Check Your Understanding

Use the CGF and formulas developed above to answer the following questions.

Write the MGF and CGF of a Bernoulli variable with success probability \(p\), and state the domain of its CGF.
For \(X\sim\operatorname{Bernoulli}(0.7)\), calculate the first three cumulants and determine the sign of the third cumulant.
For \(Y\sim\operatorname{Binomial}(6,0.5)\), calculate its first four cumulants.
Explain why the fourth cumulant of a fair Bernoulli variable is negative even though its fourth central moment is positive.
Show briefly why the binomial CGF is \(n\) times the Bernoulli CGF.