Additivity for Independent Sums

Why Additivity Matters

When a random quantity is built by adding independent contributions, calculating the distribution of the sum directly can be inconvenient. Cumulants offer a simpler route: under the right conditions, each cumulant of the sum is just the sum of the corresponding cumulants of its components. This statement covers the mean and variance, but also every higher-order cumulant that exists through a moment-generating function near zero.

The key mechanism is a product rule. Independence turns the moment-generating function (MGF) of a sum into a product of the individual MGFs. Taking logarithms changes that product into a sum, and differentiating the resulting cumulant-generating function (CGF) gives the cumulants. The domain conditions matter: an MGF must be finite on an open interval containing zero, not merely at zero.

MGFs and Cumulants Near Zero

Definition: The moment-generating function of a random variable \(X\) is \(M_X(t)=E[e^{tX}]\) at each real \(t\) for which this expectation is finite. If \(M_X(t)\) is finite on an open interval containing zero, the cumulant-generating function is \(K_X(t)=\log M_X(t)\) on a neighborhood of zero. The cumulant of order \(r\), for a positive integer \(r\), is \(\kappa_r(X)=K_X^{(r)}(0)\).

The definition of \(\kappa_r(X)\) requires that the relevant derivative exist. Finiteness of the MGF near zero ensures that derivatives of every order exist there. Here is the needed justification, including the choice of a bound strictly inside the MGF's domain.

Suppose \(M_X(t)\) is finite for \(|t|<a\), where \(a>0\). Choose \(b\) with \(0<b<a\). Then \(M_X(b)\) and \(M_X(-b)\) are finite, and

E[e^{b|X|}] =E[e^{bX}\mathbf{1}_{\{X\geq 0\}}]+E[e^{-bX}\mathbf{1}_{\{X<0\}}] \leq M_X(b)+M_X(-b)<\infty.

For any nonnegative integer \(r\), there is a finite constant \(C_{r,b}\) such that \(u^r\leq C_{r,b}e^{bu/2}\) for every \(u\geq0\). This follows because \(u^re^{-bu/2}\) is bounded on \([0,\infty)\). Consequently, when \(|t|\leq b/2\),

|X|^r e^{tX} \leq |X|^r e^{|t||X|} \leq C_{r,b}e^{b|X|}.

The right-hand side is integrable by the preceding bound. Differentiating \(e^{tX}\) with respect to \(t\) gives \(X^r e^{tX}\); dominated convergence therefore justifies taking these derivatives inside the expectation on a neighborhood of zero. For the successive derivatives, the same argument applies at each order, using the bound for that order and a slightly smaller neighborhood if needed. Thus \(M_X\) has derivatives of every order near zero. Since \(M_X(0)=1\) and \(M_X\) is continuous, it is positive on some neighborhood of zero, so \(K_X=\log M_X\) is also differentiable there to every order.

Additivity for Two Independent Variables

Theorem: Let \(X\) and \(Y\) be independent random variables. Suppose their MGFs are finite on open intervals containing zero. Then the MGF of \(S=X+Y\) is finite on a neighborhood of zero, and on a sufficiently small common neighborhood,

M_S(t)=M_X(t)M_Y(t), \qquad K_S(t)=K_X(t)+K_Y(t).

In particular, for every positive integer \(r\),

\kappa_r(X+Y)=\kappa_r(X)+\kappa_r(Y).

Proof. Choose \(\delta>0\) small enough that both \(M_X(t)\) and \(M_Y(t)\) are finite whenever \(|t|<\delta\). For each such \(t\), \(e^{tX}\) and \(e^{tY}\) are nonnegative integrable random variables. Independence gives the factorization of their expectation:

M_S(t) =E[e^{t(X+Y)}] =E[e^{tX}e^{tY}] =E[e^{tX}]E[e^{tY}] =M_X(t)M_Y(t).

Both factors are strictly positive, so taking their logarithms is valid and yields \(K_S(t)=K_X(t)+K_Y(t)\) for \(|t|<\delta\). All three CGFs have derivatives of every order on a possibly smaller neighborhood of zero, by the MGF argument above. Differentiating the equality \(r\) times and evaluating at zero gives

\kappa_r(S) =K_S^{(r)}(0) =K_X^{(r)}(0)+K_Y^{(r)}(0) =\kappa_r(X)+\kappa_r(Y).

This proves both identities. \(\square\)

Finite Sums

The two-variable result extends directly to any fixed finite number of mutually independent random variables. Mutual independence is the condition that the joint distribution factors across the whole collection; it is stronger than requiring only pairwise independence when there are three or more variables.

Theorem: Let \(X_1,\ldots,X_m\) be mutually independent, where \(m\) is a positive integer, and suppose each \(M_{X_j}\) is finite on an open interval containing zero. Then, for \(S=\sum_{j=1}^{m}X_j\), the MGF of \(S\) is finite near zero and

K_S(t)=\sum_{j=1}^{m}K_{X_j}(t), \qquad \kappa_r(S)=\sum_{j=1}^{m}\kappa_r(X_j) \quad\text{for every positive integer }r.

Proof. Since there are finitely many MGFs, their domains all contain some common interval \((-\delta,\delta)\) with \(\delta>0\). Mutual independence factors the expectation of the product:

M_S(t) =E\!\left[\prod_{j=1}^{m}e^{tX_j}\right] =\prod_{j=1}^{m}E[e^{tX_j}] =\prod_{j=1}^{m}M_{X_j}(t), \qquad |t|<\delta.

Taking logarithms gives \(K_S(t)=\sum_{j=1}^{m}K_{X_j}(t)\). Each term has derivatives of every order near zero, so differentiating this finite sum \(r\) times and setting \(t=0\) gives the stated cumulant identity. \(\square\)

Worked Calculations

Worked Example: A Sum of Two Independent Bernoulli Variables

Let \(X\sim\operatorname{Bernoulli}(0.3)\) and \(Y\sim\operatorname{Bernoulli}(0.4)\) be independent, and let \(S=X+Y\). For a Bernoulli variable with success probability \(p\),

M(t)=1-p+pe^t, \qquad K(t)=\log(1-p+pe^t).

The theorem gives the CGF of \(S\) as

K_S(t)=\log(0.7+0.3e^t)+\log(0.6+0.4e^t).

For a Bernoulli variable, differentiating \(K\) at zero gives \(\kappa_1=p\), \(\kappa_2=p(1-p)\), and \(\kappa_3=p(1-p)(1-2p)\). Applying these formulas separately,

\begin{aligned} \kappa_1(S)&=0.3+0.4=0.7,\\ \kappa_2(S)&=0.3(0.7)+0.4(0.6)=0.21+0.24=0.45,\\ \kappa_3(S)&=0.3(0.7)(1-0.6)+0.4(0.6)(1-0.8)\\ &=0.084+0.048=0.132. \end{aligned}

As a check, \(E[S]=E[X]+E[Y]=0.7\), and independence gives \(\operatorname{Var}(S)=0.21+0.24=0.45\), agreeing with the first two cumulants. The third cumulant is not generally zero, and its value here is the sum \(0.132\).

Worked Example: Independent Poisson Variables

Suppose \(X\sim\operatorname{Poisson}(2)\) and \(Y\sim\operatorname{Poisson}(3)\) are independent. The Poisson MGF with parameter \(\lambda\) is \(M(t)=\exp(\lambda(e^t-1))\), finite for all real \(t\). Hence

K_X(t)=2(e^t-1), \qquad K_Y(t)=3(e^t-1), \qquad K_{X+Y}(t)=5(e^t-1).

For every positive integer \(r\), the \(r\)th derivative of \(e^t\) is \(e^t\), whose value at zero is 1. Therefore

\kappa_r(X+Y) =K_{X+Y}^{(r)}(0) =5 =2+3 =\kappa_r(X)+\kappa_r(Y).

In particular, the mean and variance of the sum are both 5, and every higher-order cumulant is also 5.

Worked Example: Independent Normal Variables

Let \(X\sim N(1,4)\) and \(Y\sim N(-2,9)\) be independent; the second argument denotes variance. A normal variable \(N(\mu,\sigma^2)\) has CGF \(K(t)=\mu t+\sigma^2t^2/2\). Thus

K_X(t)=t+2t^2, \qquad K_Y(t)=-2t+\frac{9}{2}t^2.

Adding these functions gives

K_{X+Y}(t)=-t+\frac{13}{2}t^2.

The coefficient of \(t\) gives \(\kappa_1(X+Y)=-1\); the second derivative is 13, so \(\kappa_2(X+Y)=13\). Every derivative of order three or higher is zero, so all higher cumulants vanish. These results also follow by adding the individual cumulants: the means sum to \(1+(-2)=-1\), and the variances sum to \(4+9=13\).

Why Independence Is Essential

Independence was used at the exact point where the expectation of a product was factored into a product of expectations. Without it, the MGF of a sum need not be the product of the individual MGFs, so additivity of cumulants is not guaranteed.

For example, let \(X\sim N(0,1)\) and set \(Y=X\). Both variables have an MGF finite everywhere, but they are not independent. Their individual second cumulants are each 1. The sum is \(X+Y=2X\), whose variance—and hence second cumulant—is \(4\), not \(1+1=2\). Equivalently, \(M_{X+Y}(t)=E[e^{2tX}]=e^{2t^2}\), whereas \(M_X(t)M_Y(t)=e^{t^2}\). The product identity fails because the variables are dependent.

Common mistake: It is tempting to argue that “the cumulant of a sum is the sum of the cumulants” without checking independence or the MGF condition. The correction is to verify mutual independence and finiteness of each MGF on a neighborhood of zero first. Only then can the product factorization, logarithm, and derivative argument be applied.

Additivity is useful because it lets us calculate cumulants component by component, rather than deriving the full distribution of a sum or expanding its moments from scratch. The theorem applies to each order separately, and it does not require the summands to have identical distributions.

Check Your Understanding

Use the MGF factorization and cumulant definitions to answer these questions.

State the condition on the MGF domain that ensures cumulants of every order are defined.
If \(X\) and \(Y\) are independent and both MGFs are finite near zero, what is \(K_{X+Y}(t)\) on a common neighborhood of zero?
Two independent variables have second cumulants 4 and 7. What is the second cumulant of their sum, and which familiar quantity does it represent?
Why does pairwise independence alone not justify the finite-sum theorem for three or more variables?
For \(X\sim N(0,1)\) and \(Y=X\), calculate the variance of \(X+Y\) and explain why adding the individual variances gives the wrong answer.