Why Dependence Changes a Sum’s Cumulants
For independent random variables, cumulants of a sum add one variable at a time. Dependence changes that picture: the sum’s cumulants also include terms describing interactions among distinct variables. For variance, those terms are covariances. At order three, they include mixed third-order joint cumulants.
The key is to keep track of every ordered choice of variables. Some choices repeat an index, such as \((X_1,X_1,X_2)\); others use distinct indices. The expansion below counts all of them, so multiplicities arise automatically rather than being added by guesswork.
Joint Cumulants and the Sum Expansion
Let \(X_1,\ldots,X_m\) have a joint moment-generating function that is finite on an open neighborhood of the origin. Write their joint cumulant-generating function as
For an ordered list of indices \(i_1,\ldots,i_r\), each between \(1\) and \(m\), the corresponding order-\(r\) joint cumulant is
Indices in this definition may repeat. For example, \(\kappa(X_1,X_1,X_2)\) is an order-three joint cumulant, not an order-two cumulant. The moment-generating-function condition ensures that the derivatives used here exist and that \(K\) is smooth near the origin.
To prove the formula, evaluate the joint cumulant-generating function along the diagonal:
Define \(g(t)=K(t,\ldots,t)\). Differentiating a function along this diagonal applies the operator \(D=\sum_{j=1}^m \partial/\partial t_j\) to \(K\). Repeating that operation \(r\) times gives
The derivatives can be taken in any order because \(K\) has continuous derivatives of these orders near the origin. By definition, \(g^{(r)}(0)=\kappa_r(S)\), and each term in the final sum is the joint cumulant indexed by its ordered tuple. This proves the expansion. \(\square\)
The theorem does not assume independence. If some variables are independent, certain mixed cumulants vanish, but without independence the mixed terms must generally be retained.
Variance and Third-Cumulant Forms
At order two, a joint cumulant is a covariance, including when the two arguments are the same: \(\kappa(X_i,X_j)=\operatorname{Cov}(X_i,X_j)\). Therefore the expansion gives
The last equality uses the symmetry \(\operatorname{Cov}(X_i,X_j)=\operatorname{Cov}(X_j,X_i)\). Each distinct pair occurs twice in the ordered double sum, once in each order.
At order three, the joint cumulant has the explicit form
Equivalently, it is the third mixed central moment:
This follows by centering the variables in the defining log moment-generating function. The centered variables have mean zero, so in the degree-three term of the log expansion, only the joint third moment remains. In particular, with all three arguments equal, the third cumulant is the third central moment. Applying the sum expansion yields
Repeated indices matter here too. For two variables \(X\) and \(Y\), the eight ordered triples group into four types:
There are three placements of the single \(Y\) in a triple with two \(X\)'s, and three placements of the single \(X\) in a triple with two \(Y\)'s. These are the source of the coefficients 3.
Worked Example: Variance and Third Cumulant from a Joint Bernoulli Law
Let \(X\) and \(Y\) be Bernoulli variables with joint probabilities \(P(X=1,Y=1)=0.3\), \(P(X=1,Y=0)=0.2\), \(P(X=0,Y=1)=0.1\), and \(P(X=0,Y=0)=0.4\). Find the variance and third cumulant of \(S=X+Y\).
The marginal success probabilities are \(P(X=1)=0.3+0.2=0.5\) and \(P(Y=1)=0.3+0.1=0.4\). Thus
Also \(E[XY]=P(X=1,Y=1)=0.3\), so
The variance expansion gives \(\operatorname{Var}(S)=0.25+0.24+2(0.1)=0.69\). Check this from the distribution: \(S\) is 0, 1, or 2 with probabilities \(0.4,0.3,0.3\). Hence \(E[S]=0.9\), \(E[S^2]=0.3+4(0.3)=1.5\), and \(\operatorname{Var}(S)=1.5-(0.9)^2=0.69\).
For the third cumulant, \(\kappa_3(X)=0\) because \(X\) is Bernoulli with success probability \(0.5\), and \(\kappa_3(Y)=0.4(0.6)(1-2(0.4))=0.048\). Since \((X-0.5)^2=0.25\) for both values of \(X\),
For a Bernoulli variable with success probability \(p\), its centered square satisfies \((Y-p)^2=p(1-p)+(1-2p)(Y-p)\). With \(p=0.4\), this gives
Consequently,
Directly, the centered values of \(S\) are \(-0.9,0.1,1.1\), with probabilities \(0.4,0.3,0.3\). Their third moment is \(0.4(-0.9)^3+0.3(0.1)^3+0.3(1.1)^3=0.108\), confirming the expansion.
Worked Example: A Sum of Identical Dependent Variables
Let \(Z\sim\operatorname{Bernoulli}(0.25)\), and set \(X_1=X_2=X_3=Z\). These variables are completely dependent. For \(S=X_1+X_2+X_3=3Z\), calculate the second and third cumulants from the ordered-tuple expansion.
There are \(3^2=9\) ordered pairs, and every pair contributes \(\kappa_2(Z)=0.25(0.75)=3/16\). Thus \(\operatorname{Var}(S)=9(3/16)=27/16\). This agrees with \(S=3Z\), whose variance is \(9\operatorname{Var}(Z)=27/16\).
There are \(3^3=27\) ordered triples. Every triple contributes \(\kappa_3(Z)=0.25(0.75)(1-2(0.25))=3/32\). Therefore
As a direct check, the third cumulant of \(3Z\) is \(3^3\kappa_3(Z)=27(3/32)=81/32\). The number 27 counts ordered triples, not just the possible patterns of repeated indices.
Worked Example: Covariance Terms in a Three-Variable Sum
Let \(U\) and \(V\) be independent Rademacher variables, each taking \(1\) and \(-1\) with probability \(1/2\). Define \(X_1=U\), \(X_2=U+V\), and \(X_3=V\). Find the variance and third cumulant of \(S=X_1+X_2+X_3\).
Each variable has mean zero. Their variances are \(1,2,1\). The covariances are \(\operatorname{Cov}(X_1,X_2)=E[U(U+V)]=1\), \(\operatorname{Cov}(X_1,X_3)=E[UV]=0\), and \(\operatorname{Cov}(X_2,X_3)=E[(U+V)V]=1\), using independence and the zero means. Hence
Indeed \(S=2U+2V\), so independence gives \(\operatorname{Var}(S)=4\operatorname{Var}(U)+4\operatorname{Var}(V)=8\). Finally, the joint distribution of \((U,V)\) is unchanged by replacing \((U,V)\) with \((-U,-V)\). This sends \(S\) to \(-S\), so \(S\) has a symmetric distribution about zero and \(E[S^3]=0\). Since \(E[S]=0\), \(\kappa_3(S)=0\). The order-three expansion therefore sums to zero, although its individual mixed terms need not all vanish.
Common Mistake: Keeping Only the Individual Cumulants
A tempting but incorrect step is to write \(\kappa_r(X_1+\cdots+X_m)=\sum_i\kappa_r(X_i)\) for dependent variables. That identity omits every mixed joint cumulant. It is valid when the variables are independent, because the joint cumulants involving more than one independent component vanish; it is not a general rule for dependent sums.
For variance, the omission is immediately visible: leaving out covariances would give \(0.25+0.24=0.49\) in the Bernoulli example, rather than the correct \(0.69\). At order three, mixed terms can add to or subtract from the individual third cumulants. The reliable procedure is to write the ordered-index sum first, then use independence or other structure to identify which terms are zero.
Check Your Understanding
Use the ordered-tuple expansion to answer each question.
- For \(S=X_1+X_2+X_3\), how many ordered terms appear in the expansion of \(\kappa_3(S)\)?
- Write the variance of \(X_1+X_2+X_3\) in terms of the individual variances and distinct pairwise covariances.
- Why does the order-three expansion for \(X+Y\) contain a coefficient of 3 on each mixed-cumulant type?
- For dependent \(X\) and \(Y\), what terms are omitted by the claim \(\kappa_3(X+Y)=\kappa_3(X)+\kappa_3(Y)\)?
- If \(X_1=X_2=Z\), how many ordered pairs contribute to the variance expansion, and what is each pair’s covariance?