Cumulants of Independent Linear Combinations

From Independent Inputs to Linear Combinations

Many statistical quantities are built by combining the same underlying random variables in different ways. A contrast subtracts one input from another; a weighted sum assigns unequal coefficients; and two outputs may share some inputs even when the original inputs are independent. Cumulants make these constructions manageable: for each order, independence turns the calculation into a sum over inputs, while the coefficients determine each input’s contribution.

The key is to keep track of the coefficient attached to each input. In particular, an input with coefficient $a$ contributes its $r$th cumulant multiplied by $a^r$. Negative coefficients therefore reverse odd-order contributions but not even-order contributions.

Definition: For a random variable $X$ whose moment-generating function $M_X(t)=E[e^{tX}]$ is finite on an open interval containing zero, its cumulant-generating function is $K_X(t)=\log M_X(t)$ on that interval. The $r$th cumulant of $X$ is $\kappa_r(X)=K_X^{(r)}(0)$, for $r=1,2,\ldots$. In particular, $\kappa_1(X)=E[X]$ and $\kappa_2(X)=\operatorname{Var}(X)$.

Suppose $X_1,\ldots,X_n$ are independent, and let $Y=\sum_{i=1}^n a_iX_i$, where the $a_i$ are fixed real numbers. If every $X_i$ has a moment-generating function finite near zero, then $Y$ does too. The following result gives all of its cumulants directly from the inputs.

Theorem: Let $X_1,\ldots,X_n$ be independent random variables, each with a moment-generating function finite on an open interval containing zero. For fixed real coefficients $a_1,\ldots,a_n$, and every integer $r\geq 1$, $$ \kappa_r\!\left(\sum_{i=1}^n a_iX_i\right) =\sum_{i=1}^n a_i^r\kappa_r(X_i). $$ A term with $a_i=0$ is understood to contribute zero.

Why the Coefficients Are Raised to the Cumulant Order

The formula follows from the cumulant-generating function and the product rule for the moment-generating function of independent variables. Here are the domain and differentiation details. For each $i$, choose $\delta_i>0$ so that $M_{X_i}(s)$ is finite whenever $|s|<\delta_i$. If $a_i\ne 0$, then $M_{X_i}(a_it)$ is finite whenever $|t|<\delta_i/|a_i|$. Since there are finitely many inputs, there is a common open interval around zero on which all these moment-generating functions are finite. Coefficients equal to zero impose no restriction.

On that common interval, independence and finiteness give $M_Y(t)=\prod_{i=1}^n M_{X_i}(a_it)$. Taking logarithms and differentiating $r$ times at zero proves the result:

\begin{aligned} K_Y(t) &=\log E\!\left[\exp\!\left(t\sum_{i=1}^n a_iX_i\right)\right]\\ &=\log E\!\left[\prod_{i=1}^n e^{ta_iX_i}\right] =\log\!\left(\prod_{i=1}^n M_{X_i}(a_it)\right)\\ &=\sum_{i=1}^n K_{X_i}(a_it),\\ K_Y^{(r)}(0) &=\sum_{i=1}^n a_i^r K_{X_i}^{(r)}(0) =\sum_{i=1}^n a_i^r\kappa_r(X_i). \end{aligned}

The chain rule supplies $a_i^r$: each of the $r$ differentiations of $K_{X_i}(a_it)$ contributes one factor of $a_i$. This argument also handles negative and zero coefficients. The regularity assumption ensures that the cumulants used in the calculation are defined.

Several Linear Combinations at Once

Write $m$ linear combinations of the same $n$ independent inputs as $Y_j=\sum_{i=1}^n a_{ji}X_i$, for $j=1,\ldots,m$. For any fixed real numbers $c_1,\ldots,c_m$, their scalar combination can be rewritten by collecting the coefficient of each input:

\sum_{j=1}^m c_jY_j =\sum_{i=1}^n\left(\sum_{j=1}^m c_ja_{ji}\right)X_i.

Applying the theorem to this expression gives a useful formula for any scalar combination of the outputs.

Corollary: Under the theorem’s assumptions, for every integer $r\geq1$, $$ \kappa_r\!\left(\sum_{j=1}^m c_jY_j\right) =\sum_{i=1}^n\left(\sum_{j=1}^m c_ja_{ji}\right)^r\kappa_r(X_i). $$ The coefficient is first combined across outputs for each input, and then raised to the $r$th power.

This order matters. If two outputs use the same input, that input’s coefficients must be added before taking a power. Calculating the outputs separately and treating them as independent would generally lose this shared-input contribution.

Worked Example: A Difference of Independent Poisson Counts

Let $X_1\sim\operatorname{Poisson}(4)$ and $X_2\sim\operatorname{Poisson}(1)$ be independent, and define the contrast $D=X_1-X_2$. A Poisson random variable with parameter $\lambda$ has cumulant-generating function $K(t)=\lambda(e^t-1)$, so every positive-order cumulant equals $\lambda$.

The coefficients in $D$ are $1$ and $-1$. Thus, for every $r\geq1$, $$ \kappa_r(D)=1^r(4)+(-1)^r(1)=4+(-1)^r. $$ In particular, $\kappa_1(D)=3$, $\kappa_2(D)=5$, $\kappa_3(D)=3$, and $\kappa_4(D)=5$. The mean is the difference of the Poisson means, while the variance is the sum of the variances. The third cumulant reflects the minus sign, whereas the fourth does not.

The contrast need not itself have a Poisson distribution. The calculation uses the Poisson cumulants of its independent inputs, not a claim that their difference is Poisson.

Worked Example: A Weighted Sum of Exponential Variables

Let $X_1$ and $X_2$ be independent exponential random variables with rates $1$ and $2$, respectively, and set $W=2X_1-3X_2$. For an exponential variable with rate $\lambda$, $K(t)=-\log(1-t/\lambda)$ for $t<\lambda$, and hence $\kappa_r=(r-1)!/\lambda^r$.

The first variable has $\kappa_r(X_1)=(r-1)!$; the second has $\kappa_r(X_2)=(r-1)!/2^r$. Therefore $$ \kappa_r(W) =2^r(r-1)!+(-3)^r\frac{(r-1)!}{2^r} =(r-1)!\left(2^r+\frac{(-3)^r}{2^r}\right). $$

For the first four orders, the calculations are $$ \begin{aligned} \kappa_1(W)&=2-\frac{3}{2}=\frac{1}{2},\\ \kappa_2(W)&=4+\frac{9}{4}=\frac{25}{4},\\ \kappa_3(W)&=2\left(8-\frac{27}{8}\right)=\frac{37}{4},\\ \kappa_4(W)&=6\left(16+\frac{81}{16}\right)=\frac{1011}{8}. \end{aligned} $$ The variance is positive because both independent inputs contribute nonnegative terms at order two. The negative coefficient affects the odd cumulants but not the even ones.

Worked Example: Two Outputs With a Shared Input

Let $X_1,X_2,X_3$ be independent Poisson variables with parameters $2,3,5$, respectively, and define $A=X_1+X_2$ and $B=X_2-X_3$. The input $X_2$ appears in both outputs, with coefficient $1$ in each. For any real $c,d$, $$ cA+dB=cX_1+(c+d)X_2-dX_3. $$

Each input’s cumulants all equal its Poisson parameter. Applying the theorem gives, for every integer $r\geq1$, $$ \kappa_r(cA+dB)=2c^r+3(c+d)^r+5(-d)^r. $$ For instance, taking $c=d=1$ gives $A+B=X_1+2X_2-X_3$, so $$ \kappa_1(A+B)=2+3(2)-5=3,\qquad \kappa_2(A+B)=2+3(4)+5=19, $$ and $$ \kappa_3(A+B)=2+3(8)-5=21,\qquad \kappa_4(A+B)=2+3(16)+5=55. $$ The coefficient of $X_2$ is $2$, not two separate independent copies with coefficients $1$. Its contributions are $3(2)^r$ at order $r$.

The shared input also makes $A$ and $B$ dependent. Their covariance can be found directly by expanding and using independence: all covariances between distinct $X_i$ vanish, leaving $$ \operatorname{Cov}(A,B) =\operatorname{Cov}(X_1+X_2,X_2-X_3) =\operatorname{Var}(X_2)=3. $$ Thus independent inputs do not imply that linear combinations of those inputs are independent.

Covariance and the Role of Shared Inputs

For finite-variance independent inputs, the same coefficient bookkeeping gives a compact covariance formula. If $Y_j=\sum_i a_{ji}X_i$ and $Y_k=\sum_i a_{ki}X_i$, then bilinearity of covariance yields $$ \operatorname{Cov}(Y_j,Y_k) =\sum_{i=1}^n a_{ji}a_{ki}\operatorname{Var}(X_i). $$ Indeed, expanding the covariance produces terms $a_{ji}a_{k\ell}\operatorname{Cov}(X_i,X_\ell)$; independence makes these zero when $i\ne\ell$, leaving the displayed sum. An input contributes to this covariance only when both combinations include it with nonzero coefficients.

This order-two calculation is a useful check on cumulant work: setting $r=2$ in the scalar combination formula gives the variance, and the covariance formula shows how overlap between two outputs matters. Higher-order mixed cumulants require a separate joint formulation; here the result concerns cumulants of scalar combinations.

Common Pitfall: Adding Output Cumulants as Though Outputs Were Independent

A tempting but generally incorrect argument is: “$A$ uses $X_1,X_2$, $B$ uses $X_2,X_3$, so the cumulant of $A+B$ is the sum of the cumulants of $A$ and $B$.” Additivity applies to independent summands, but $A$ and $B$ share $X_2$ and need not be independent. In the Poisson example, $\kappa_2(A)=5$ and $\kappa_2(B)=8$, so adding them gives $13$, not $\operatorname{Var}(A+B)=19$.

The correction is to rewrite the target as a linear combination of the original independent inputs before applying additivity. Here $A+B=X_1+2X_2-X_3$, whose variance is $2+4(3)+5=19$. The same rule works at every cumulant order: combine coefficients belonging to the same input first, then apply the input-wise formula.

Key takeaway: For independent inputs, the $r$th cumulant of a linear combination is the sum of each input’s $r$th cumulant multiplied by its coefficient to the $r$th power. When outputs share inputs, combine their coefficients input by input before calculating a cumulant of their sum.

Check Your Understanding

Use the independent-input formulas to answer each question.

If $X_1,X_2$ are independent with finite moment-generating functions near zero, what is $\kappa_r(3X_1-2X_2)$ in terms of their cumulants?
Why does a negative coefficient change the sign of odd-order cumulant contributions but not even-order contributions?
Suppose $A=X_1+X_2$ and $B=X_2-X_3$, with independent inputs. What coefficient of $X_2$ appears in $A+B$?
For independent Poisson inputs with parameters $2$ and $7$, find the first and second cumulants of $X_1-X_2$.
What additional condition on the inputs is needed to use the covariance formula for two linear combinations?