Joint Cumulants and Independence

Why Joint Cumulants Detect Independence

Covariance records one kind of association between two variables, but it cannot by itself settle whether they are independent. Joint cumulants provide a stronger diagnostic: independence forces every cumulant that mixes the two groups to be zero. Under a local moment-generating-function condition, the reverse is true as well: if all such mixed cumulants vanish, then the groups are independent.

The key is to consider all orders together. A zero mixed cumulant of order two is just zero covariance; higher orders can reveal dependence that covariance misses. The moment-generating-function condition makes the converse possible because it lets us recover the joint cumulant-generating function from its derivatives near the origin.

Joint Cumulants and Mixed Orders

Definition: Let \(Z=(Z_1,\ldots,Z_d)\) be a random vector whose moment-generating function \(M_Z(t)=E[\exp(\sum_{j=1}^d t_j Z_j)]\) is finite in an open neighborhood of \(0\in\mathbb{R}^d\). Its joint cumulant-generating function is \(K_Z(t)=\log M_Z(t)\) in a sufficiently small neighborhood of zero. For indices \(i_1,\ldots,i_r\in\{1,\ldots,d\}\), the joint cumulant is \(\kappa(Z_{i_1},\ldots,Z_{i_r})= \left.\frac{\partial^r K_Z(t)}{\partial t_{i_1}\cdots\partial t_{i_r}}\right|_{t=0}\). Indices may repeat.

This definition extends ordinary cumulants to collections of variables. In particular, \(\kappa(X,Y)=\operatorname{Cov}(X,Y)\). A cumulant such as \(\kappa(X,X,Y)\) is a third-order joint cumulant: it involves two occurrences of \(X\) and one of \(Y\).

Now divide the coordinates into two groups, \(U=(X_1,\ldots,X_p)\) and \(V=(Y_1,\ldots,Y_q)\). A joint cumulant is mixed across the groups if its list of arguments contains at least one coordinate from \(U\) and at least one from \(V\). For example, \(\kappa(X_1,X_1,Y_2)\) is mixed, while \(\kappa(X_1,X_1)\) is not.

We assume local existence of the joint moment-generating function for the combined vector \((U,V)\): it is finite on some open neighborhood of the origin in \(\mathbb{R}^{p+q}\). This ensures that the joint cumulant-generating function is well defined and analytic near zero, which will be essential for the converse.

Independence Makes Mixed Cumulants Vanish

Theorem: Suppose the joint moment-generating function of \((U,V)\) is finite in a neighborhood of the origin. If the random vectors \(U\) and \(V\) are independent, then every joint cumulant involving at least one coordinate from each vector is zero.

Independence here means that the random vectors are independent, or equivalently that the sigma-fields they generate are independent. The statement concerns all mixed cumulants, at every order, including cumulants with repeated coordinates.

Proof. Independence gives, for \(t\in\mathbb{R}^p\) and \(s\in\mathbb{R}^q\) sufficiently close to zero,

M_{(U,V)}(t,s)=E[e^{t^\mathsf{T}U+s^\mathsf{T}V}] =E[e^{t^\mathsf{T}U}]E[e^{s^\mathsf{T}V}] =M_U(t)M_V(s).

The moment-generating functions are positive wherever finite, so taking logarithms yields

K_{(U,V)}(t,s)=K_U(t)+K_V(s).

Every term on the right depends on only one group of coordinates. Thus any derivative that differentiates at least once with respect to a coordinate of \(t\) and at least once with respect to a coordinate of \(s\) is zero. Evaluating such derivatives at the origin gives every mixed joint cumulant, so each is zero. \(\square\)

Worked Example: Independent Bernoulli Variables

Let \(X\sim\operatorname{Bernoulli}(p)\) and \(Y\sim\operatorname{Bernoulli}(q)\) be independent, where \(0<p,q<1\). Their joint moment-generating function factors:

M_{(X,Y)}(t,s) =(1-p+pe^t)(1-q+qe^s).

Therefore the joint cumulant-generating function is

K_{(X,Y)}(t,s) =\log(1-p+pe^t)+\log(1-q+qe^s).

The first summand depends only on \(t\), and the second only on \(s\). For example, \(\kappa(X,Y)=\left.\frac{\partial^2K}{\partial t\,\partial s}\right|_{(0,0)}=0\), and \(\kappa(X,X,Y)=\left.\frac{\partial^3K}{\partial t^2\,\partial s}\right|_{(0,0)}=0\). The same separation shows that every mixed derivative of every order is zero. The variables are bounded, so their joint moment-generating function is finite everywhere; in particular, the local existence condition holds.

When Vanishing Mixed Cumulants Imply Independence

Theorem: Let \(U\in\mathbb{R}^p\) and \(V\in\mathbb{R}^q\) be random vectors whose joint moment-generating function is finite in an open neighborhood of the origin. If every joint cumulant involving at least one coordinate from \(U\) and at least one coordinate from \(V\) is zero, then \(U\) and \(V\) are independent.

The hypothesis includes mixed cumulants of every order, not just covariances or cumulants up to some fixed order. The local moment-generating-function condition ensures analyticity, so these derivatives determine the cumulant-generating function near zero.

Proof. Write \(K(t,s)=\log M_{(U,V)}(t,s)\). Because the joint moment-generating function is finite on an open neighborhood of the origin, \(K\) is analytic there. Choose a product of open boxes around the origin contained in that neighborhood. Its Taylor series has coefficients given by the joint cumulants, with the corresponding factorial denominators.

By assumption, every Taylor coefficient involving a positive power of at least one \(t\)-coordinate and a positive power of at least one \(s\)-coordinate is zero. The remaining terms involve only \(t\), only \(s\), or neither. Since \(K(0,0)=\log M_{(U,V)}(0,0)=0\), the Taylor series therefore gives, throughout a sufficiently small product box,

$$ K(t,s)=K(t,0)+K(0,s). $$

Exponentiating both sides shows that the joint moment-generating function factors there:

M_{(U,V)}(t,s)=M_U(t)M_V(s).

The product \(M_U(t)M_V(s)\) is the moment-generating function of the product of the marginal distributions of \(U\) and \(V\). By the uniqueness theorem for moment-generating functions, two probability distributions whose moment-generating functions are finite on a neighborhood of the origin and agree there must be the same distribution. Hence the joint distribution of \((U,V)\) is the product of its marginals, which is precisely independence. \(\square\)

The theorem is an all-orders characterization under its stated condition. It does not say that a finite list of zero mixed cumulants establishes independence; the Taylor-series argument requires every mixed coefficient to vanish.

Worked Example: Zero Covariance but a Nonzero Mixed Cumulant

Let \(X\) take the values \(-1,0,1\), each with probability \(1/3\), and set \(Y=X^2\). Both variables are bounded, so their joint moment-generating function exists everywhere. First calculate

E[X]=0,\qquad E[Y]=E[X^2]=\frac23,\qquad E[XY]=E[X^3]=0.

Thus \(\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]=0\). But \(X\) and \(Y\) are not independent: \(Y\) is determined by \(X\), and \(P(Y=1\mid X=0)=0\), whereas \(P(Y=1)=2/3\).

The third-order joint cumulant is nonzero. For these variables, the third cumulant with two copies of \(X\) and one of \(Y\) is

\begin{aligned} \kappa(X,X,Y) &=E[(X-E[X])^2(Y-E[Y])]\\ &=E[X^2Y]-E[X^2]E[Y]\\ &=E[X^4]-\left(\frac23\right)^2\\ &=\frac23-\frac49=\frac29. \end{aligned}

So covariance alone misses this dependence, while a higher-order mixed cumulant detects it. The example also illustrates why the converse theorem requires vanishing mixed cumulants at every order.

Worked Example: Independent Poisson Variables

Let \(X\sim\operatorname{Poisson}(\lambda)\) and \(Y\sim\operatorname{Poisson}(\mu)\) be independent, with \(\lambda,\mu>0\). A Poisson variable with mean \(\lambda\) has moment-generating function \(\exp\{\lambda(e^t-1)\}\). Independence therefore gives

K_{(X,Y)}(t,s)=\lambda(e^t-1)+\mu(e^s-1).

The function separates into a term in \(t\) and a term in \(s\). For instance, \(\kappa(X,Y)=\left.\frac{\partial^2K}{\partial t\,\partial s}\right|_{(0,0)}=0\), and \(\kappa(X,Y,Y)=\left.\frac{\partial^3K}{\partial t\,\partial s^2}\right|_{(0,0)}=0\). In fact every mixed derivative vanishes. The Poisson moment-generating functions are finite for all real arguments, so both the local existence condition and the independence theorem apply.

Interpreting the Criterion Carefully

Joint cumulants are useful because the two directions fit together: independence implies that the joint cumulant-generating function separates, and separation forces all mixed derivatives to vanish. Conversely, local analyticity means a complete collection of vanishing mixed derivatives forces that same separation near the origin. The uniqueness of moment-generating functions then translates local separation into independence of the distributions.

Zero covariance is not enough. It says only that the mixed cumulant of order two vanishes. The bounded example above has zero covariance but a nonzero third-order mixed cumulant.
Vanishing through a fixed order is not the theorem. Independence requires all mixed cumulants to vanish under this characterization. Checking a finite number of orders cannot use the full Taylor-series argument.
The local existence condition matters for the converse. It supplies an analytic cumulant-generating function near the origin. Without such a condition, the derivatives at the origin need not determine the function and distribution in the way used in the proof.

A common mistake is to reason, “the covariance is zero, so the variables are independent.” The correction is to separate the claims: independence guarantees zero covariance when the relevant moments exist, but zero covariance alone does not guarantee independence. To use joint cumulants as an independence criterion here, require all mixed cumulants to vanish and verify local joint moment-generating-function existence.

Key takeaway: For random vectors with a joint moment-generating function finite near the origin, independence is equivalent to the vanishing of every joint cumulant that mixes coordinates from the two groups.

Check Your Understanding

Use the definitions and results above to answer these questions.

What makes a joint cumulant mixed across two groups of coordinates? Give one example using repeated indices.
Why does independence make every mixed derivative of the joint cumulant-generating function zero?
Which condition allows the converse proof to recover the joint cumulant-generating function from its Taylor coefficients?
In the example \(Y=X^2\) with \(X\) uniform on \(\{-1,0,1\}\), what are \(\operatorname{Cov}(X,Y)\) and \(\kappa(X,X,Y)\)?
Why does checking that all mixed cumulants through order four vanish not, by itself, establish independence under the theorem?