Joint Cumulant-Generating Functions

From a Vector’s MGF to Its Joint Cumulants

For a random vector, the moment-generating function depends on a vector of arguments rather than a single scalar. Its logarithm records not only the cumulants of each coordinate but also how coordinates vary together. The partial derivatives of this logarithm at the origin are the mixed cumulants.

The key condition is local: the moment-generating function must be finite in an open neighborhood of the origin. This condition ensures that the derivatives used to define cumulants exist. Finiteness at the origin alone is automatic, and existence of every polynomial moment alone is not enough.

Definition: Let $X=(X_1,\ldots,X_d)$ be a random vector. Its joint moment-generating function is $$ M_X(t)=E\!\left[\exp\!\left(\sum_{j=1}^d t_jX_j\right)\right], $$ for those $t=(t_1,\ldots,t_d)\in\mathbb{R}^d$ for which the expectation is finite. If $M_X(t)$ is finite on an open neighborhood of $0$, the joint cumulant-generating function on that neighborhood is $$ K_X(t)=\log M_X(t). $$

For nonnegative integers $\alpha_1,\ldots,\alpha_d$, write $\partial^\alpha$ for the partial derivative that differentiates $\alpha_j$ times with respect to $t_j$. The multi-index $\alpha=(\alpha_1,\ldots,\alpha_d)$ records how often each coordinate appears.

Definition: If $M_X$ is finite in an open neighborhood of $0$, the joint cumulant with $\alpha_j$ copies of $X_j$ is $$ \kappa_{\alpha}(X)= \left. \frac{\partial^{\alpha_1+\cdots+\alpha_d}K_X(t)} {\partial t_1^{\alpha_1}\cdots\partial t_d^{\alpha_d}} \right|_{t=0}, $$ for $\alpha_1+\cdots+\alpha_d\geq 1$. Equivalently, a mixed cumulant with specified coordinates $X_{i_1},\ldots,X_{i_r}$, where indices may repeat, is $\left.\partial_{i_1}\cdots\partial_{i_r}K_X(t)\right|_{t=0}$.

The coordinates need not be distinct. For example, $\kappa(X_1,X_1,X_2)$ differentiates twice with respect to $t_1$ and once with respect to $t_2$. This repeated-coordinate notation is useful even when the random vector has only a few components.

Why Local Existence Is Enough

The neighborhood condition is not merely a technical convention. It guarantees that derivatives can be brought inside the expectation and that the logarithm is differentiable near the origin. The justification uses an integrable bound.

Theorem: If $M_X(t)$ is finite on an open neighborhood of $0\in\mathbb{R}^d$, then $M_X$ has partial derivatives of every finite order near the origin, obtained by differentiating its integrand. The function $K_X(t)=\log M_X(t)$ also has derivatives of every finite order there. In particular, its mixed derivatives at the origin define the joint cumulants.

Proof. Choose $a>0$ small enough that the closed cube $[-a,a]^d$ lies inside the neighborhood where $M_X$ is finite. For any vector $x\in\mathbb{R}^d$, $$ \exp\!\left(a\sum_{j=1}^d |x_j|\right) \leq \sum_{\varepsilon\in\{-1,1\}^d} \exp\!\left(a\sum_{j=1}^d \varepsilon_jx_j\right). $$ Taking expectations shows that $E[\exp(a\sum_j|X_j|)]<\infty$, since the sum on the right has finitely many terms, each with finite expectation.

For any fixed derivative order, the corresponding derivative of $\exp(t\cdot X)$ is a product of powers of coordinates of $X$ times $\exp(t\cdot X)$. When $|t_j|\leq a/2$, its absolute value is bounded by $$ \left(\prod_{j=1}^d |X_j|^{m_j}\right) \exp\!\left(\frac{a}{2}\sum_{j=1}^d |X_j|\right) \leq C\exp\!\left(a\sum_{j=1}^d |X_j|\right), $$ for a finite constant $C$ depending on the derivative orders and $a$. The inequality follows because any fixed power of a nonnegative number is bounded by a constant times an exponential with positive rate. The right-hand side is integrable by the preceding bound. Dominated convergence therefore justifies differentiating under the expectation, at each finite order. Also $M_X(t)>0$ for real $t$ in this neighborhood, because it is the expectation of a strictly positive random variable. Thus $\log M_X(t)$ is differentiable there as well. $\square$

The first two derivatives recover familiar quantities. Differentiating once gives $\partial_j M_X(t)=E[X_j\exp(t\cdot X)]$. At $t=0$, $M_X(0)=1$, so $\partial_jK_X(0)=E[X_j]$. Differentiating the logarithm twice gives $$ \partial_i\partial_jK_X(t) = \frac{\partial_i\partial_jM_X(t)}{M_X(t)} - \frac{\partial_iM_X(t)\,\partial_jM_X(t)}{M_X(t)^2}. $$ At the origin this becomes $\kappa(X_i,X_j)=E[X_iX_j]-E[X_i]E[X_j]=\operatorname{Cov}(X_i,X_j)$. Thus first-order cumulants are means, and second-order mixed cumulants are covariances.

Connection to the Joint-Moment Formula

The derivative definition also gives the standard formula for a mixed cumulant in terms of joint moments. Given $X_{i_1},\ldots,X_{i_r}$, let $\Pi_r$ be the set of partitions of $\{1,\ldots,r\}$. Then $$ \kappa(X_{i_1},\ldots,X_{i_r}) = \sum_{\pi\in\Pi_r} (|\pi|-1)!(-1)^{|\pi|-1} \prod_{B\in\pi} E\!\left[\prod_{q\in B}X_{i_q}\right]. $$

To see why, introduce auxiliary variables $z_1,\ldots,z_r$ and consider the cumulant-generating function of the scalar sum $\sum_{q=1}^r z_qX_{i_q}$. Its moment-generating function is $$ 1+u(z), \qquad u(z)=E\!\left[\exp\!\left(\sum_{q=1}^r z_qX_{i_q}\right)\right]-1. $$ The local existence condition ensures the expansions around zero are valid. In $\log(1+u)=\sum_{k\geq1}(-1)^{k-1}u^k/k$, the coefficient of $z_1\cdots z_r$ in $u^k$ comes from assigning the $r$ distinct labels to $k$ nonempty groups. Each such assignment contributes the product of the corresponding joint moments. For a fixed partition into $k$ blocks, there are $k!$ ways to order those blocks; division by $k$ leaves $(k-1)!$. The sign is $(-1)^{k-1}$. The coefficient of $z_1\cdots z_r$ in the logarithm is the mixed derivative at zero, giving the formula.

Worked Example: A Dependent Bernoulli Pair

Let $(X,Y)$ take the values $(0,0),(1,0),(0,1),(1,1)$ with probabilities $0.4,0.2,0.2,0.2$, respectively. Find its joint cumulant-generating function and its first and second order cumulants.

Because the vector has finite support, its joint MGF is finite for every real pair $(s,t)$. Summing the four possible contributions gives $$ M_{(X,Y)}(s,t)=0.4+0.2e^s+0.2e^t+0.2e^{s+t}, \qquad K_{(X,Y)}(s,t)=\log\!\left(0.4+0.2e^s+0.2e^t+0.2e^{s+t}\right). $$

At the origin, $M(0,0)=1$. The first derivatives are $M_s(0,0)=0.4$ and $M_t(0,0)=0.4$. Therefore $E[X]=K_s(0,0)=0.4$ and $E[Y]=K_t(0,0)=0.4$. Also $M_{st}(0,0)=0.2$, since only the $(1,1)$ outcome contributes to $E[XY]$. Hence $$ \kappa(X,Y)=K_{st}(0,0)=0.2-(0.4)(0.4)=0.04. $$ The mixed second cumulant is the covariance; here it is positive, consistent with the probability $P(X=1,Y=1)=0.2$ exceeding $P(X=1)P(Y=1)=0.16$.

Worked Joint CGFs in Other Models

Worked Example: A Bivariate Normal Vector

Let $Z\sim N(0,1)$ and $W\sim N(0,4)$ be independent, and define $X=Z$ and $Y=Z+W$. Find the joint CGF and the means and covariance.

For any real $s,t$, $sX+tY=(s+t)Z+tW$. Using the normal MGF for each independent term, $$ M_{(X,Y)}(s,t) =E[e^{(s+t)Z}]E[e^{tW}] =\exp\!\left(\frac{(s+t)^2}{2}\right)\exp(2t^2). $$ Thus the CGF, finite on all of $\mathbb{R}^2$, is $$ K_{(X,Y)}(s,t)=\frac{(s+t)^2}{2}+2t^2 =\frac{s^2}{2}+st+\frac{5t^2}{2}. $$

The first derivatives at zero give $E[X]=K_s(0,0)=0$ and $E[Y]=K_t(0,0)=0$. The mixed second derivative is $K_{st}(s,t)=1$, so $\operatorname{Cov}(X,Y)=1$. The other second derivatives are $K_{ss}=1$ and $K_{tt}=5$, giving $\operatorname{Var}(X)=1$ and $\operatorname{Var}(Y)=5$. Since this CGF is a polynomial of degree two, all its derivatives of order three or higher are zero.

Worked Example: A Shared Poisson Component

Let $N\sim\operatorname{Poisson}(3)$ and $Q\sim\operatorname{Poisson}(2)$ be independent, and set $X=N$, $Y=N+Q$. Find the joint CGF, the means, and the mixed second cumulant.

The joint exponential is $e^{sX+tY}=e^{(s+t)N+tQ}$. The Poisson CGF with rate $\lambda$ is $\lambda(e^u-1)$, so the joint CGF is $$ K_{(X,Y)}(s,t)=3(e^{s+t}-1)+2(e^t-1). $$ It is finite for all real $s,t$. Differentiating at zero gives $$ E[X]=K_s(0,0)=3,\qquad E[Y]=K_t(0,0)=3+2=5. $$ A derivative with respect to both arguments yields $K_{st}(s,t)=3e^{s+t}$, and therefore $$ \kappa(X,Y)=K_{st}(0,0)=3. $$ The shared component $N$ contributes to both coordinates; the independent component $Q$ contributes only to $Y$. The derivative calculation makes the resulting cross-cumulant explicit.

Common Pitfall and Practical Use

A common mistake is to assume that the existence of all moments guarantees a CGF and hence cumulants through derivatives of its logarithm. It does not. A random variable can have finite moments of every order while its MGF is infinite for every positive argument. In that case the derivatives of $\log M$ cannot be justified by a neighborhood-of-zero MGF, even though moments themselves exist. The correction is to check the local MGF condition before using this derivative definition; other ways of defining cumulants require their own conditions and are not interchangeable without justification.

Once local existence is established, the joint CGF provides a compact way to organize coordinate-level and cross-coordinate information. Its first derivatives identify means, its second derivatives identify covariances, and higher mixed derivatives define higher-order joint cumulants. In calculations, the argument $t$ has one coordinate for each component of the random vector, while the derivative indices specify which components enter the cumulant and how often.

Check Your Understanding

Use the definitions and derivations above to answer these questions.

For a random vector with two coordinates, write the partial derivative that defines $\kappa(X_1,X_1,X_2)$.
Why does finiteness of the joint MGF on an open neighborhood of the origin permit differentiation under the expectation?
What does the first derivative $K_{t_j}(0)$ equal? What does $K_{t_i t_j}(0)$ equal?
In the shared Poisson example, which component accounts for the mixed second cumulant, and what is its value?
Why is existence of all moments by itself not the stated condition for defining cumulants as derivatives of the log MGF?