Multivariate MGFs and Mixed Moments

Why Mixed Moments Need a Joint MGF

For a random vector, moments describe not only the behavior of each coordinate but also how coordinates vary together. For example, \(E[XY]\) is a joint, or mixed, moment: it involves both random variables at once. The joint moment-generating function packages these quantities into a function of several arguments, with a separate argument for each coordinate.

The key condition is that the MGF be finite in a neighborhood of the origin. Under that condition, its partial derivatives can be brought inside the expectation, and each derivative contributes a factor from the corresponding coordinate. This makes the derivative order and the exponents in the mixed moment match exactly.

Definition and Mixed-Moment Formula

For a multi-index \(\alpha=(\alpha_1,\ldots,\alpha_d)\) of nonnegative integers, write \(|\alpha|=\sum_{j=1}^d\alpha_j\), and use \(\partial^\alpha=\partial_{t_1}^{\alpha_1}\cdots\partial_{t_d}^{\alpha_d}\). The matching mixed moment is \(E[\prod_{j=1}^d X_j^{\alpha_j}]\). An exponent of zero contributes a factor of one.

Proof. Choose \(b>0\) so that the closed cube \([-b,b]^d\) is inside a neighborhood on which \(M_X\) is finite. For each possible sign vector \(\varepsilon\in\{-1,1\}^d\), the value \(M_X(b\varepsilon)\) is finite. Since \[ \exp\!\left(b\sum_{j=1}^d|X_j|\right) =\max_{\varepsilon\in\{-1,1\}^d}\exp(b\varepsilon^{\mathsf T}X) \leq \sum_{\varepsilon\in\{-1,1\}^d}\exp(b\varepsilon^{\mathsf T}X), \] taking expectations shows that \(E[\exp(b\sum_j|X_j|)]<\infty\).

Fix \(a\) with \(0<a<b\). For \(\|t\|_\infty\leq a\), where \(\|t\|_\infty=\max_j|t_j|\), the absolute value of the formal derivative integrand is bounded by \[ \left|\prod_{j=1}^d X_j^{\alpha_j}\exp(t^{\mathsf T}X)\right| \leq \left(\sum_{j=1}^d|X_j|\right)^{|\alpha|} \exp\!\left(a\sum_{j=1}^d|X_j|\right). \] The right side is at most \(C\exp(b\sum_j|X_j|)\) for a finite constant \(C\): the function \(u^{|\alpha|}e^{-(b-a)u}\) is bounded for \(u\geq0\). This bound is integrable by the preceding paragraph. In particular it proves finiteness of the mixed absolute moments.

To justify differentiation, take a smaller cube \(\|t\|_\infty\leq a\) and choose \(\delta>0\) with \(a+\delta<b\). A difference quotient for one more derivative in coordinate \(j\) introduces the factor \((e^{hX_j}-1)/h\). For \(0<|h|\leq\delta\), the mean value theorem gives \[ \left|\frac{e^{hX_j}-1}{h}\right| \leq |X_j|e^{|h||X_j|}. \] Thus the difference quotient is dominated by a polynomial in \(\sum_j|X_j|\) times \(\exp((a+\delta)\sum_j|X_j|)\), which is in turn bounded by a constant times \(\exp(b\sum_j|X_j|)\). Dominated convergence permits the derivative to pass inside the expectation. Repeating this argument for each coordinate and each finite derivative order gives \[ \partial^\alpha M_X(t) =E\!\left[\prod_{j=1}^d X_j^{\alpha_j}\exp(t^{\mathsf T}X)\right] \] on a neighborhood of the origin. Setting \(t=0\) proves the formula. The same integrable bounds and dominated convergence give continuity of these derivatives. \(\square\)

Calculating Mixed Moments

Marginals and Linear Transformations

Proof. By the definition of \(Y\), \[ M_Y(u)=E[\exp(u^{\mathsf T}AX)] =E[\exp((A^{\mathsf T}u)^{\mathsf T}X)] =M_X(A^{\mathsf T}u). \] Because the linear map \(u\mapsto A^{\mathsf T}u\) is continuous and sends \(0\) to \(0\), all sufficiently small \(u\) map into a neighborhood on which \(M_X\) is finite. Thus the displayed identity holds with finite values near the origin. Taking \(A\) to select a single coordinate gives the marginal statement. \(\square\)

This identity makes the joint MGF useful for studying sums and other linear combinations of coordinates. For example, if \(Y=X_1+\cdots+X_d\), then \(M_Y(u)=M_X(u,\ldots,u)\) wherever the right side is finite. This substitution gives the MGF of the sum; it does not imply that the coordinates are independent.

Interpretation and a Common Pitfall

Mixed derivatives encode joint products. For instance, if \(X=(X_1,X_2,X_3)\), then \(\partial_{t_1}^2\partial_{t_3}M_X(0)\) is \(E[X_1^2X_3]\), while \(\partial_{t_1}\partial_{t_2}M_X(0)\) is \(E[X_1X_2]\). Repeated differentiation with respect to one coordinate creates a repeated power of that coordinate.

• Incorrect: Replace \(E[X_1X_2]\) by \(E[X_1]E[X_2]\) without establishing independence.
• Correction: The mixed derivative gives \(E[X_1X_2]\). Independence is a separate condition under which the expectation factors into marginal expectations.
• Incorrect: Differentiate a joint MGF at the origin merely because the formula is finite at the origin.
• Correction: Finiteness at the origin alone is automatic, since \(M_X(0)=1\). The differentiation result requires finiteness in a neighborhood of the origin.

The neighborhood condition is substantive: it supplies exponential integrability that dominates the polynomial factors created by differentiation. When it holds, the joint MGF provides a systematic way to compute mixed moments and to check the dependence structure represented by them.