Generating Functions and Transforms of Mixtures

Why Transforms of Mixtures Are Useful

A mixture is generated by selecting a component and then drawing an observation from that component’s distribution. That two-stage description makes transforms especially convenient: average each component’s transform using the mixing weights, and the result is the transform of the mixture. This gives a direct route to calculations without first deriving the mixture’s density or mass function.

The transform depends on the kind of random variable. A probability-generating function (PGF) applies to nonnegative integer-valued variables; a moment-generating function (MGF) applies where the relevant exponential expectation is finite; and a characteristic function exists for every real-valued random variable. The distinction matters because their existence and differentiation properties are not identical.

Mixture Setup and Transform Identities

For a continuous mixture, replace the finite label by a mixing parameter \(\theta\) with probability distribution \(G\), and let the conditional distribution of \(X\) given \(\theta\) be \(F_\theta\). Provided the conditional expectations are measurable, the corresponding identity is

For nonnegative \(h\), this identity holds even if either side is infinite, by Tonelli’s theorem. For signed \(h\), it holds when \(E[|h(X)|]<\infty\). These conditions clarify when an expectation can be averaged across components.

For a continuous mixture, each finite sum is replaced by an integral over \(G\). The characteristic-function identity holds for every real \(t\). The PGF identity holds for \(0\leq s\leq1\) when the components are nonnegative integer-valued. The MGF identity holds wherever its expectation is finite; for a finite mixture, this is equivalent to finiteness of the MGF for every component having positive weight.

Proof. For the characteristic function, take \(h(x)=e^{itx}\) in the mixture expectation identity. This function is bounded in absolute value by 1, so its expectation exists for every real \(t\). Thus

For a PGF with \(0\leq s\leq1\), \(s^X\) is bounded between 0 and 1 (with the usual convention \(0^0=1\)), so the same conditional-expectation argument applies. For the MGF, \(e^{tX}\) is nonnegative, so the identity follows from the nonnegative-function version of the mixture formula, including when an expectation is infinite. In a finite mixture, a positive-weight infinite component expectation makes the weighted sum infinite; otherwise all terms are finite and the identity gives the finite MGF. This proves the claims. \(\square\)

Using Transform Derivatives to Find Moments

Once a mixture transform has been found, moments can often be obtained by differentiation. For an MGF, if \(E[e^{a|X|}]<\infty\) for some \(a>0\), then \(M_X\) is differentiable to every order in a neighborhood of zero and

The exponential-moment condition justifies differentiating inside the expectation. In fact, for any \(b<a\), derivatives near zero are dominated by a constant multiple of \(e^{a|X|}\), since polynomial factors in \(|X|\) are bounded by a constant times \(e^{(a-b)|X|}\). Dominated convergence therefore applies. For a mixture, the same derivatives can be taken term by term when a common exponential-moment bound holds for the positive-weight components; the resulting moments are the weighted averages of the component raw moments.

For a nonnegative integer-valued \(X\), the PGF provides factorial moments: when the relevant moments are finite,

More generally, the \(r\)th derivative at 1 gives \(E[X(X-1)\cdots(X-r+1)]\). One way to justify the first two identities is to differentiate the series \(G_X(s)=\sum_{k=0}^{\infty}P(X=k)s^k\) for \(0\leq s<1\), then let \(s\) increase to 1. The resulting nonnegative terms increase to \(kP(X=k)\), or to \(k(k-1)P(X=k)\); monotone convergence gives the stated expectations, finite or infinite. If finite, the ordinary second moment follows from \(X^2=X(X-1)+X\).

Common Mistake: Averaging Component Parameters

A common error is to average a component parameter first and then substitute that average into a transform. For example, in the Poisson mixture above, the weighted average of the component means is 3. It would be incorrect to conclude that the mixture is \(\operatorname{Poisson}(3)\), whose PGF is \(\exp(3(s-1))\). The correct PGF is the weighted sum \(\frac13e^{s-1}+\frac23e^{4(s-1)}\), and these functions are not equal in general. At \(s=0\), for instance, the mixture gives \(\frac13e^{-1}+\frac23e^{-4}\), whereas a \(\operatorname{Poisson}(3)\) variable gives \(e^{-3}\). Matching a mean does not make the distributions, or their transforms, the same.

The same correction applies to all three transforms: average the component transforms with the mixture weights. Do not average the component transforms’ logarithms, and do not assume that a transform exists merely because each component formula looks familiar. For an MGF, check the common domain on which the mixture expectation is finite; for a characteristic function, no such existence restriction is needed.