Why Mixture Cumulants Need More Than Averaging
A population can contain several subpopulations, each with its own distribution. If a measurement is generated by first selecting a subpopulation and then sampling from it, the result has a finite mixture distribution. Its overall mean is a weighted average of component means. Its variance and third cumulant, however, also reflect how far the component means lie from the overall mean. These between-component contributions can be substantial even when the individual components have little variation or skewness.
We derive the first three cumulants by conditioning on the component label. The key calculation is to express the deviation from the mixture mean as a within-component deviation plus a shift in the component’s mean.
Finite Mixtures and Cumulants Through Order Three
The finite absolute third-moment condition ensures that the expectations used below exist. The third cumulant is the third central moment; unlike the variance, it can be positive, negative, or zero. When a cumulant-generating function exists near zero, these definitions agree with its first, second, and third derivatives at zero.
Proof. The events \(\{J=j\}\) partition the sample space. Applying the definition of conditional expectation to this finite partition gives \[ E[g(X)]=\sum_{j=1}^m E[g(X)\mathbf{1}_{\{J=j\}}] =\sum_{j=1}^m P(J=j)E[g(X)\mid J=j] =\sum_{j=1}^m\pi_j E_j[g(X)]. \] The sum is finite, so no interchange of an infinite sum and an integral is required. \(\square\)
The Mean, Variance, and Third Cumulant
Let \(\delta_j=\mu_j-\mu\) denote the displacement of component \(j\)'s mean from the mixture mean. Within that component, write \(X-\mu=(X-\mu_j)+\delta_j\). The first term has conditional mean zero; the second is the component-level shift. Expanding powers of this decomposition gives the mixture formulas.
By the finite-partition identity, \(E[X]=\sum_j\pi_jE_j[X]=\sum_j\pi_j\mu_j\).
For component \(j\), write \(X-\mu=(X-\mu_j)+\delta_j\), where \(E_j[X-\mu_j]=0\).
Take the conditional second and third moments of this expression, then weight them by \(\pi_j\) and sum.
Proof of the variance formula. The mixture mean has already been established. For each component, \[ E_j[(X-\mu)^2] =E_j[(X-\mu_j+\delta_j)^2] =\sigma_j^2+2\delta_jE_j[X-\mu_j]+\delta_j^2 =\sigma_j^2+\delta_j^2. \] Weight these conditional expectations and sum over the components: \[ \operatorname{Var}(X)=E[(X-\mu)^2] =\sum_{j=1}^m\pi_j E_j[(X-\mu)^2] =\sum_{j=1}^m\pi_j(\sigma_j^2+\delta_j^2). \] This is the stated formula because \(\delta_j=\mu_j-\mu\). \(\square\)
Proof of the third-cumulant formula. Expand the cube within component \(j\): \[ \begin{aligned} E_j[(X-\mu)^3] &=E_j[(X-\mu_j+\delta_j)^3]\\ &=E_j[(X-\mu_j)^3] +3\delta_jE_j[(X-\mu_j)^2] +3\delta_j^2E_j[X-\mu_j] +\delta_j^3\\ &=\kappa_{3,j}+3\delta_j\sigma_j^2+\delta_j^3. \end{aligned} \] The term containing \(E_j[X-\mu_j]\) vanishes because the component deviation is centered. Applying the finite-partition identity to \((X-\mu)^3\) gives \[ \kappa_3=E[(X-\mu)^3] =\sum_{j=1}^m\pi_j E_j[(X-\mu)^3] =\sum_{j=1}^m\pi_j\{\kappa_{3,j}+3\delta_j\sigma_j^2+\delta_j^3\}. \] Substituting \(\delta_j=\mu_j-\mu\) proves the result. \(\square\)
The variance formula has a within-component term, \(\sum_j\pi_j\sigma_j^2\), and a between-component term, \(\sum_j\pi_j(\mu_j-\mu)^2\). The third cumulant has three contributions: component third cumulants, a cross-term involving component variance and displacement, and a cubic displacement term. The cross-term matters because a component with substantial variance can contribute differently depending on whether its mean is above or below the mixture mean.
Worked Calculations
Worked Example: A Mixture of Two Normal Distributions
Suppose \(P(J=1)=0.7\) and \(P(J=2)=0.3\), with \(X\mid J=1\sim N(0,1)\) and \(X\mid J=2\sim N(4,4)\). Find the first three cumulants of the mixture. Here the second argument of \(N(\mu,\sigma^2)\) is the variance.
The component means are \(\mu_1=0,\mu_2=4\), and the component variances are \(\sigma_1^2=1,\sigma_2^2=4\). Both normal components have third cumulant zero. First, \[ \mu=0.7(0)+0.3(4)=1.2. \] The displacements are \(\delta_1=-1.2\) and \(\delta_2=2.8\). Thus \[ \begin{aligned} \kappa_2 &=0.7\{1+(-1.2)^2\}+0.3\{4+(2.8)^2\}\\ &=0.7(2.44)+0.3(11.84)\\ &=1.708+3.552=5.26. \end{aligned} \] For the third cumulant, include the variance-displacement and cubic-displacement terms, even though each component is symmetric: \[ \begin{aligned} \kappa_3 &=0.7\{3(-1.2)(1)+(-1.2)^3\} +0.3\{3(2.8)(4)+(2.8)^3\}\\ &=0.7(-3.6-1.728)+0.3(33.6+21.952)\\ &=0.7(-5.328)+0.3(55.552)\\ &=-3.7296+16.6656=12.936. \end{aligned} \] The mixture has positive third cumulant despite both components having third cumulant zero. Its asymmetry comes from the unequal weights, locations, and variances of the components.
Worked Example: A Mixture of Two Point Masses
Let \(X=0\) with probability \(0.75\) and \(X=3\) with probability \(0.25\). Regard each value as a component with zero within-component variance and zero within-component third cumulant.
The mean is \[ \mu=0.75(0)+0.25(3)=0.75. \] The component displacements are \(-0.75\) and \(2.25\). Since both component variances are zero, \[ \begin{aligned} \kappa_2 &=0.75(-0.75)^2+0.25(2.25)^2\\ &=0.75(0.5625)+0.25(5.0625)\\ &=0.421875+1.265625=1.6875. \end{aligned} \] Likewise, \[ \begin{aligned} \kappa_3 &=0.75(-0.75)^3+0.25(2.25)^3\\ &=0.75(-0.421875)+0.25(11.390625)\\ &=-0.31640625+2.84765625=2.53125. \end{aligned} \] All the variance and third-cumulant contributions here come from the component locations: within each component, \(X\) is constant.
Worked Example: Mixing Two Bernoulli Components
Choose component 1 with probability \(0.75\), and let \(X\mid J=1\sim\operatorname{Bernoulli}(0.2)\). Choose component 2 with probability \(0.25\), and let \(X\mid J=2\sim\operatorname{Bernoulli}(0.8)\). A Bernoulli distribution with success probability \(p\) has mean \(p\), variance \(p(1-p)\), and third cumulant \(p(1-p)(1-2p)\).
The component means are \(0.2\) and \(0.8\), both component variances are \(0.16\), and the component third cumulants are \(0.096\) and \(-0.096\). The mixture mean is \[ \mu=0.75(0.2)+0.25(0.8)=0.35. \] Thus \(\delta_1=-0.15\) and \(\delta_2=0.45\). Its variance is \[ \begin{aligned} \kappa_2 &=0.75\{0.16+(-0.15)^2\}+0.25\{0.16+(0.45)^2\}\\ &=0.75(0.1825)+0.25(0.3625)\\ &=0.136875+0.090625=0.2275. \end{aligned} \] The third cumulant is \[ \begin{aligned} \kappa_3 &=0.75\{0.096+3(-0.15)(0.16)+(-0.15)^3\}\\ &\quad+0.25\{-0.096+3(0.45)(0.16)+(0.45)^3\}\\ &=0.75(0.020625)+0.25(0.211125)\\ &=0.01546875+0.05278125=0.06825. \end{aligned} \] As a check, the marginal variable is itself Bernoulli: \(P(X=1)=0.75(0.2)+0.25(0.8)=0.35\). Its variance is \(0.35(0.65)=0.2275\), and its third cumulant is \(0.35(0.65)(1-2(0.35))=0.06825\), agreeing with the mixture calculation.
Common Pitfall and Interpretation
A tempting but incorrect rule is to say that a mixture’s cumulants are just the weighted averages of the component cumulants. For the variance, that would retain only \(\sum_j\pi_j\sigma_j^2\). For the third cumulant, it would retain only \(\sum_j\pi_j\kappa_{3,j}\). Both omit effects of the component means.
The correction is to center every component at the mixture mean, not at its own mean alone. In the variance, this adds squared displacements. In the third cumulant, it adds both cubic displacements and the cross-term \(3(\mu_j-\mu)\sigma_j^2\). The normal-mixture example makes the point directly: the component third cumulants vanish, while the mixture third cumulant is \(12.936\).
These formulas are useful whenever a population is naturally divided into latent or observed groups—for example, a heterogeneous population, a model with latent classes, or a sampling scheme that selects among different distributions. They quantify how overall dispersion and asymmetry arise from both the distributions within groups and the differences among groups.
Check Your Understanding
Use the component-label representation and keep within-component and between-component terms distinct.
- For a two-component mixture with weights \(0.4\) and \(0.6\), and component means \(1\) and \(3\), calculate the mixture mean and the two displacements \(\mu_j-\mu\).
- Which part of the mixture variance remains if every component is a point mass? Explain why.
- In the third-cumulant formula, which term involves both a component variance and its mean’s displacement from the mixture mean?
- Can a mixture have a nonzero third cumulant when every component has third cumulant zero? Identify the terms that can produce it.
- For the Bernoulli mixture in the worked example, verify the marginal success probability directly from the component weights and probabilities.