Why Condition Before Calculating Cumulants?
A latent variable divides a population into subgroups, each with its own conditional distribution. The unconditional distribution combines two sources of variation: observations vary within each subgroup, and the subgroup-specific distributions may differ from one another. Conditional cumulants separate these contributions. For variance, the separation is the familiar within-group plus between-group decomposition. At third order, a covariance term also appears: differences in subgroup means can be associated with differences in subgroup variances.
We will work with a finite-valued latent variable so that all conditional distributions can be handled directly. This setting includes many mixture models and keeps the regularity conditions explicit. No independence between the response \(X\) and the latent variable \(Z\) is assumed; indeed, their dependence is what the conditional distributions describe.
Conditional Cumulants
Suppose \(Z\) takes values \(z_1,\ldots,z_k\), each with positive probability, and \(X\) has a moment-generating function finite on an open interval containing zero under each conditional distribution given \(Z=z_j\). Because there are finitely many states, the interval can be chosen common to all states by taking the intersection of the individual intervals. The conditional cumulant-generating function in state \(z_j\) is the logarithm of the conditional moment-generating function.
The first three cumulants are the mean, variance, and third central moment, respectively. Thus \(\kappa_1(X\mid Z)=m_Z\), \(\kappa_2(X\mid Z)=v_Z\), and \(\kappa_3(X\mid Z)=E[(X-m_Z)^3\mid Z]\). The moment-generating-function condition ensures these quantities exist. All unconditional moments used below are finite as well: there are finitely many states, and each conditional distribution has a moment-generating function near zero.
The Conditional-Cumulant Identities
The variance identity says that unconditional variance consists of average conditional variance plus the variance of the conditional means. The third-cumulant identity has three contributions: the average within-state third cumulant, the third cumulant of the state-specific means, and a covariance between those means and the state-specific variances. We prove both by centering within and between states.
Let \(\mu=E[X]\), and define \(A=m_Z-\mu\) and \(\varepsilon=X-m_Z\). By iterated expectation, \(E[m_Z]=\mu\), so \(E[A]=0\). Also, \(E[\varepsilon\mid Z]=0\), \(E[\varepsilon^2\mid Z]=v_Z\), and \(E[\varepsilon^3\mid Z]=\kappa_3(X\mid Z)\). Since \(X-\mu=A+\varepsilon\), expanding its square gives
Because \(A\) is a function of \(Z\), \(E[A\varepsilon]=E[A\,E(\varepsilon\mid Z)]=0\). Also, \(E[A^2]=\operatorname{Var}(m_Z)\), and \(E[\varepsilon^2]=E[E(\varepsilon^2\mid Z)]=E[v_Z]\). Substitution proves the total-variance identity. \(\square\)
For the third cumulant, expand the cube instead. The third cumulant of \(X\) is \(E[(X-\mu)^3]\), so
The second term vanishes because \(E[A^2\varepsilon]=E[A^2E(\varepsilon\mid Z)]=0\). For the third term, condition on \(Z\): \(E[A\varepsilon^2]=E[A\,E(\varepsilon^2\mid Z)]=E[A v_Z]\). Since \(E[A]=0\), this is \(\operatorname{Cov}(m_Z,v_Z)\). Finally, \(E[A^3]=\kappa_3(m_Z)\), and \(E[\varepsilon^3]=E[\kappa_3(X\mid Z)]\). These equalities give the stated third-cumulant decomposition. \(\square\)
Worked Calculations
Worked Example: A Bernoulli Mixture
Let \(P(Z=0)=0.7\) and \(P(Z=1)=0.3\). Suppose \(X\mid Z=0\sim\operatorname{Bernoulli}(0.2)\), while \(X\mid Z=1\sim\operatorname{Bernoulli}(0.8)\). Calculate the unconditional variance and third cumulant from the conditional identities, then check them from the marginal distribution.
For a Bernoulli random variable with success probability \(p\), the mean is \(p\), the variance is \(p(1-p)\), and the third cumulant is \(p(1-p)(1-2p)\). The two conditional means are therefore \(0.2\) and \(0.8\); both conditional variances are \(0.16\). The conditional third cumulants are \(0.2(0.8)(0.6)=0.096\) and \(0.8(0.2)(-0.6)=-0.096\).
The unconditional mean is \(E[m_Z]=0.7(0.2)+0.3(0.8)=0.38\). The variance of the conditional mean is \(0.7(0.2-0.38)^2+0.3(0.8-0.38)^2=0.0756\). Consequently, \(\operatorname{Var}(X)=0.16+0.0756=0.2356\).
The average conditional third cumulant is \(0.7(0.096)+0.3(-0.096)=0.0384\). The third cumulant of \(m_Z\) is \(0.7(-0.18)^3+0.3(0.42)^3=0.018144\). The conditional variance is constant at \(0.16\), so \(\operatorname{Cov}(m_Z,v_Z)=0\). Thus \(\kappa_3(X)=0.0384+0.018144=0.056544\).
Marginally, \(P(X=1)=0.7(0.2)+0.3(0.8)=0.38\), so \(X\) is Bernoulli with success probability \(0.38\). Its variance is \(0.38(0.62)=0.2356\), and its third cumulant is \(0.38(0.62)(1-2(0.38))=0.056544\), agreeing with both calculations.
Worked Example: A Poisson Rate Chosen by a Latent State
Let \(P(Z=1)=P(Z=3)=1/2\), and suppose \(X\mid Z=z\sim\operatorname{Poisson}(z)\). Find the unconditional variance and third cumulant.
A Poisson distribution with rate \(z\) has conditional mean, variance, and third cumulant all equal to \(z\). Therefore \(E[v_Z]=E[Z]=(1+3)/2=2\), and \(\operatorname{Var}(m_Z)=\operatorname{Var}(Z)=((1-2)^2+(3-2)^2)/2=1\). The total variance is \(2+1=3\).
The average conditional third cumulant is also \(E[Z]=2\). The conditional mean \(m_Z=Z\) takes the values 1 and 3 with equal probabilities, centered at 2, so its third cumulant is \(((-1)^3+1^3)/2=0\). Since \(m_Z=v_Z=Z\), \(\operatorname{Cov}(m_Z,v_Z)=\operatorname{Var}(Z)=1\). The third-cumulant identity gives \(\kappa_3(X)=2+0+3(1)=5\).
This calculation shows that the marginal third cumulant need not equal the average of the conditional third cumulants: that average is 2 here, whereas the unconditional third cumulant is 5. The difference comes from variation in the conditional means and their association with the conditional variances.
Worked Example: Why the Covariance Term Matters
Let \(Z\) take values 0 and 1 with equal probability. Given \(Z=0\), let \(X\) be Bernoulli with success probability \(1/2\); given \(Z=1\), let \(X=1\) with probability 1. Calculate the third-cumulant contributions.
The conditional means are \(m_0=1/2\) and \(m_1=1\), and the conditional variances are \(v_0=1/4\) and \(v_1=0\). Both conditional third cumulants are zero: the first conditional Bernoulli distribution is symmetric about its mean, and the second is constant. Hence \(E[\kappa_3(X\mid Z)]=0\).
The conditional mean \(m_Z\) takes \(1/2\) and 1 with equal probabilities. Its overall mean is \(3/4\), and its centered values are \(-1/4\) and \(1/4\); therefore \(\kappa_3(m_Z)=0\). For the covariance term, \(E[m_Zv_Z]=\tfrac12(\tfrac12\cdot\tfrac14)+\tfrac12(1\cdot0)=1/16\), while \(E[m_Z]E[v_Z]=(3/4)(1/8)=3/32\). Thus \(\operatorname{Cov}(m_Z,v_Z)=1/16-3/32=-1/32\).
The decomposition gives \(\kappa_3(X)=0+0+3(-1/32)=-3/32\). Directly, the marginal probabilities are \(P(X=0)=1/4\) and \(P(X=1)=3/4\), so \(X\) is Bernoulli with success probability \(3/4\). Its third cumulant is \((3/4)(1/4)(1-2(3/4))=-3/32\), as required. Omitting the covariance term would incorrectly give zero.
Interpretation and a Common Pitfall
For variance, conditional variability and variation among conditional means add. At third order, these components do not simply add: the term \(3\operatorname{Cov}(m_Z,v_Z)\) records whether states with larger means also tend to have larger conditional variances. It can be positive, negative, or zero. In the last example it is negative, and it determines the sign of the unconditional third cumulant.
A common mistake is to assert that the unconditional cumulant is the average of the conditional cumulants. That reasoning ignores changes in the conditional distribution across latent states. For variance, it drops \(\operatorname{Var}(m_Z)\); for the third cumulant, it drops both \(\kappa_3(m_Z)\) and the covariance term. The correction is to identify the conditional mean and variance first, then use the relevant decomposition. The average conditional cumulant equals the unconditional one only when the additional terms vanish.
Check Your Understanding
Use the conditional identities to answer the following questions.
- In the variance identity, what does \(\operatorname{Var}(m_Z)\) measure that \(E[v_Z]\) does not?
- If \(v_Z\) is constant across states, what happens to \(\operatorname{Cov}(m_Z,v_Z)\)? Explain.
- Suppose every conditional distribution is symmetric about its conditional mean. Which term in the third-cumulant decomposition must then be zero?
- For the Poisson mixture above, why is the unconditional third cumulant larger than the average conditional third cumulant?
- Can the covariance contribution to the third-cumulant identity be negative? Identify the worked example that demonstrates this and give its value.