Shared Latent Variables and False Sum Factorizations

Why a Shared Latent Variable Matters

Two counts can be independent once a random environment is known, yet dependent when that environment is unobserved. For example, two clinics may have independent patient counts conditional on a shared disease intensity. On days with high intensity, both counts tend to be high; on days with low intensity, both tend to be low. Averaging over the intensity therefore connects their outcomes.

The resulting transform calculation is easy to misstate. Conditional independence gives a product of conditional transforms. It does not generally give a product of the two marginal transforms. The distinction matters whenever a sum is formed from components sharing a random rate, probability, or other latent variable.

Definition: Random variables \(X\) and \(Y\) are conditionally independent given \(Z\) if, for all measurable sets \(A\) and \(B\), \(P(X\in A,Y\in B\mid Z)=P(X\in A\mid Z)P(Y\in B\mid Z)\) almost surely. This is independence within each conditional distribution given \(Z\); it does not assert that \(X\) and \(Y\) are independent after \(Z\) is averaged out.

The Correct Joint Transform

For nonnegative integer-valued \(X\) and \(Y\), their joint probability generating function (PGF) is \(G_{X,Y}(s,t)=E[s^X t^Y]\), for \(0\leq s,t\leq1\), with \(0^0=1\). Given \(Z\), write \(G_{X\mid Z}(s)=E[s^X\mid Z]\) and \(G_{Y\mid Z}(t)=E[t^Y\mid Z]\). The same principle applies to characteristic functions, which exist for every real argument because their integrands are bounded.

Theorem: If \(X\) and \(Y\) are conditionally independent given \(Z\), then

G_{X,Y}(s,t) =E\!\left[G_{X\mid Z}(s)G_{Y\mid Z}(t)\right].

The marginal PGFs are \(G_X(s)=E[G_{X\mid Z}(s)]\) and \(G_Y(t)=E[G_{Y\mid Z}(t)]\). In general, the joint PGF is not \(G_X(s)G_Y(t)\).

Proof. For \(s,t\in[0,1]\), the random variables \(s^X\) and \(t^Y\) are bounded and measurable, so conditional expectations and the tower property apply. Conditional independence implies \(E[s^X t^Y\mid Z]=E[s^X\mid Z]E[t^Y\mid Z]\). Taking expectations of both sides gives

E[s^X t^Y] =E\!\left[E[s^X\mid Z]E[t^Y\mid Z]\right] =E\!\left[G_{X\mid Z}(s)G_{Y\mid Z}(t)\right]. \quad\square

The tempting but generally invalid step is to replace the expectation of the product of the conditional PGFs by the product of their expectations. Those are different operations. For characteristic functions, the corresponding identity is

E[e^{iuX+ivY}] =E\!\left[ E[e^{iuX}\mid Z]E[e^{ivY}\mid Z] \right], \qquad u,v\in\mathbb R.

Here the conditional-independence identity applies to bounded complex-valued functions as well. A moment generating function version is also valid at arguments where the relevant expectations exist and the conditional expectations are well-defined; the PGF and characteristic-function identities avoid needing moment-generating-function existence.

Worked Example: Two Poisson Counts With a Shared Gamma Rate

Let \(\Lambda\sim\operatorname{Gamma}(2,1)\), using shape and rate parameters, so its density is \(f(\lambda)=\lambda e^{-\lambda}\) for \(\lambda>0\). Conditional on \(\Lambda\), let \(N_1\sim\operatorname{Poisson}(\Lambda)\) and \(N_2\sim\operatorname{Poisson}(2\Lambda)\), independently. Find the joint PGF, compare it with the product of marginal PGFs, and calculate the covariance.

Joint PGF. Conditional Poisson PGFs give \(G_{N_1\mid\Lambda}(s)=\exp[-\Lambda(1-s)]\) and \(G_{N_2\mid\Lambda}(t)=\exp[-2\Lambda(1-t)]\). Conditional independence therefore yields

G_{N_1,N_2}(s,t) =E\!\left[\exp\{-\Lambda[(1-s)+2(1-t)]\}\right] =\left(1+(1-s)+2(1-t)\right)^{-2}.

For the last equality, the Gamma Laplace transform is \(E[e^{-c\Lambda}]=(1+c)^{-2}\) for \(c\geq0\), obtained by integrating \(\int_0^\infty \lambda e^{-(1+c)\lambda}\,d\lambda=(1+c)^{-2}\).

Marginal product and a direct check. Setting the other PGF argument to \(1\) gives \(G_{N_1}(s)=(2-s)^{-2}\) and \(G_{N_2}(t)=(3-2t)^{-2}\). Their product is not the joint PGF in general. At \(s=t=0\), the actual probability that both counts are zero is \(G_{N_1,N_2}(0,0)=4^{-2}=1/16\). Multiplying marginal zero probabilities instead gives \(G_{N_1}(0)G_{N_2}(0)=(1/4)(1/9)=1/36\).

Covariance. The conditional means are \(\Lambda\) and \(2\Lambda\), and the conditional covariance is zero. Since \(\operatorname{Var}(\Lambda)=2\), the shared rate produces covariance \(\operatorname{Cov}(N_1,N_2)=\operatorname{Cov}(\Lambda,2\Lambda)=4\). The counts are not independent despite being independent conditional on \(\Lambda\).

How the Shared Environment Creates Dependence

The covariance calculation in the example is a general consequence of conditional independence. It makes explicit which part of the dependence remains after conditional independence has removed the within-environment association.

Theorem: Suppose \(X\) and \(Y\) have finite second moments and are conditionally independent given \(Z\). Then

\operatorname{Cov}(X,Y) =\operatorname{Cov}\!\left(E[X\mid Z],E[Y\mid Z]\right).

Proof. Conditional independence and the finite second moments imply \(E[XY\mid Z]=E[X\mid Z]E[Y\mid Z]\) almost surely. Taking expectations gives \(E[XY]=E[E[X\mid Z]E[Y\mid Z]]\). Also, the tower property gives \(E[X]=E[E[X\mid Z]]\) and \(E[Y]=E[E[Y\mid Z]]\). Substitution into the definition of covariance gives

\begin{aligned} \operatorname{Cov}(X,Y) &=E[XY]-E[X]E[Y]\\ &=E[E[X\mid Z]E[Y\mid Z]] -E[E[X\mid Z]]E[E[Y\mid Z]]\\ &=\operatorname{Cov}(E[X\mid Z],E[Y\mid Z]). \end{aligned} \quad\square

Thus a latent variable induces positive covariance when it tends to move the two conditional means in the same direction, and negative covariance when it moves them in opposite directions. Zero covariance alone does not establish independence.

Worked Example: A Shared Random Success Probability

Let \(P\sim\operatorname{Uniform}(0,1)\). Given \(P\), let \(X\) and \(Y\) be independent Bernoulli random variables with success probability \(P\). Find their joint PGF and compare the probability of two successes with the value under marginal independence.

Conditional on \(P\), the two PGFs are \(1-P+Ps=1+P(s-1)\) and \(1+P(t-1)\). Therefore

\begin{aligned} G_{X,Y}(s,t) &=E[(1+P(s-1))(1+P(t-1))]\\ &=1+\frac{s+t-2}{2}+\frac{(s-1)(t-1)}{3}, \end{aligned}

because \(E[P]=1/2\) and \(E[P^2]=1/3\). Each marginal is Bernoulli with success probability \(1/2\), so the product of marginal PGFs would be \((1+s)(1+t)/4\). In particular, \(P(X=1,Y=1)=E[P^2]=1/3\), whereas marginal independence would give \(P(X=1)P(Y=1)=1/4\). The shared probability also gives \(\operatorname{Cov}(X,Y)=\operatorname{Var}(P)=1/12\).

Transforms of Sums

For \(S=X+Y\), the PGF is found from the joint PGF along the diagonal: \(G_S(z)=G_{X,Y}(z,z)\). This identity follows directly from \(z^{X+Y}=z^Xz^Y\); it does not require independence. By contrast, \(G_X(z)G_Y(z)\) is the PGF of the sum when \(X\) and \(Y\) are independent. When they share a latent variable, use the conditional product first, then average over that variable.

Worked Example: The Sum of the Shared-Rate Counts

Continue with the Gamma-Poisson model above. Find the PGF and probability mass function of \(S=N_1+N_2\), and check the probability at zero against the false marginal-product calculation.

Given \(\Lambda\), the independent Poisson counts have sum \(S\mid\Lambda\sim\operatorname{Poisson}(3\Lambda)\). Thus, for \(0\leq z\leq1\),

G_S(z)=G_{N_1,N_2}(z,z) =E[e^{-3\Lambda(1-z)}] =(1+3(1-z))^{-2}.

The mass function can also be obtained directly from the mixture. For integers \(k\geq0\),

\begin{aligned} P(S=k) &=\int_0^\infty e^{-3\lambda}\frac{(3\lambda)^k}{k!} \lambda e^{-\lambda}\,d\lambda\\ &=\frac{3^k}{k!}\int_0^\infty \lambda^{k+1}e^{-4\lambda}\,d\lambda\\ &=\frac{3^k}{k!}\frac{\Gamma(k+2)}{4^{k+2}} =\frac{(k+1)3^k}{4^{k+2}}. \end{aligned}

At \(k=0\), this is \(1/16\), matching the joint calculation. Multiplying marginal PGFs would instead give \(G_{N_1}(0)G_{N_2}(0)=1/36\), the wrong zero-count probability for the sum.

When Factorization Is and Is Not Justified

For transforms, the correct general comparison is

E[A(Z)B(Z)] \quad\text{versus}\quad E[A(Z)]E[B(Z)],

where \(A(Z)=G_{X\mid Z}(s)\) and \(B(Z)=G_{Y\mid Z}(t)\). The expressions agree if, for example, one conditional transform is constant in \(Z\), or if the relevant conditional transforms have zero covariance. But conditional independence by itself does not imply either condition. Nor does equality at one particular pair of transform arguments establish independence.

If \(X\) and \(Y\) are genuinely independent without conditioning, their joint PGF does factor as \(G_X(s)G_Y(t)\). In settings with a latent variable, that stronger independence must be justified separately. A useful diagnostic is to ask whether learning that the latent rate or probability was high would change the expected value of both variables. If so, a shared source of variation can associate them even when their conditional outcomes are independent.

Common pitfall: “The counts are conditionally independent, so \(G_{X,Y}(s,t)=G_X(s)G_Y(t)\).” This replaces an expectation of a product of conditional transforms with a product of expectations. The correct identity is \(G_{X,Y}(s,t)=E[G_{X\mid Z}(s)G_{Y\mid Z}(t)]\). For a sum, evaluate this joint PGF at \(s=t=z\); use a product of marginal PGFs only when unconditional independence has been established.

Check Your Understanding

Use the conditional-transform identity and the role of the shared latent variable to answer each question.

If \(X\) and \(Y\) are conditionally independent given \(Z\), write their joint characteristic function in terms of conditional characteristic functions.
In the shared Gamma-rate example, why is \(P(N_1=0,N_2=0)\) not the product of the two marginal zero probabilities?
Suppose \(X\) and \(Y\) are conditionally independent given \(Z\), with finite second moments. What determines their covariance?
For a dependent pair of nonnegative integer-valued variables, how can the PGF of \(X+Y\) be obtained from the joint PGF?
Give one condition under which the product of marginal transforms could equal the joint transform even though the variables are described using a latent variable.