Why Factorization Criteria Matter
Conditional independence is often introduced through an equality involving events. For proofs and calculations, it is useful to have equivalent formulations that fit the objects at hand: a factorization of conditional laws, an identity for products of test functions, or a statement about conditional expectations.
These formulations are equivalent, but the almost-sure qualifications matter. Conditional expectations and conditional laws are defined only up to null sets, and a separate null set may arise for each choice of test functions. We will make that distinction explicit.
Setting and Definition
Let \(X\), \(Y\), and \(Z\) be random elements taking values in standard Borel spaces \(S\), \(T\), and \(U\), respectively, on a probability space \((\Omega,\mathcal F,P)\). Write \(\sigma(Z)\) for the sigma-field generated by \(Z\), and use regular conditional laws given \(Z\): let \(K_Z\) be a conditional law of \(X\) given \(Z\), \(L_Z\) a conditional law of \(Y\) given \(Z\), and \(Q_Z\) a conditional law of \((X,Y)\) given \(Z\). Here, for example, \(K_Z(\omega,A)=K(Z(\omega),A)\) for a probability kernel \(K\) and \(A\in\mathcal B(S)\).
The equivalence in this definition is part of the factorization criteria proved below. Standard Borel state spaces ensure that the regular conditional laws exist and that the product kernel \(z\mapsto K_z\otimes L_z\) is a probability kernel.
The Main Equivalence Theorem
- \(X\) and \(Y\) are conditionally independent given \(Z\).
- For every bounded Borel-measurable \(f:S\to\mathbb R\) and \(g:T\to\mathbb R\), $$ E[f(X)g(Y)\mid \sigma(Z)] = E[f(X)\mid \sigma(Z)]E[g(Y)\mid \sigma(Z)] \quad\text{almost surely.} $$
- For every bounded Borel-measurable \(f:S\to\mathbb R\), $$ E[f(X)\mid \sigma(Y,Z)] = E[f(X)\mid \sigma(Z)] \quad\text{almost surely.} $$
- For every bounded Borel-measurable \(g:T\to\mathbb R\), $$ E[g(Y)\mid \sigma(X,Z)] = E[g(Y)\mid \sigma(Z)] \quad\text{almost surely.} $$
We prove the equivalences in detail. In the proof, each conditional-expectation identity is asserted almost surely for its specified test functions. It is not necessary to choose a single null set that works simultaneously for every bounded measurable function.
Proof of the Theorem
Conditional independence and kernel factorization. For fixed \(A\in\mathcal B(S)\) and \(B\in\mathcal B(T)\), the definition of conditional law gives \(Q_Z(A\times B)=P(X\in A,Y\in B\mid\sigma(Z))\), \(K_Z(A)=P(X\in A\mid\sigma(Z))\), and \(L_Z(B)=P(Y\in B\mid\sigma(Z))\), almost surely. Thus conditional independence gives \(Q_z(A\times B)=K_z(A)L_z(B)\) almost surely for each fixed rectangle \(A\times B\).
To conclude equality of the kernels on a common full-measure set, choose countable pi-systems generating \(\mathcal B(S)\) and \(\mathcal B(T)\); such systems exist for standard Borel spaces. There are only countably many rectangles formed from these systems. Intersect the corresponding full-measure sets. For every \(z\) in this intersection, \(Q_z\) and \(K_z\otimes L_z\) agree on the generating rectangles. The pi-lambda theorem then gives equality of these probability measures on \(\mathcal B(S)\otimes\mathcal B(T)\). Conversely, equality of the kernels almost everywhere gives the conditional-independence identity on every rectangle, and hence the definition.
Kernel factorization implies the product-test identity. For bounded Borel \(f\) and \(g\), integration against the conditional joint law yields $$ E[f(X)g(Y)\mid\sigma(Z)] =\int_{S\times T}f(x)g(y)\,Q_Z(dx,dy). $$ If \(Q_Z=K_Z\otimes L_Z\) almost surely, integration against the product measure separates: $$ \int_{S\times T}f(x)g(y)\,(K_Z\otimes L_Z)(dx,dy) = \left(\int_S f\,dK_Z\right)\left(\int_T g\,dL_Z\right). $$ The two factors are versions of \(E[f(X)\mid\sigma(Z)]\) and \(E[g(Y)\mid\sigma(Z)]\), proving condition 2.
The product-test identity implies kernel factorization. Apply condition 2 to indicator functions \(f=1_A\) and \(g=1_B\). It gives the rectangle identity in the definition of conditional independence for each fixed pair \(A,B\). The countable generating-class argument above then proves \(Q_z=K_z\otimes L_z\) for \(P_Z\)-almost every \(z\).
The product-test identity implies the conditional-expectation identity. Fix bounded Borel \(f:S\to\mathbb R\), and write \(m=E[f(X)\mid\sigma(Z)]\). For bounded Borel \(g:T\to\mathbb R\) and \(k:U\to\mathbb R\), the product-test identity and the tower property give $$ E[f(X)g(Y)k(Z)] = E[m\,E[g(Y)\mid\sigma(Z)]\,k(Z)] = E[m\,g(Y)k(Z)]. $$ The functions of the form \(g(Y)k(Z)\) generate, through finite linear combinations and bounded monotone limits, the bounded \(\sigma(Y,Z)\)-measurable functions. Therefore \(E[(f(X)-m)H]=0\) for every bounded \(\sigma(Y,Z)\)-measurable \(H\). This is the defining property of conditional expectation, so \(E[f(X)\mid\sigma(Y,Z)]=m\) almost surely.
The conditional-expectation identity implies the product-test identity. Suppose condition 3 holds, and let \(f,g\) be bounded Borel functions. Since \(g(Y)\) is \(\sigma(Y,Z)\)-measurable, the tower property gives $$ \begin{aligned} E[f(X)g(Y)\mid\sigma(Z)] &=E\!\left[E[f(X)\mid\sigma(Y,Z)]g(Y)\mid\sigma(Z)\right]\\ &=E\!\left[E[f(X)\mid\sigma(Z)]g(Y)\mid\sigma(Z)\right]\\ &=E[f(X)\mid\sigma(Z)]E[g(Y)\mid\sigma(Z)]. \end{aligned} $$ Thus condition 3 implies condition 2. Interchanging \(X\) and \(Y\) proves the corresponding implication from condition 4. Condition 2 itself is symmetric in \(X\) and \(Y\); the preceding argument therefore also gives condition 4. All four conditions are equivalent. \(\square\)
Worked Example: A Finite Conditional Product Law
Worked Example: A Finite Conditional Product Law
Let \(Z\) take values \(0\) and \(1\), each with probability \(1/2\), and let \(X,Y\) be binary random variables whose conditional joint laws are products. Given \(Z=0\), set \(P(X=1\mid Z=0)=1/3\) and \(P(Y=1\mid Z=0)=1/2\). Given \(Z=1\), set \(P(X=1\mid Z=1)=3/4\) and \(P(Y=1\mid Z=1)=1/4\).
For \(z=0\), the conditional joint probability of \(X=1,Y=1\) is \((1/3)(1/2)=1/6\). For \(z=1\), it is \((3/4)(1/4)=3/16\). More generally, each of the four conditional cell probabilities is the product of its two conditional marginal probabilities, so \(Q_z=K_z\otimes L_z\) at both values of \(z\).
Taking \(f(x)=x\) and \(g(y)=y\), the product-test identity reads \(E[XY\mid Z]=E[X\mid Z]E[Y\mid Z]\). At \(Z=0\), both sides equal \(1/6\); at \(Z=1\), both equal \(3/16\). The conditional-expectation criterion also holds: for either value of \(Y\), \(E[X\mid Y,Z]=E[X\mid Z]\), because the conditional joint probabilities factor.
Worked Example: Conditional Independence Need Not Be Marginal Independence
Worked Example: Conditional Independence Need Not Be Marginal Independence
Let \(Z\) be Bernoulli with success probability \(1/2\), and set \(X=Z\) and \(Y=Z\). Given \(Z=z\), both \(X\) and \(Y\) equal \(z\) with probability one. Their conditional joint law is \(\delta_{(z,z)}=\delta_z\otimes\delta_z\), so they are conditionally independent given \(Z\).
Marginally, however, \(X=Y\) always, so \(P(X=Y)=1\). Also \(E[X]=E[Y]=1/2\), while \(E[XY]=E[Z^2]=1/2\), which differs from \(E[X]E[Y]=1/4\). Thus the factorization is conditional on \(Z\); it does not assert factorization after \(Z\) is omitted.
Worked Example: A Product-Test Failure Detects Dependence
Worked Example: A Product-Test Failure Detects Dependence
Let \(X=Y\) be a fair binary random variable and let \(Z\) be constant. Then \(E[X\mid Z]=E[Y\mid Z]=1/2\), but \(E[XY\mid Z]=E[X^2]=1/2\), since \(X^2=X\). The product-test identity with \(f(x)=x\) and \(g(y)=y\) would require \(1/2=(1/2)(1/2)=1/4\), which is false.
The conditional-expectation criterion fails as well: \(E[X\mid Y,Z]=Y\), whereas \(E[X\mid Z]=1/2\). Since \(Y\) is not almost surely \(1/2\), these conditional expectations are not equal. Either failed test rules out conditional independence.
Worked Example: Factorization for Continuous Conditional Laws
Worked Example: Factorization for Continuous Conditional Laws
Let \(Z\) have any probability law on \(\mathbb R\). Specify a conditional joint law of \((X,Y)\) given \(Z=z\) by taking \(X\) uniform on \([z,z+1]\), \(Y\) uniform on \([-z,1-z]\), and their conditional joint law to be the product of these two uniform laws. For bounded Borel \(f,g:\mathbb R\to\mathbb R\), direct integration gives $$ E[f(X)g(Y)\mid Z=z] = \left(\int_0^1 f(z+u)\,du\right) \left(\int_0^1 g(v-z)\,dv\right). $$
The first factor is \(E[f(X)\mid Z=z]\), and the second is \(E[g(Y)\mid Z=z]\). Thus the product-test criterion holds for every bounded Borel pair \(f,g\), and the conditional laws factor. The calculation uses the change of variables \(x=z+u\) and \(y=v-z\), each with unit Jacobian.
How to Use the Criteria Carefully
The most convenient criterion depends on the question. To verify a conditional law, check \(Q_z=K_z\otimes L_z\). To test an identity involving moments or other bounded transformations, use the product-test formulation. To show that observing \(Y\) in addition to \(Z\) does not change the conditional expectation of a function of \(X\), use the conditional-expectation formulation.
There is also a useful distinction between “for each test pair, almost surely” and “for all test pairs, on one common full-measure set.” The theorem’s conditional-expectation statements have the first meaning. Kernel factorization is a statement about probability measures and can be made on a common full-measure set by using countable generating pi-systems and then extending by the pi-lambda theorem. This avoids an unjustified intersection over an uncountable collection of null sets.
Check Your Understanding
Use the equivalences and their almost-sure qualifications to answer the following questions.
- What does the product-test criterion say when \(f\) and \(g\) are indicator functions?
- Why can countable generating pi-systems be used to obtain a common full-measure set for conditional-kernel equality?
- Which conditional-expectation identity expresses that learning \(Y\) in addition to \(Z\) does not change the conditional expectation of \(f(X)\)?
- In the example \(X=Y=Z\), why does conditional independence hold even though \(X\) and \(Y\) are marginally dependent?
- Why does failure of the product-test identity for one bounded pair disprove conditional independence?