Tutorials › Probability Theory (Graduate) › Conditional Independence of Random Elements

Home  /  Tutorials  /  Conditional Laws and Disintegration

Conditional Laws and Disintegration · Tutorial 266 of 946

Conditional Independence of Random Elements

Learn to recognize conditional independence from either conditional expectations or conditional probability kernels, and understand why it need not imply ordinary independence.

Advanced 10 min read

What You'll Learn

  • Define conditional independence of random elements given a sigma-algebra
  • Express the definition using conditional expectations of bounded measurable functions
  • Describe conditional joint and marginal laws as probability kernels
  • Prove that conditional expectation and kernel formulations are equivalent
  • Distinguish conditional independence from ordinary independence

Why Condition on a Sigma-Algebra?

Two random elements can be dependent overall but become independent once some information is known. Conditional independence makes this idea precise: after conditioning on a sigma-algebra, the conditional joint law of the two elements is the product of their separate conditional laws.

The conditioning information may be the information generated by a random element, or it may be an abstract sigma-algebra \(\mathcal G\subseteq\mathcal F\). We will define the concept in both conditional-expectation and conditional-law terms, and prove that these definitions agree when the random elements take values in standard Borel spaces.

Conditional Independence Through Expectations

Let \((\Omega,\mathcal F,\mathbb P)\) be a probability space, let \(\mathcal G\subseteq\mathcal F\) be a sub-sigma-algebra, and let \(X:\Omega\to S\) and \(Y:\Omega\to T\) be random elements. Here \(S\) and \(T\) are measurable spaces, and \(X,Y\) are measurable with respect to their respective sigma-algebras.

Definition: The random elements \(X\) and \(Y\) are conditionally independent given \(\mathcal G\) if, for every bounded measurable \(f:S\to\mathbb R\) and \(g:T\to\mathbb R\), $$ \mathbb E[f(X)g(Y)\mid\mathcal G] = \mathbb E[f(X)\mid\mathcal G]\, \mathbb E[g(Y)\mid\mathcal G] \quad\text{almost surely}. $$ We write \(X\perp\!\!\!\perp Y\mid\mathcal G\).

Each conditional expectation in this definition is defined only up to almost-sure equality. The statement therefore means that the equality holds almost surely for each chosen pair \(f,g\); it does not initially assert one common exceptional null set for every possible pair of functions.

It is enough to use indicator functions in this definition. Indeed, taking \(f=\mathbf 1_A\) and \(g=\mathbf 1_B\) gives the event formulation $$ \mathbb E[\mathbf 1_{\{X\in A\}}\mathbf 1_{\{Y\in B\}}\mid\mathcal G] = \mathbb E[\mathbf 1_{\{X\in A\}}\mid\mathcal G]\, \mathbb E[\mathbf 1_{\{Y\in B\}}\mid\mathcal G], $$ for measurable \(A\subseteq S\) and \(B\subseteq T\). Conversely, the indicator formulation extends first to simple functions by linearity and then to bounded measurable functions by bounded approximation and conditional expectation convergence.

Conditional Laws as Probability Kernels

Suppose now that \(S\) and \(T\) are standard Borel spaces. A standard Borel space is a measurable space isomorphic to a Borel subset of a complete separable metric space. For a random element taking values in such a space, a conditional distribution given \(\mathcal G\) exists, even when \(\mathcal G\) is not generated by a random element.

Write \(K_X(\omega,\cdot)\) and \(K_Y(\omega,\cdot)\) for conditional laws of \(X\) and \(Y\) given \(\mathcal G\), and \(K_{X,Y}(\omega,\cdot)\) for a conditional law of the pair \((X,Y)\). These are probability kernels: for each \(\omega\), the value is a probability measure, and for each measurable set, its value as a function of \(\omega\) is \(\mathcal G\)-measurable. For example, \(K_X\) satisfies $$ K_X(\,\cdot\,,A) = \mathbb E[\mathbf 1_{\{X\in A\}}\mid\mathcal G] \quad\text{almost surely} $$ for every measurable \(A\subseteq S\). The other conditional laws satisfy the corresponding identity.

For each \(\omega\), the product \(K_X(\omega,\cdot)\otimes K_Y(\omega,\cdot)\) is a probability measure on \(S\times T\). It is the conditional-law counterpart of multiplying the separate conditional expectations.

Definition: For standard Borel \(S,T\), \(X\) and \(Y\) are conditionally independent given \(\mathcal G\) in the conditional-law sense if $$ K_{X,Y}(\omega,\cdot) = K_X(\omega,\cdot)\otimes K_Y(\omega,\cdot) \quad\text{for almost every }\omega. $$ The equality is an equality of probability measures on \(S\times T\).

Conditional laws are unique up to almost-sure equality. Thus the definition does not depend on which versions of the kernels are chosen. The standard Borel assumption matters: it provides conditional probability kernels and countable determining classes that let us compare the random probability measures on a single event of probability one.

Equivalence of the Two Definitions

Theorem: Let \(X\) and \(Y\) take values in standard Borel spaces \(S\) and \(T\), and let \(\mathcal G\subseteq\mathcal F\). Then the following are equivalent:
  • \(X\perp\!\!\!\perp Y\mid\mathcal G\) according to the conditional-expectation definition;
  • the conditional joint law equals the product of the conditional marginal laws almost surely.
Proof. Standard Borel spaces have countable generating pi-systems; choose such systems \(\mathcal A\) for \(S\) and \(\mathcal B\) for \(T\). Suppose first that the conditional-expectation definition holds. For each \(A\in\mathcal A\) and \(B\in\mathcal B\), applying it to the two indicator functions gives $$ K_{X,Y}(\omega,A\times B) = K_X(\omega,A)K_Y(\omega,B) $$ almost surely. There are only countably many pairs \((A,B)\), so there is one event of probability one on which these identities hold for every such pair. For each \(\omega\) in this event, the probability measures \(K_{X,Y}(\omega,\cdot)\) and \(K_X(\omega,\cdot)\otimes K_Y(\omega,\cdot)\) agree on every generating rectangle \(A\times B\). The rectangles form a pi-system generating the product sigma-algebra, so uniqueness of measures implies that the two probability measures agree on all measurable subsets of \(S\times T\).
Proof continued. Conversely, suppose the conditional joint law equals the product kernel almost surely. For measurable \(A\subseteq S\) and \(B\subseteq T\), the conditional-law identities give $$ \begin{aligned} \mathbb E[\mathbf 1_{\{X\in A\}}\mathbf 1_{\{Y\in B\}}\mid\mathcal G] &=K_{X,Y}(\,\cdot\,,A\times B)\\ &=K_X(\,\cdot\,,A)K_Y(\,\cdot\,,B)\\ &=\mathbb E[\mathbf 1_{\{X\in A\}}\mid\mathcal G]\, \mathbb E[\mathbf 1_{\{Y\in B\}}\mid\mathcal G] \end{aligned} $$ almost surely. Linearity gives the same identity for simple functions \(f,g\). Every bounded measurable real-valued function is a pointwise limit of uniformly bounded simple functions. Applying bounded convergence to the products and to the conditional expectations extends the identity to all bounded measurable \(f,g\). This is conditional independence by definition. \(\square\)

Worked Example: Conditional Independence with Dependence Overall

Worked Example: Conditional Independence with Dependence Overall

Let \(Z\) be Bernoulli with probabilities \(\mathbb P(Z=0)=\mathbb P(Z=1)=1/2\), and let \(U,V\) be independent uniform random variables on \([0,1]\), independent of \(Z\). Define \(p_0=q_0=0.1\), \(p_1=q_1=0.9\), and set $$ X=\mathbf 1_{\{U\le p_Z\}}, \qquad Y=\mathbf 1_{\{V\le q_Z\}}, \qquad \mathcal G=\sigma(Z). $$

Given \(Z=z\), the events defining \(X\) and \(Y\) depend on the independent variables \(U\) and \(V\), respectively. For \(x,y\in\{0,1\}\), write \(r_z(1)=p_z\), \(r_z(0)=1-p_z\), \(s_z(1)=q_z\), and \(s_z(0)=1-q_z\). Then $$ \mathbb P(X=x,Y=y\mid Z=z)=r_z(x)s_z(y). $$ The conditional joint law is therefore the product of the conditional Bernoulli laws, so \(X\perp\!\!\!\perp Y\mid\sigma(Z)\).

They are not independent without conditioning. Since \(p_z=q_z\), $$ \mathbb E[X]=\mathbb E[Y]=\tfrac12, \qquad \mathbb E[XY]=\tfrac12(0.1^2)+\tfrac12(0.9^2)=0.41. $$ If \(X\) and \(Y\) were independent, their product expectation would be \(\mathbb E[X]\mathbb E[Y]=0.25\), not \(0.41\). The variation in their conditional success probabilities, driven by the shared \(Z\), produces dependence in their unconditional joint law.

Worked Example: A Conditionally Known Random Element

Worked Example: A Conditionally Known Random Element

Let \(Z\) be any random element, let \(X\) be independent of \(Z\), and take \(\mathcal G=\sigma(Z)\). Set \(Y=Z\), regarding \(Y\) as taking values in the same standard Borel space as \(Z\). For bounded measurable \(f,g\), \(g(Y)=g(Z)\) is \(\mathcal G\)-measurable. Independence of \(X\) and \(Z\) gives $$ \mathbb E[f(X)g(Y)\mid\mathcal G] = g(Z)\mathbb E[f(X)\mid\mathcal G] = g(Z)\mathbb E[f(X)]. $$ Also \(\mathbb E[g(Y)\mid\mathcal G]=g(Z)\) and \(\mathbb E[f(X)\mid\mathcal G]=\mathbb E[f(X)]\), so the conditional-expectation identity holds.

In terms of kernels, the conditional law of \(Y\) given \(\mathcal G\) is the point mass \(\delta_Z\), while the conditional law of \(X\) is its ordinary law \(\mu_X\). The conditional joint law is \(\mu_X\otimes\delta_Z\), precisely the product of those two conditional laws. Knowing \(Y\) exactly does not prevent conditional independence: the product statement concerns the conditional laws given the available information.

Worked Example: Dependence That Remains After Conditioning

Worked Example: Dependence That Remains After Conditioning

Let \(X=Y\) be a fair Bernoulli random variable and let \(\mathcal G=\{\varnothing,\Omega\}\), the trivial sigma-algebra. For \(A=B=\{1\}\), $$ \mathbb E[\mathbf 1_{\{X=1\}}\mathbf 1_{\{Y=1\}}\mid\mathcal G] = \mathbb P(X=1)=\tfrac12, $$ whereas $$ \mathbb E[\mathbf 1_{\{X=1\}}\mid\mathcal G]\, \mathbb E[\mathbf 1_{\{Y=1\}}\mid\mathcal G] = \tfrac12\cdot\tfrac12=\tfrac14. $$ The defining identity fails. Equivalently, the joint law puts mass \(1/2\) at \((0,0)\) and \(1/2\) at \((1,1)\); the product of the marginal laws assigns mass \(1/4\) to each of the four points. The conditional joint law is not the product of the conditional marginals.

What the Definition Does and Does Not Say

Conditional independence is a statement about the full conditional joint law, not merely about one numerical summary such as conditional covariance. In particular, checking that a covariance vanishes does not in general establish the defining identities for all bounded measurable functions.

It is also important to distinguish “independent given \(\mathcal G\)” from “independent.” The first concerns conditional laws at the level of the information in \(\mathcal G\); averaging those conditional laws can produce an unconditional joint law that is not a product, as the shared-\(Z\) example shows. Conversely, a random element measurable with respect to \(\mathcal G\) has a point-mass conditional law, and the definition still applies without requiring that element to be nonrandom.

The kernel formulation is especially useful when conditional distributions are already available: conditional independence says that the conditional joint distribution contains no further interaction beyond the two conditional marginal distributions. The expectation formulation can be more convenient when the quantities of interest are functions of the random elements. The theorem ensures that, for standard Borel state spaces, these are not competing notions but two precise descriptions of the same property.

Check Your Understanding

Use the definitions and examples above to check your understanding.

  1. State conditional independence of \(X\) and \(Y\) given \(\mathcal G\) using bounded measurable functions.
  2. What does the conditional joint-law formulation assert about \(K_{X,Y}\), \(K_X\), and \(K_Y\)?
  3. Why does the proof use countable generating pi-systems when passing from conditional-expectation identities to equality of kernels?
  4. In the shared-\(Z\) example, what calculation shows that \(X\) and \(Y\) are dependent without conditioning?
  5. Why does the equality \(X=Y\) in the final example prevent conditional independence given the trivial sigma-algebra?

Previous

Next

Back to Tutorials
Clinical Biostats