Conditional Independence Under Measurable Maps

Why Transformations Preserve Conditional Independence

Observations are often transformed before they are analyzed. A vector may be summarized by its maximum, a measurement may be thresholded into a binary indicator, or a value may be replaced by its absolute value. These operations can change the distribution of an observation, but they do not create dependence between two observations that were conditionally independent, provided each transformation uses only its own observation.

The phrase “given the same conditioning variable” is essential. Conditional independence is a statement about the joint behavior of two random elements after conditioning on \(Z\). The result below shows that separate measurable transformations preserve that statement.

Conditional Independence and Measurable Maps

Let \(X\), \(Y\), and \(Z\) be random elements taking values in measurable spaces. Write \(\sigma(X)\) for the sigma-field generated by \(X\), and similarly for \(Y\) and \(Z\). We use the event-based definition of conditional independence.

Definition: \(X\) and \(Y\) are conditionally independent given \(Z\) if, for every measurable set \(A\) in the state space of \(X\) and every measurable set \(B\) in the state space of \(Y\), \[ \mathbb{E}\bigl[\mathbf{1}_{\{X\in A\}}\mathbf{1}_{\{Y\in B\}}\mid \sigma(Z)\bigr] = \mathbb{E}\bigl[\mathbf{1}_{\{X\in A\}}\mid \sigma(Z)\bigr]\, \mathbb{E}\bigl[\mathbf{1}_{\{Y\in B\}}\mid \sigma(Z)\bigr] \quad\text{almost surely}. \]

Conditional expectations are defined up to almost-sure equality, so the null set on which this identity may fail can depend on \(A\) and \(B\). The definition requires the identity for each fixed pair of measurable sets.

Let \(f\) and \(g\) be measurable maps from the state spaces of \(X\) and \(Y\), respectively, into measurable spaces. The transformed observations are \(f(X)\) and \(g(Y)\). The central point is that every event determined by \(f(X)\) is already determined by \(X\): for a measurable set \(C\), the event \(\{f(X)\in C\}\) is the event \(\{X\in f^{-1}(C)\}\).

Theorem: If \(X\) and \(Y\) are conditionally independent given \(Z\), then \(f(X)\) and \(g(Y)\) are conditionally independent given \(Z\), for any measurable maps \(f\) and \(g\) into measurable spaces.

Proof. Fix measurable sets \(C\) and \(D\) in the respective state spaces of \(f(X)\) and \(g(Y)\). Measurability of \(f\) and \(g\) implies that \(f^{-1}(C)\) and \(g^{-1}(D)\) are measurable. Therefore, \[ \begin{aligned} &\mathbb{E}\bigl[\mathbf{1}_{\{f(X)\in C\}}\mathbf{1}_{\{g(Y)\in D\}}\mid \sigma(Z)\bigr]\\ &\qquad= \mathbb{E}\bigl[\mathbf{1}_{\{X\in f^{-1}(C)\}}\mathbf{1}_{\{Y\in g^{-1}(D)\}}\mid \sigma(Z)\bigr]. \end{aligned} \] By conditional independence of \(X\) and \(Y\), the last expression equals \[ \mathbb{E}\bigl[\mathbf{1}_{\{X\in f^{-1}(C)\}}\mid \sigma(Z)\bigr]\, \mathbb{E}\bigl[\mathbf{1}_{\{Y\in g^{-1}(D)\}}\mid \sigma(Z)\bigr]. \] Replacing each preimage event by its equivalent transformed event gives \[ \mathbb{E}\bigl[\mathbf{1}_{\{f(X)\in C\}}\mid \sigma(Z)\bigr]\, \mathbb{E}\bigl[\mathbf{1}_{\{g(Y)\in D\}}\mid \sigma(Z)\bigr]. \] This is the defining factorization for conditional independence of \(f(X)\) and \(g(Y)\) given \(Z\). Since \(C\) and \(D\) were arbitrary, the result follows.

\(\square\)

The theorem places no requirement that \(f\) or \(g\) be one-to-one. A transformation may discard information, combine coordinates, or take values in a smaller state space. What matters is that the first transformed observation is a function of \(X\) alone and the second is a function of \(Y\) alone.

Transformed Conditional Laws

When the state spaces are standard Borel, conditional distributions can be represented by probability kernels. This gives a useful distributional description of the same preservation result. Suppose \(\mu_z\) and \(\nu_z\) are conditional laws of \(X\) and \(Y\) given \(Z=z\). For a measurable map \(f\), the image of \(\mu_z\) under \(f\) is the probability measure \(f_{\#}\mu_z\) defined by \[ (f_{\#}\mu_z)(C)=\mu_z(f^{-1}(C)). \] This is also called the pushforward measure.

Result: If \(X\) and \(Y\) are conditionally independent given \(Z\), and their conditional laws are \(\mu_z\) and \(\nu_z\), then conditional laws of \(f(X)\) and \(g(Y)\) given \(Z=z\) are \(f_{\#}\mu_z\) and \(g_{\#}\nu_z\). Their joint conditional law is \[ (f_{\#}\mu_z)\otimes(g_{\#}\nu_z) \quad\text{for almost every }z. \]

Proof. Conditional independence means that the conditional joint law of \((X,Y)\) given \(Z=z\) is \(\mu_z\otimes\nu_z\) for almost every \(z\). Applying the map \((x,y)\mapsto(f(x),g(y))\) sends this product measure to the conditional joint law of \((f(X),g(Y))\). On measurable rectangles \(C\times D\), its value is \[ \begin{aligned} (\mu_z\otimes\nu_z)\bigl(f^{-1}(C)\times g^{-1}(D)\bigr) &=\mu_z(f^{-1}(C))\,\nu_z(g^{-1}(D))\\ &=(f_{\#}\mu_z)(C)\,(g_{\#}\nu_z)(D). \end{aligned} \] These are the rectangle probabilities of \((f_{\#}\mu_z)\otimes(g_{\#}\nu_z)\). Probability measures on a product of standard Borel spaces are determined by their values on measurable rectangles, so the joint conditional law is the stated product measure. \(\square\)

Worked Examples

Worked Example: Dependent Coordinates Within One Vector

Let \(B,C_1,C_2\) be independent fair Bernoulli random variables, and set \(X=(B,B)\) and \(Y=(C_1,C_2)\). Take \(Z\) to be constant. Then \(X\) and \(Y\) are independent random vectors, so they are conditionally independent given \(Z\). However, the two coordinates of \(X\) are not independent: they are equal with probability one.

Transform the vectors by \(U=X_1+X_2\) and \(V=\max(Y_1,Y_2)\). Then \(U=2B\), while \(V=1\) exactly when at least one of \(C_1,C_2\) equals one. Thus \[ \mathbb{P}(U=2)=\tfrac12,\qquad \mathbb{P}(V=1)=1-\mathbb{P}(C_1=0,C_2=0)=1-\tfrac14=\tfrac34. \] Since \(B\) is independent of \((C_1,C_2)\), \[ \mathbb{P}(U=2,V=1)=\tfrac12\cdot\tfrac34=\tfrac38. \] This equals \(\mathbb{P}(U=2)\mathbb{P}(V=1)\). The same independence for all events follows because \(U\) is a function of \(B\) and \(V\) is a function of \((C_1,C_2)\), which are independent. The calculation uses the joint law of each vector; it does not assume that the coordinates within \(X\) are independent.

Worked Example: Thresholding Categorical Observations Given a Latent Variable

Let \(Z\) take values \(0\) and \(1\). Suppose \(X\) and \(Y\), each taking values in \(\{0,1,2\}\), are conditionally independent given \(Z\), with conditional probabilities \[ \begin{array}{c|ccc|ccc} &\mathbb{P}(X=0\mid Z)&\mathbb{P}(X=1\mid Z)&\mathbb{P}(X=2\mid Z)& \mathbb{P}(Y=0\mid Z)&\mathbb{P}(Y=1\mid Z)&\mathbb{P}(Y=2\mid Z)\\ Z=0&0.5&0.3&0.2&0.2&0.5&0.3\\ Z=1&0.2&0.3&0.5&0.4&0.1&0.5 \end{array} \] Define \(U=\mathbf{1}_{\{X>0\}}\) and \(V=\mathbf{1}_{\{Y=2\}}\). For \(Z=0\), the transformed success probabilities are \[ \mathbb{P}(U=1\mid Z=0)=0.3+0.2=0.5,\qquad \mathbb{P}(V=1\mid Z=0)=0.3. \] Conditional independence gives \[ \mathbb{P}(U=1,V=1\mid Z=0)=0.5\cdot0.3=0.15. \] For \(Z=1\), they are \[ \mathbb{P}(U=1\mid Z=1)=0.3+0.5=0.8,\qquad \mathbb{P}(V=1\mid Z=1)=0.5, \] and hence \[ \mathbb{P}(U=1,V=1\mid Z=1)=0.8\cdot0.5=0.4. \] In each stratum, the joint probability is the product of the transformed marginal probabilities. Thresholding changes the conditional laws, but preserves their conditional independence.

Worked Example: Transforming Conditionally Gaussian Observations

Let \(Z\) be any real-valued random variable, and let \(E_1,E_2\) be independent standard normal random variables, independent of \(Z\). Set \(X=Z+E_1\) and \(Y=-Z+2E_2\). Given \(Z=z\), \(X\) and \(Y\) are independent, with laws \(N(z,1)\) and \(N(-z,4)\), respectively.

Consider the transformed binary observations \(U=\mathbf{1}_{\{|X|>1\}}\) and \(V=\mathbf{1}_{\{Y>0\}}\). Write \(\Phi\) for the standard normal distribution function. Conditional on \(Z=z\), standardizing \(X\) gives \[ \begin{aligned} \mathbb{P}(U=1\mid Z=z) &=\mathbb{P}(X<-1\mid Z=z)+\mathbb{P}(X>1\mid Z=z)\\ &=\Phi(-1-z)+1-\Phi(1-z). \end{aligned} \] Since \(Y=-z+2E_2\), the event \(Y>0\) is the event \(E_2>z/2\), so \[ \mathbb{P}(V=1\mid Z=z)=1-\Phi(z/2). \] The two indicators are functions of \(X\) and \(Y\), respectively. Therefore \[ \mathbb{P}(U=1,V=1\mid Z=z) = \bigl[\Phi(-1-z)+1-\Phi(1-z)\bigr]\bigl[1-\Phi(z/2)\bigr]. \] The transformed observations are conditionally independent even though the transformations are nonlinear and \(U\) discards the sign of \(X\).

What the Result Does and Does Not Say

The preservation result concerns transformations applied separately to the two conditionally independent random elements, with conditioning still on the same \(Z\). It does not say that arbitrary transformations of the pair \((X,Y)\) preserve independence. For instance, two quantities that both use \(X\) and \(Y\) need not be independent merely because \(X\) and \(Y\) are conditionally independent.

Nor does the result impose independence among the coordinates of a vector. If \(X\) is vector-valued, its coordinates may have any joint distribution allowed by its conditional law. A measurable summary \(f(X)\) uses that full joint law. When calculating a transformed probability, first determine the event in the original vector’s state space and then use the conditional law of the whole vector. Multiplying coordinate probabilities is justified only when the relevant coordinates themselves are known to be conditionally independent.

The practical benefit is that one can simplify or recode observations without losing a valid conditional-independence factorization. The conditional laws may become easier to describe after thresholding or summarizing, while the structural reason for their independence remains the factorization of the original conditional joint law.

Check Your Understanding

Use the event definition of conditional independence to answer the following.

Why is \(\{f(X)\in C\}\) an event determined by \(X\) for measurable \(f\) and \(C\)?
Does the preservation theorem require \(f\) or \(g\) to be one-to-one? Explain.
In the vector example, why is it invalid to multiply the marginal probabilities of \(X_1=1\) and \(X_2=1\) as if those coordinates were independent?
If \(\mu_z\) is a conditional law of \(X\) given \(Z=z\), how is the conditional probability \(\mathbb{P}(f(X)\in C\mid Z=z)\) expressed using \(\mu_z\)?
What condition on the inputs to two transformations allows the preservation result to apply?