Independence of Random Elements Through Measurable Sets

Independence as a Statement About Measurable Sets

Independence is often introduced through events: two events are independent when the probability of their intersection is the product of their probabilities. Random elements are measurable maps, so the events they determine arise as inverse images of measurable sets in their state spaces. This gives a precise way to express independence that works for real-valued variables, vectors, and random elements taking values in more general measurable spaces.

The important point is the quantifier: independence requires factorization for every measurable set in each state space. A few successful calculations may provide evidence, but the definition itself ranges over the whole collection.

Random Elements and Their Generated Sigma-Algebras

A measurable space is a pair \((E,\mathcal E)\), where \(E\) is a set and \(\mathcal E\) is a sigma-algebra of subsets of \(E\). Given a probability space \((\Omega,\mathcal F,P)\), a random element with values in \((E,\mathcal E)\) is a measurable map \(X:\Omega\to E\). Measurability means that \(X^{-1}(A)\in\mathcal F\) for every \(A\in\mathcal E\).

Definition: The sigma-algebra generated by a random element \(X:\Omega\to E\) is \(\sigma(X)=\{X^{-1}(A):A\in\mathcal E\}\). Because \(X\) is measurable, this collection is a sigma-algebra contained in \(\mathcal F\).

For \(A\in\mathcal E\), the event \(X^{-1}(A)\) is also written \(\{X\in A\}\). Thus \(P(X\in A)\) means \(P(X^{-1}(A))\). A real-valued random variable is a random element of \((\mathbb R,\mathcal B(\mathbb R))\), where \(\mathcal B(\mathbb R)\) is the Borel sigma-algebra. A random vector in \(\mathbb R^d\) is treated in the same way, using \(\mathcal B(\mathbb R^d)\).

Recall that two sigma-algebras \(\mathcal G_1,\mathcal G_2\subseteq\mathcal F\) are independent if \(P(G_1\cap G_2)=P(G_1)P(G_2)\) for every \(G_1\in\mathcal G_1\) and \(G_2\in\mathcal G_2\). Independence of random elements is independence of their generated sigma-algebras. The following theorem translates that definition into the state spaces.

The Measurable-Set Criterion

Theorem: Let \(X:\Omega\to E\) and \(Y:\Omega\to F\) be random elements taking values in measurable spaces \((E,\mathcal E)\) and \((F,\mathcal H)\). Then \(X\) and \(Y\) are independent if and only if \[ P(X\in A,\ Y\in B)=P(X\in A)P(Y\in B) \] for every \(A\in\mathcal E\) and \(B\in\mathcal H\).

Proof. By definition, \(X\) and \(Y\) are independent exactly when \(\sigma(X)\) and \(\sigma(Y)\) are independent. Every event in \(\sigma(X)\) has the form \(X^{-1}(A)\) for some \(A\in\mathcal E\), and every event in \(\sigma(Y)\) has the form \(Y^{-1}(B)\) for some \(B\in\mathcal H\). Therefore independence of the two sigma-algebras is exactly the assertion that \(P(X^{-1}(A)\cap Y^{-1}(B))=P(X^{-1}(A))P(Y^{-1}(B))\) for all such \(A,B\). Rewriting these inverse images as events of the form \(\{X\in A\}\) and \(\{Y\in B\}\) gives the stated criterion. \(\square\)

This result is simple, but it is useful because it makes the scope of the definition explicit. The sets \(A\) and \(B\) are selected from the measurable spaces where the random elements take their values; they need not be singletons, intervals, or sets of any special geometric form.

Worked Example: Two Independent Bits

Let \(\Omega=\{(0,0),(0,1),(1,0),(1,1)\}\), with each outcome assigned probability \(1/4\). Define \(X(i,j)=i\) and \(Y(i,j)=j\). These are real-valued random variables. To verify independence, take any Borel sets \(A,B\subseteq\mathbb R\), and put \(A_0=A\cap\{0,1\}\) and \(B_0=B\cap\{0,1\}\). Write \(a=|A_0|\) and \(b=|B_0|\). Each of the four pairs occurs with probability \(1/4\), so \(P(X\in A)=a/2\) and \(P(Y\in B)=b/2\). There are exactly \(ab\) pairs \((i,j)\) with \(i\in A_0\) and \(j\in B_0\), so \(P(X\in A,Y\in B)=ab/4=(a/2)(b/2)\). The factorization holds for arbitrary Borel \(A\) and \(B\), including sets that contain neither, one, or both of the possible values. The criterion proves that \(X\) and \(Y\) are independent.

Real-Valued Variables: A Measurable Set Can Reveal Dependence

For real-valued random variables, the criterion ranges over Borel subsets of \(\mathbb R\). It is often enough to disprove independence by finding just one pair of Borel sets for which factorization fails: the criterion requires factorization for all pairs, so one failure rules it out.

Worked Example: A Variable and Its Reflection Are Dependent

Let \(\Omega=[0,1]\) with Lebesgue probability measure, let \(X(\omega)=\omega\), and let \(Y(\omega)=1-\omega\). Both are Borel-measurable real-valued random variables. Choose the Borel set \(A=B=[0,1/2]\). Since \(X\) is uniform on \([0,1]\), \(P(X\in A)=1/2\). Also \(Y\) is uniform on \([0,1]\), so \(P(Y\in B)=1/2\).

The event \(\{X\in A,Y\in B\}\) requires both \(\omega\leq 1/2\) and \(1-\omega\leq 1/2\). The second inequality is equivalent to \(\omega\geq 1/2\), so the intersection is the singleton \(\{1/2\}\). Its Lebesgue probability is zero. Consequently, \(P(X\in A,Y\in B)=0\), whereas \(P(X\in A)P(Y\in B)=1/4\). The measurable-set criterion fails, and \(X,Y\) are not independent.

This example also illustrates why marginal distributions alone do not determine independence. Both variables have the same uniform marginal distribution, but their values are linked by the identity \(Y=1-X\). Independence concerns the joint behavior, tested here by the probabilities of simultaneous membership in measurable sets.

Finite Families of Random Elements

The same description applies to any finite family. A family of sigma-algebras is independent when every finite choice of one event from each sigma-algebra has an intersection probability equal to the product of the individual probabilities. For a finite family of random elements, each such event is an inverse image of a measurable set.

Theorem: Let \(n\geq 2\), and for each \(i\in\{1,\ldots,n\}\), let \(X_i:\Omega\to E_i\) be a random element with values in \((E_i,\mathcal E_i)\). The random elements \(X_1,\ldots,X_n\) are mutually independent if and only if, for every choice \(A_i\in\mathcal E_i\), \[ P\left(\bigcap_{i=1}^{n}\{X_i\in A_i\}\right) =\prod_{i=1}^{n}P(X_i\in A_i). \]

Proof. Mutual independence of the random elements means that the sigma-algebras \(\sigma(X_1),\ldots,\sigma(X_n)\) are mutually independent. By definition, this requires the factorization \(P(\bigcap_{i=1}^{n}G_i)=\prod_{i=1}^{n}P(G_i)\) for every choice \(G_i\in\sigma(X_i)\). Each \(G_i\) is an inverse image \(X_i^{-1}(A_i)\) for some \(A_i\in\mathcal E_i\), and every such inverse image belongs to \(\sigma(X_i)\). Substituting \(G_i=\{X_i\in A_i\}\) gives the displayed condition in both directions. \(\square\)

For a family of more than two random elements, checking factorization only for pairs is not, by itself, the definition of mutual independence. The finite-family condition concerns intersections involving one event from every selected sigma-algebra. It is this joint requirement that the displayed criterion records.

Worked Example: Independence of Two Random Vectors

Let \(\Omega=\{0,1\}^4\), with all \(16\) outcomes equally likely. Write an outcome as \((\omega_1,\omega_2,\omega_3,\omega_4)\), and define the random vectors \(U=(\omega_1,\omega_2)\) and \(V=(\omega_3,\omega_4)\), each taking values in \(\mathbb R^2\). Take arbitrary Borel sets \(C,D\subseteq\mathbb R^2\). Put \(C_0=C\cap\{0,1\}^2\), \(D_0=D\cap\{0,1\}^2\), and let \(c=|C_0|\), \(d=|D_0|\).

Exactly \(4d\) of the \(16\) outcomes have \(V\in D\), and exactly \(4c\) have \(U\in C\). Hence \(P(U\in C)=c/4\) and \(P(V\in D)=d/4\). For the simultaneous event, the first two coordinates can be chosen in \(c\) ways and the last two in \(d\) ways. Thus there are \(cd\) qualifying outcomes and \(P(U\in C,V\in D)=cd/16=(c/4)(d/4)\). This holds for every pair of Borel sets \(C,D\), so the two random vectors are independent.

What the Criterion Does—and Does Not—Say

For a random vector, a measurable set in its state space may describe several coordinates at once. For example, a Borel subset of \(\mathbb R^2\) may impose a condition on both coordinates together. The criterion for independence of two random vectors tests such sets for each vector, rather than replacing a vector by one of its coordinates. This is why stating the state space and its sigma-algebra matters.

There is also a useful distinction between verifying a definition and disproving it. To prove independence directly from the criterion, one must establish the factorization for every pair of measurable sets (or use an applicable theorem that reduces the verification). To disprove independence, one counterexample pair suffices, as the reflected-variable example shows.

Key takeaway: Random elements \(X\) and \(Y\) are independent exactly when the probabilities of their inverse-image events factor for every measurable \(A\) and \(B\) in their respective state spaces. For real-valued variables and random vectors, those sets are Borel sets.

Check Your Understanding

Use the measurable-set criterion in answering the following questions.

For a measurable map \(X:\Omega\to E\), how is the event \(\{X\in A\}\) related to \(X^{-1}(A)\)?
What quantifier over measurable sets appears in the criterion for two independent random elements?
In the reflection example, which two Borel sets demonstrate that factorization fails?
For two random vectors, what state space and sigma-algebra supply the measurable sets in the criterion?
Why does finding one pair of measurable sets with nonfactorizing probabilities prove dependence?