Tutorials › Measure Theory › Independence of Random Variables

Home  /  Tutorials  /  Probability Spaces and Random Variables

Probability Spaces and Random Variables · Tutorial 737 of 798

Independence of Random Variables

Learn how independence of random variables is expressed by events, Borel preimages, and distributional factorization.

Intermediate 10 min read

What You'll Learn

  • Define independence of sigma-algebras and random variables.
  • Express independence using probabilities of Borel preimages.
  • Relate independence to joint laws and distribution functions.
  • Check independence explicitly in finite probability models.
  • Identify why verifying just one pair of events is not enough.
  • Prove that measurable transformations preserve independence.

From Independent Events to Independent Random Variables

Independence of random variables is a statement about the events they determine. It is not enough to compare their values at one outcome, or to test one pair of threshold events: the required factorization must hold for every relevant pair of measurable events. The sigma-algebra formulation makes this requirement precise, while the Borel-preimage formulation connects it directly to distributions.

Throughout, \((\Omega,\mathcal{F},\mathbb{P})\) is a probability space, so \(\mathbb{P}(\Omega)=1\). Random variables in this tutorial are real-valued measurable maps \(X:\Omega\to\mathbb{R}\). Equalities of probabilities are exact; no almost-sure qualification is needed for events. Recall that two events \(A,B\in\mathcal{F}\) are independent when \(\mathbb{P}(A\cap B)=\mathbb{P}(A)\mathbb{P}(B)\).

Definition: The sigma-algebra generated by a real-valued random variable \(X\) is \(\sigma(X)=\{X^{-1}(C):C\in\mathcal{B}_{\mathbb{R}}\}\), where \(\mathcal{B}_{\mathbb{R}}\) is the Borel sigma-algebra. Two sigma-algebras \(\mathcal{G},\mathcal{H}\subseteq\mathcal{F}\) are independent if \(\mathbb{P}(G\cap H)=\mathbb{P}(G)\mathbb{P}(H)\) for every \(G\in\mathcal{G}\) and \(H\in\mathcal{H}\). The random variables \(X,Y\) are independent if \(\sigma(X)\) and \(\sigma(Y)\) are independent.

For a finite collection \(X_1,\ldots,X_n\), mutual independence means that for every choice of Borel sets \(C_1,\ldots,C_n\subseteq\mathbb{R}\),

$$ \mathbb{P}\left(\bigcap_{i=1}^{n}\{X_i\in C_i\}\right) =\prod_{i=1}^{n}\mathbb{P}(X_i\in C_i). $$

For an infinite collection, the corresponding requirement is imposed on every finite subcollection. This is mutual independence, not merely pairwise independence: pairwise independence checks only each pair of variables separately.

Borel Preimages and Distributional Statements

The definition using generated sigma-algebras has a direct event form. For any Borel sets \(C,D\), the events \(\{X\in C\}=X^{-1}(C)\) and \(\{Y\in D\}=Y^{-1}(D)\) belong to \(\sigma(X)\) and \(\sigma(Y)\). Thus independence of the sigma-algebras implies

$$ \mathbb{P}(X\in C,\ Y\in D) =\mathbb{P}(X\in C)\mathbb{P}(Y\in D) \qquad(C,D\in\mathcal{B}_{\mathbb{R}}). $$

Conversely, this Borel-preimage condition implies independence of \(\sigma(X)\) and \(\sigma(Y)\). This converse is the standard pi-lambda theorem criterion for independence: the Borel preimages form generating families, and the factorization extends from those families to the generated sigma-algebras. Its full proof belongs with the later treatment of pi-system criteria for independence. Consequently, either formulation can be used to define independence.

The law of \(X\) is the probability measure \(\mathbb{P}_X\) on \(\mathcal{B}_{\mathbb{R}}\) given by \(\mathbb{P}_X(C)=\mathbb{P}(X\in C)\). The joint law \(\mathbb{P}_{(X,Y)}\) is the law on \(\mathcal{B}_{\mathbb{R}^2}\) of the map \(\omega\mapsto(X(\omega),Y(\omega))\). Independence says that its probabilities on Borel rectangles factor:

$$ \mathbb{P}_{(X,Y)}(C\times D)=\mathbb{P}_X(C)\mathbb{P}_Y(D). $$

Equivalently, the joint law is the product probability measure \(\mathbb{P}_X\times\mathbb{P}_Y\). The implication from rectangle factorization to equality of these measures uses the standard uniqueness theorem on a generating pi-system; its proof is deferred to the later treatment of uniqueness of probability laws. For real-valued \(X,Y\), a familiar special case uses their joint and marginal distribution functions. Independence implies, for all \(s,t\in\mathbb{R}\),

$$ \mathbb{P}(X\le s,\ Y\le t) =\mathbb{P}(X\le s)\mathbb{P}(Y\le t). $$

The converse also holds when this identity is verified for every pair \(s,t\); it follows from the same standard pi-system criterion, not from checking just a single threshold pair.

Worked Example: Two Independent Coordinates on Four Outcomes

Let \(\Omega=\{(0,0),(0,1),(1,0),(1,1)\}\), let \(\mathcal{F}\) be its power set, and assign probability \(1/4\) to each outcome. Define \(X(i,j)=i\) and \(Y(i,j)=j\). Verify independence for arbitrary Borel preimages.

For any Borel set \(C\subseteq\mathbb{R}\), the preimage \(\{X\in C\}\) is determined by which of \(0,1\) belong to \(C\). If neither belongs, its probability is \(0\); if just one belongs, its probability is \(2/4=1/2\); if both belong, its probability is \(4/4=1\). The same three possibilities apply to \(\{Y\in D\}\) for any Borel \(D\).

Write \(r=|\{0,1\}\cap C|\) and \(q=|\{0,1\}\cap D|\), where each of \(r,q\) is \(0,1,\) or \(2\). The event \(\{X\in C,Y\in D\}\) consists of exactly the \(rq\) outcome pairs whose first coordinate is in \(C\) and second coordinate is in \(D\). Its probability is \(rq/4\). The marginal probabilities are \(r/2\) and \(q/2\), whose product is \(rq/4\). Thus the Borel-preimage identity holds for every \(C,D\). Check: for \(C=\{1\}\) and \(D=\{0\}\), the intersection is the single outcome \((1,0)\), with probability \(1/4\); the product of the marginal probabilities is \((2/4)(2/4)=1/4\).

Worked Example: Identical Uniform Random Variables Are Dependent

On \(\Omega=[0,1]\) with its Lebesgue sigma-algebra and probability measure given by Lebesgue measure restricted to \([0,1]\), let \(X(\omega)=\omega\) and \(Y(\omega)=\omega\). Both are real-valued random variables with the uniform law on \([0,1]\). Test the Borel sets \(C=D=(-\infty,1/2]\).

The event \(\{X\in C\}\) is \([0,1/2]\), of probability \(1/2\), and \(\{Y\in D\}\) is the same event, also of probability \(1/2\). Their intersection is \([0,1/2]\), so

$$ \mathbb{P}(X\in C,\ Y\in D)=\frac12 \qquad\text{but}\qquad \mathbb{P}(X\in C)\mathbb{P}(Y\in D)=\frac12\cdot\frac12=\frac14. $$

The factorization fails, so \(X\) and \(Y\) are not independent. Check: their joint distribution function at \(s=t=1/2\) is \(1/2\), whereas the product of the marginal distribution functions there is \(1/2\cdot1/2=1/4\). The closed endpoint does not change these values, since a singleton has Lebesgue measure zero.

Measurable Transformations Preserve Independence

Proposition: Let \(X_1,\ldots,X_n\) be mutually independent real-valued random variables on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). For each \(i\), let \(g_i:\mathbb{R}\to\mathbb{R}\) be Borel measurable. Then \(g_1(X_1),\ldots,g_n(X_n)\) are mutually independent.

Proof. Fix arbitrary Borel sets \(C_1,\ldots,C_n\subseteq\mathbb{R}\). Since each \(g_i\) is Borel measurable, \(g_i^{-1}(C_i)\) is Borel. For each \(i\), the event \(\{g_i(X_i)\in C_i\}\) equals \(\{X_i\in g_i^{-1}(C_i)\}\). Applying mutual independence of \(X_1,\ldots,X_n\) to these Borel preimages gives

$$ \mathbb{P}\left(\bigcap_{i=1}^{n}\{g_i(X_i)\in C_i\}\right) =\prod_{i=1}^{n}\mathbb{P}(X_i\in g_i^{-1}(C_i)) =\prod_{i=1}^{n}\mathbb{P}(g_i(X_i)\in C_i). $$

This is the defining Borel-preimage identity for mutual independence of the transformed variables. \(\square\)

For example, applying separate thresholding or other Borel functions to mutually independent variables cannot introduce dependence between them. The requirement that each transformation acts on its own variable matters: a function of two variables, such as \(X_1+X_2\), is not covered by this proposition as a transformation of one coordinate.

Worked Example: Independent, Unequally Distributed Binary Variables

Let \(\Omega=\{(0,0),(0,1),(1,0),(1,1)\}\), with all subsets measurable. Assign probabilities \(1/2,1/6,1/4,1/12\), respectively, to these four outcomes, and again set \(X(i,j)=i\), \(Y(i,j)=j\). Check the joint probabilities against the marginal laws.

The total probability is \(1/2+1/6+1/4+1/12=6/12+2/12+3/12+1/12=1\). The marginal probabilities are \(\mathbb{P}(X=0)=1/2+1/6=2/3\), \(\mathbb{P}(X=1)=1/4+1/12=1/3\), \(\mathbb{P}(Y=0)=1/2+1/4=3/4\), and \(\mathbb{P}(Y=1)=1/6+1/12=1/4\).

For the four possible coordinate pairs, the products of the marginals are \((2/3)(3/4)=1/2\) for \((0,0)\), \((2/3)(1/4)=1/6\) for \((0,1)\), \((1/3)(3/4)=1/4\) for \((1,0)\), and \((1/3)(1/4)=1/12\) for \((1,1)\). These are exactly the assigned joint probabilities. Since every Borel preimage under either binary-valued variable selects a subset of \(\{0,1\}\), adding the verified point probabilities gives the required factorization for arbitrary Borel sets. Check: the four assigned probabilities sum to \(1/2+1/6+1/4+1/12=1\), and the product probabilities sum to \((2/3+1/3)(3/4+1/4)=1\).

A Common Mistake: Testing Only One Pair of Events

To establish independence, the factorization must hold for every pair of Borel sets (equivalently, every pair of events from the generated sigma-algebras). A single successful test proves only that particular pair of events is independent. For the identical uniform variables in the earlier example, take instead \(C=D=(-\infty,0]\). Both preimages are empty, so the intersection probability is \(0\) and the product of the marginal probabilities is \(0\). This one test passes. But the test at \(C=D=(-\infty,1/2]\) gives \(1/2\ne1/4\), so the variables are dependent.

A related error is to identify independence with a statement about the values of the variables at a particular outcome. Independence is a property of probabilities of events across the probability space. The Borel-preimage definition gives the exact test, while the sigma-algebra definition records all events observable from each variable.

Check Your Understanding

For each question, distinguish carefully between a single event calculation and the full independence requirement.

  1. State the definition of independence for two sigma-algebras and give the Borel-preimage condition for two real-valued random variables.
  2. For the identical uniform variables \(X=Y\) on \([0,1]\), calculate the joint and product probabilities at the threshold \(1/2\). What does the mismatch establish?
  3. Why does independence of \(X_1,\ldots,X_n\) imply independence of \(g_1(X_1),\ldots,g_n(X_n)\) when the \(g_i\) are Borel measurable?
  4. In the four-outcome unequal binary example, compute \(\mathbb{P}(X=0,Y=1)\) and compare it with \(\mathbb{P}(X=0)\mathbb{P}(Y=1)\).
  5. Explain why verifying factorization for one pair of Borel sets does not establish independence.

Previous

Next

Back to Tutorials