Why Infinite-Family Independence Needs a Finite Definition
For a finite collection of random variables, independence is expressed by a factorization involving events chosen from each variable’s sigma-algebra. For an infinite sequence, it is tempting to ask for a similar factorization involving all variables at once. That is not the right definition: probability measures are defined on events, and an infinite intersection of events need not express the intended joint behavior.
Instead, independence of an infinite family is determined by examining every finite selection from it. This is both precise and useful. It means that finite-dimensional calculations can establish a property of an entire sequence, provided the tests cover every finite subfamily.
The same finite-subfamily convention gives the standard definition for sigma-algebras. A family \((\mathcal{F}_i)_{i\in I}\) of sigma-algebras is independent when events chosen one at a time from any finite number of distinct sigma-algebras satisfy the corresponding factorization.
Random elements are independent when their generated sigma-algebras are independent. In particular, a family of random variables is independent if the sigma-algebras each variable generates are independent. This definition requires the factorization for every finite subfamily, not merely for pairs.
Testing Independence on Generating Classes
A sigma-algebra may contain far too many events to check one by one. The useful reduction is to test the factorization on smaller classes that generate the sigma-algebras. A pi-system is a collection of sets closed under finite intersections. A lambda-system contains the whole space, is closed under complements, and is closed under countable disjoint unions. The pi-lambda theorem says that a lambda-system containing a pi-system also contains the sigma-algebra generated by that pi-system.
The hypothesis is a collection of finite event tests: for each finite selection of indices, it suffices to use events from the corresponding generating classes. Here is a proof that extends those tests to arbitrary events.
Proof. Fix any finite list of distinct indices \(i_1,\ldots,i_n\). Write \(\mathcal{F}_k=\mathcal{F}_{i_k}\) and \(\mathcal{C}_k=\mathcal{C}_{i_k}\) for this proof. We show, one coordinate at a time, that the factorization extends from \(\mathcal{C}_k\) to \(\mathcal{F}_k\).
For \(0\leq k\leq n\), consider the assertion that the factorization holds when the first \(k\) events are arbitrary members of their sigma-algebras and the remaining \(n-k\) events belong to their generating classes. The assertion for \(k=0\) is the hypothesis.
Suppose it holds for \(k-1\), where \(1\leq k\leq n\). Fix arbitrary \(E_j\in\mathcal{F}_j\) for \(1\leq j<k\), and fix \(C_j\in\mathcal{C}_j\) for \(k<j\leq n\). Let \(B\) be the intersection of all these fixed events, and let \(q\) be the product of their probabilities. If there are no fixed events, take \(B=\Omega\) and \(q=1\). Define $$ \mathcal{L} = \left\{A\in\mathcal{F}_k: \mathbb{P}(A\cap B)=\mathbb{P}(A)q\right\}. $$
This is a lambda-system. First, the induction hypothesis applied with the \(k\)-th event equal to \(\Omega\in\mathcal{C}_k\) gives \(\mathbb{P}(B)=q\), so \(\Omega\in\mathcal{L}\). If \(A\in\mathcal{L}\), then $$ \mathbb{P}(A^c\cap B) =\mathbb{P}(B)-\mathbb{P}(A\cap B) =q-\mathbb{P}(A)q =\mathbb{P}(A^c)q, $$ so \(A^c\in\mathcal{L}\). If \(A_1,A_2,\ldots\) are disjoint members of \(\mathcal{L}\), countable additivity gives $$ \mathbb{P}\left(\left(\bigcup_{m=1}^{\infty}A_m\right)\cap B\right) =\sum_{m=1}^{\infty}\mathbb{P}(A_m\cap B) =q\sum_{m=1}^{\infty}\mathbb{P}(A_m) =q\,\mathbb{P}\left(\bigcup_{m=1}^{\infty}A_m\right). $$ Thus their union belongs to \(\mathcal{L}\).
The induction hypothesis also shows that every \(C\in\mathcal{C}_k\) belongs to \(\mathcal{L}\). Since \(\mathcal{C}_k\) is a pi-system and generates \(\mathcal{F}_k\), the pi-lambda theorem gives \(\mathcal{F}_k\subseteq\mathcal{L}\). This proves the factorization with the \(k\)-th event arbitrary, completing the induction step. After \(n\) steps, all \(n\) events may be arbitrary members of their sigma-algebras. The finite list of indices was arbitrary, so the sigma-algebras are independent. \(\square\)
Worked Examples
Worked Example: Independent Bernoulli Coordinates
Let \(\Omega=\{0,1\}^{\mathbb{N}}\) carry the product probability measure under which each coordinate has probability \(1/2\) of being 1 and the coordinates are product-distributed. Define \(X_j(\omega)=\omega_j\). For each coordinate, use the generating class \(\mathcal{C}_j=\{\varnothing,\{X_j=0\},\{X_j=1\},\Omega\}\). It is a pi-system and generates \(\sigma(X_j)\).
Choose distinct coordinates \(j_1,\ldots,j_n\) and specify one value \(a_k\in\{0,1\}\) for each. The product measure assigns probability \(2^{-n}\) to this prescribed pattern, while each single-coordinate event has probability \(1/2\). Hence $$ \mathbb{P}(X_{j_1}=a_1,\ldots,X_{j_n}=a_n) =2^{-n} =\prod_{k=1}^n\mathbb{P}(X_{j_k}=a_k). $$ Allowing any of the four choices in \(\mathcal{C}_{j_k}\) preserves the factorization: an empty event makes both sides zero, and a whole-space event contributes a factor of 1. The theorem therefore shows that the entire sequence \((X_j)_{j\in\mathbb{N}}\) is independent.
Worked Example: Rational Threshold Tests for Uniform Variables
Let \((U_j)_{j\in\mathbb{N}}\) be independent uniform random variables on \([0,1]\). For each \(j\), take the pi-system $$ \mathcal{C}_j=\{\varnothing,\Omega\}\cup\{\{U_j\leq r\}:r\in\mathbb{Q}\}. $$ It generates \(\sigma(U_j)\), since rational lower thresholds generate the Borel sets of the real line. For any distinct \(j_1,\ldots,j_n\) and rational \(r_1,\ldots,r_n\), the events depend on different independent coordinates, and $$ \mathbb{P}(U_{j_k}\leq r_k) = \begin{cases} 0,&r_k<0,\\ r_k,&0\leq r_k\leq 1,\\ 1,&r_k>1. \end{cases} $$ For instance, with thresholds \(1/4,\,3/5,\,7/8\), the joint probability is $$ \mathbb{P}(U_{j_1}\leq 1/4,\ U_{j_2}\leq 3/5,\ U_{j_3}\leq 7/8) =\frac14\cdot\frac35\cdot\frac78 =\frac{21}{160}. $$ The same factorization holds for every finite selection of rational thresholds, including thresholds outside \([0,1]\) and the trivial events \(\varnothing,\Omega\). The theorem extends it to all events in the generated sigma-algebras. Rational thresholds are therefore enough to certify independence of the entire sequence.
Worked Example: Independence of Random Blocks
Again let \((X_j)_{j\in\mathbb{N}}\) be independent fair Bernoulli variables, and group them into disjoint pairs: $$ Y_m=(X_{2m-1},X_{2m}),\qquad m\in\mathbb{N}. $$ Each \(Y_m\) takes values in the four-point set \(\{0,1\}^2\). Use the class of all subsets of this finite state space as the generating class for \(\sigma(Y_m)\). For any finite set of distinct blocks, specifying a state in each block specifies values of distinct original coordinates. If \(y_k=(a_k,b_k)\), then $$ \mathbb{P}(Y_{m_1}=y_1,\ldots,Y_{m_n}=y_n) =2^{-2n} =\prod_{k=1}^n\mathbb{P}(Y_{m_k}=y_k). $$ Because every event for a block is a union of its possible states, summing this equality over the states in chosen subsets gives the factorization for arbitrary tests from the generating classes. For a concrete check, the event that \(Y_1\) has exactly one 1 has probability \(2/4=1/2\); the event \(Y_2=(0,0)\) has probability \(1/4\); and the event that the first coordinate of \(Y_3\) is 1 has probability \(1/2\). These events involve distinct blocks, and their joint probability is $$ \frac{1}{2}\cdot\frac{1}{4}\cdot\frac{1}{2}=\frac{1}{16}. $$ The theorem gives independence of the sigma-algebras generated by all the blocks, not just independence of the displayed events.
What the Finite Tests Do—and Do Not—Say
The finite-test criterion is powerful because it separates two tasks. First, verify factorization for every finite selection of indices using manageable generating classes. Second, use the theorem to extend from those classes to the full sigma-algebras. The theorem does not permit checking only a few finite selections: every finite subfamily must satisfy the required tests.
It is also important not to confuse independence of a family with pairwise independence. Pairwise tests involve only two indices at a time, whereas the definition for an infinite family requires the factorization for every finite number of distinct indices. A failure of a three-event factorization already prevents mutual independence, even if every pair passes its test.
Finally, the definition asks for finite intersections. It does not require an infinite-intersection identity. An infinite intersection may have probability zero even when every finite intersection has positive probability; for example, the probability that the first \(n\) independent fair bits all equal 1 is \(2^{-n}\), which tends to zero. Independence is therefore a statement about all finite-dimensional event tests, with the generating-class theorem providing a rigorous route from simple tests to full sigma-algebras.
Check Your Understanding
Try answering these questions using the definitions and criterion above.
- What must be checked to establish independence of a family indexed by an infinite set?
- Why do pairwise factorization tests not establish independence of an entire family?
- In the finite-test theorem, what properties must each generating class satisfy?
- How does the lambda-system argument extend a factorization from a generating class to its sigma-algebra?
- Why can rational threshold events suffice to test independence of real-valued random variables?