Tutorials › Probability Theory (Graduate) › Independence of Sigma-Algebras and Event Factorization

Home  /  Tutorials  /  Independence and Zero-One Laws

Independence and Zero-One Laws · Tutorial 206 of 946

Independence of Sigma-Algebras and Event Factorization

Learn to test independence by factoring probabilities of events and to state the corresponding condition for finite collections of sigma-algebras.

Intermediate 9 min read

What You'll Learn

  • Define independence of two sigma-algebras on a probability space.
  • Express independence as factorization of probabilities for every event pair.
  • State the mutual independence condition for a finite family of sigma-algebras.
  • Check independence in finite product spaces by calculating event probabilities.
  • Recognize how independence behaves when sigma-algebras are reduced or trivial.

Why Independence Is a Condition on Events

Independence is often first expressed as a statement about events: learning that one event occurred does not change the probability assigned to another. For sigma-algebras, the same idea must apply not just to one selected pair of events, but to every event measurable with respect to each sigma-algebra.

This distinction matters because a sigma-algebra records a whole collection of events. Independence of two sigma-algebras is therefore a uniform factorization condition. It is not enough to check one convenient event pair.

Throughout, let \((\Omega,\mathcal F,\mathbb P)\) be a probability space. Recall that a sigma-algebra is a collection of subsets of \(\Omega\) containing \(\Omega\) and closed under complements and countable unions.

Definition: Two sub-sigma-algebras \(\mathcal G,\mathcal H\subseteq\mathcal F\) are independent if, for every \(A\in\mathcal G\) and every \(B\in\mathcal H\), the probability of their intersection equals the product of their probabilities. In symbols, $$ \mathbb P(A\cap B)=\mathbb P(A)\mathbb P(B) \qquad (A\in\mathcal G,\ B\in\mathcal H). $$

The quantifiers are essential: the identity must hold for all choices of \(A\) and \(B\) from the respective sigma-algebras. A single successful calculation establishes factorization for that pair only.

What the Definition Requires

The condition includes events that are empty or certain, as well as events with probabilities strictly between zero and one. These cases are consistent with the definition. For example, if \(A=\varnothing\), then both sides of the identity are zero; if \(A=\Omega\), both sides equal \(\mathbb P(B)\).

For events of positive probability, the familiar conditional-probability interpretation follows directly from the definition. If \(\mathbb P(A)>0\), then independence gives $$ \mathbb P(B\mid A) =\frac{\mathbb P(A\cap B)}{\mathbb P(A)} =\mathbb P(B). $$ The sigma-algebra definition does not require conditional probability, however, and it also applies when some events have probability zero.

Independence is symmetric: if \(\mathcal G\) is independent of \(\mathcal H\), then \(\mathcal H\) is independent of \(\mathcal G\). Indeed, \(A\cap B=B\cap A\), and multiplication of real numbers is commutative.

Worked Example: Two Independent Coin Coordinates

Worked Example: Two Independent Coin Coordinates

Let \(\Omega=\{H,T\}^2\), with each of the four outcomes assigned probability \(1/4\). Let \(\mathcal G\) consist of events determined by the first coordinate, and let \(\mathcal H\) consist of events determined by the second coordinate. We check that these sigma-algebras are independent.

Each of \(\mathcal G\) and \(\mathcal H\) contains four events: the empty set, the whole space, and the two events specifying the corresponding coordinate. Write \(A\in\mathcal G\) as the event that the first coordinate belongs to a subset \(S\subseteq\{H,T\}\), and \(B\in\mathcal H\) as the event that the second coordinate belongs to \(R\subseteq\{H,T\}\). Then $$ \mathbb P(A)=\frac{|S|}{2}, \qquad \mathbb P(B)=\frac{|R|}{2}, \qquad \mathbb P(A\cap B)=\frac{|S||R|}{4}. $$ There are \(|S||R|\) outcomes in the intersection, each of probability \(1/4\). Thus $$ \mathbb P(A\cap B) =\frac{|S||R|}{4} =\frac{|S|}{2}\frac{|R|}{2} =\mathbb P(A)\mathbb P(B). $$ This calculation applies to every choice of \(S\) and \(R\), including the empty and full subsets, so it verifies independence of the two sigma-algebras.

Independence for a Finite Family

The definition extends to any finite number of sigma-algebras by requiring the analogous factorization for one event chosen from each sigma-algebra at once. This is called mutual independence of the family.

Definition: A finite family \(\mathcal G_1,\ldots,\mathcal G_n\) of sub-sigma-algebras of \(\mathcal F\) is mutually independent if, for every choice of events \(A_i\in\mathcal G_i\), one for each \(i=1,\ldots,n\), the probability of their common intersection is the product of their probabilities. In symbols, $$ \mathbb P\left(\bigcap_{i=1}^n A_i\right) =\prod_{i=1}^n\mathbb P(A_i) \qquad (A_i\in\mathcal G_i,\ 1\le i\le n). $$

For \(n=2\), this is precisely the definition of independence for two sigma-algebras. For larger \(n\), the condition concerns a simultaneous choice of events from all the sigma-algebras. When using the definition, specify the number of sigma-algebras and check the identity for arbitrary events from each one, not merely for a selected set of examples.

Worked Example: Three Coordinates

Worked Example: Three Coordinates

Let \(\Omega=\{0,1\}^3\), with each of its eight outcomes having probability \(1/8\). For \(i=1,2,3\), let \(\mathcal G_i\) be the sigma-algebra determined by coordinate \(i\). We verify mutual independence.

For each \(i\), an event \(A_i\in\mathcal G_i\) specifies that coordinate \(i\) lies in some subset \(S_i\subseteq\{0,1\}\). The intersection \(A_1\cap A_2\cap A_3\) contains exactly \(|S_1||S_2||S_3|\) outcomes. Consequently, $$ \mathbb P(A_1\cap A_2\cap A_3) =\frac{|S_1||S_2||S_3|}{8} =\frac{|S_1|}{2}\frac{|S_2|}{2}\frac{|S_3|}{2}. $$ For each \(i\), \(\mathbb P(A_i)=|S_i|/2\). Substituting these probabilities into the last expression gives $$ \mathbb P(A_1\cap A_2\cap A_3) =\mathbb P(A_1)\mathbb P(A_2)\mathbb P(A_3). $$ The calculation holds for every choice of the three subsets, including empty subsets, so the three sigma-algebras are mutually independent.

Consequences of the Definition

Two useful facts follow directly from the event-level condition. First, if independent sigma-algebras are replaced by smaller sigma-algebras, independence remains true. Second, a sigma-algebra containing only the empty event and the whole space is independent of every sub-sigma-algebra.

Theorem: Suppose \(\mathcal G\) and \(\mathcal H\) are independent sub-sigma-algebras of \(\mathcal F\). If \(\mathcal G'\subseteq\mathcal G\) and \(\mathcal H'\subseteq\mathcal H\) are also sigma-algebras, then \(\mathcal G'\) and \(\mathcal H'\) are independent.
Proof. Take arbitrary \(A\in\mathcal G'\) and \(B\in\mathcal H'\). The inclusions give \(A\in\mathcal G\) and \(B\in\mathcal H\). Since \(\mathcal G\) and \(\mathcal H\) are independent, $$ \mathbb P(A\cap B)=\mathbb P(A)\mathbb P(B). $$ This holds for every such \(A\) and \(B\), which is exactly the definition of independence for \(\mathcal G'\) and \(\mathcal H'\). \(\square\)

This result is often useful when only part of the information recorded by a sigma-algebra is under consideration. Independence is preserved when the available collections of events are reduced.

Theorem: Let \(\mathcal T=\{\varnothing,\Omega\}\). For every sub-sigma-algebra \(\mathcal G\subseteq\mathcal F\), the sigma-algebras \(\mathcal T\) and \(\mathcal G\) are independent.
Proof. Choose \(A\in\mathcal T\) and \(B\in\mathcal G\). If \(A=\varnothing\), then $$ \mathbb P(A\cap B)=0=\mathbb P(A)\mathbb P(B). $$ If \(A=\Omega\), then $$ \mathbb P(A\cap B)=\mathbb P(B)=1\cdot\mathbb P(B) =\mathbb P(A)\mathbb P(B). $$ These are all possibilities for \(A\), so the factorization holds for every \(A\in\mathcal T\) and \(B\in\mathcal G\). \(\square\)

Worked Example: Shared Information Prevents Independence

Worked Example: Shared Information Prevents Independence

Let \(\Omega=\{H,T\}\), with each outcome having probability \(1/2\). Set \(A=\{H\}\), and let \(\mathcal G=\sigma(A)\) and \(\mathcal H=\sigma(A)\). Both sigma-algebras contain \(A\). Taking this event from each gives $$ \mathbb P(A\cap A)=\mathbb P(A)=\frac12, \qquad \mathbb P(A)\mathbb P(A)=\frac14. $$ The two quantities differ, so \(\mathcal G\) and \(\mathcal H\) are not independent. The failure is already visible for one event pair; no further checks are needed to disprove the universal condition in the definition.

Factorization and Finite Unions

The defining identity can also be applied to finite unions of disjoint events within each sigma-algebra. This follows from finite additivity and helps explain why the event condition accommodates events built from several elementary outcomes.

Proposition: Let \(\mathcal G_1,\ldots,\mathcal G_n\) be mutually independent sigma-algebras. For each \(i\), suppose \(A_{i,1},\ldots,A_{i,m_i}\) are pairwise disjoint events in \(\mathcal G_i\), where \(m_i\) is a positive integer. Then $$ \mathbb P\left(\bigcap_{i=1}^n\bigcup_{j=1}^{m_i}A_{i,j}\right) = \prod_{i=1}^n\mathbb P\left(\bigcup_{j=1}^{m_i}A_{i,j}\right). $$
Proof. Distributing the finite intersections over the finite unions gives $$ \bigcap_{i=1}^n\bigcup_{j=1}^{m_i}A_{i,j} = \bigcup_{(j_1,\ldots,j_n)} \left(\bigcap_{i=1}^n A_{i,j_i}\right), $$ where the union is over all choices \(1\le j_i\le m_i\). The events in this union are pairwise disjoint: if two index tuples differ, then for some coordinate \(i\) they use distinct, disjoint events from the family \(A_{i,1},\ldots,A_{i,m_i}\). Finite additivity and mutual independence therefore yield $$ \mathbb P\left(\bigcap_{i=1}^n\bigcup_{j=1}^{m_i}A_{i,j}\right) = \sum_{(j_1,\ldots,j_n)} \prod_{i=1}^n\mathbb P(A_{i,j_i}). $$ Expanding the finite product of sums gives $$ \sum_{(j_1,\ldots,j_n)} \prod_{i=1}^n\mathbb P(A_{i,j_i}) = \prod_{i=1}^n\left(\sum_{j=1}^{m_i}\mathbb P(A_{i,j})\right). $$ Within each \(i\)-th family the events are pairwise disjoint, so finite additivity gives $$ \sum_{j=1}^{m_i}\mathbb P(A_{i,j}) = \mathbb P\left(\bigcup_{j=1}^{m_i}A_{i,j}\right). $$ Substituting this equality for each factor proves the result. \(\square\)

A Common Pitfall

The event-factorization definition is exacting but straightforward: quantify over every event in each sigma-algebra. A common error is to verify the identity only for a few events and conclude that the sigma-algebras are independent. Such checks can illustrate the condition, but independence requires the full universal statement.

For a finite family, keep the number of events in the intersection aligned with the number of sigma-algebras: one event is selected from each sigma-algebra. The factorization is then one identity involving their joint intersection. The definition itself does not assert anything about arbitrary events that do not belong to the specified sigma-algebras.

Independence of sigma-algebras is a basic language for describing separate collections of measurable information. Its central test is event factorization, and its finite-family version expresses the same requirement across several collections at once.

Check Your Understanding

Answer these questions using the event-factorization definition.

  1. State the condition required for two sub-sigma-algebras to be independent.
  2. What changes in the definition when considering a finite family of three sigma-algebras?
  3. In the two-coordinate coin example, why does counting \(|S||R|\) outcomes give the intersection probability?
  4. Why is it enough to find one event pair that fails factorization to conclude that two sigma-algebras are not independent?
  5. Explain why replacing independent sigma-algebras by sub-sigma-algebras preserves independence.

Previous

Next

Back to Tutorials
Clinical Biostats