Independence of Events and Random Variables

What Independence Requires

Independence is a statement about how probabilities combine. For two events, it says that knowing one event occurred does not change the probability assigned to the other, in the precise sense that their intersection probability equals the product of their probabilities. For random variables, the requirement is broader: the corresponding statement must hold for every pair of measurable sets of values, not just for one observed pair.

Let \((\Omega,\mathcal F,P)\) be a probability space. Events are members of \(\mathcal F\), and their probabilities are assigned by \(P\). This tutorial develops independence at the event level and then uses that definition to make independence of random variables precise.

Definition: Two events \(A,B\in\mathcal F\) are independent if

P(A\cap B)=P(A)P(B).

If this equality does not hold, the events are dependent.

The definition includes events of probability zero or one; no division by a probability is needed. When \(P(B)>0\), independence is equivalent to \(P(A\mid B)=P(A)\), but the product definition also applies when a conditional probability given \(B\) is not defined.

Worked Example: Checking Independence on a Die

A fair six-sided die is rolled once. Let \(A\) be the event that the result is even, and let \(B\) be the event that the result exceeds 3. Determine whether \(A\) and \(B\) are independent.

The sample space is \(\{1,2,3,4,5,6\}\), with each outcome having probability \(1/6\). Thus \(A=\{2,4,6\}\) and \(B=\{4,5,6\}\), so \(P(A)=3/6=1/2\) and \(P(B)=3/6=1/2\).

Their intersection is \(A\cap B=\{4,6\}\), giving \(P(A\cap B)=2/6=1/3\). But \(P(A)P(B)=(1/2)(1/2)=1/4\), and \(1/3\ne 1/4\). Therefore \(A\) and \(B\) are dependent.

Consequences for Complements

The product definition immediately implies that independence is preserved when either or both events are replaced by their complements. This property is often useful when the event of interest is easier to describe as “not occurring.”

Theorem: If \(A\) and \(B\) are independent events, then \(A^c\) and \(B\), \(A\) and \(B^c\), and \(A^c\) and \(B^c\) are all independent.

Proof. Since \(A\) and \(B\) are independent, \(P(A\cap B)=P(A)P(B)\). Because \(B=(A\cap B)\cup(A^c\cap B)\) is a disjoint union,

\begin{aligned} P(A^c\cap B) &=P(B)-P(A\cap B)\\ &=P(B)-P(A)P(B)\\ &=(1-P(A))P(B)\\ &=P(A^c)P(B). \end{aligned}

This proves independence of \(A^c\) and \(B\). Interchanging \(A\) and \(B\) proves independence of \(A\) and \(B^c\). Applying the result once more, now to \(A^c\) and \(B\), proves independence of \(A^c\) and \(B^c\). \(\square\)

Worked Example: Using a Complement

Suppose \(P(A)=0.3\), \(P(B)=0.6\), and \(A,B\) are independent. Find \(P(A^c\cap B)\) and \(P(A^c\cap B^c)\).

By the complement result, \(A^c\) is independent of \(B\), so \(P(A^c\cap B)=P(A^c)P(B)=(1-0.3)(0.6)=0.42\).

The complements are also independent. Therefore \(P(A^c\cap B^c)=P(A^c)P(B^c)=(0.7)(1-0.6)=0.28\). As a check, the four intersections of \(A\) or \(A^c\) with \(B\) or \(B^c\) have probabilities \(0.18, 0.12, 0.42,\) and \(0.28\), which sum to 1.

Independence of Random Variables

A real-valued random variable \(X\) is a measurable function from \(\Omega\) to \(\mathbb R\). For a Borel set \(C\subseteq\mathbb R\), the event \(\{X\in C\}\) consists of outcomes whose \(X\)-value lies in \(C\). Borel sets include intervals and are closed under the set operations needed to define probability distributions.

Definition: Real-valued random variables \(X\) and \(Y\) are independent if, for every pair of Borel sets \(C,D\subseteq\mathbb R\), the events \(\{X\in C\}\) and \(\{Y\in D\}\) are independent. Equivalently,

P(X\in C,\ Y\in D)=P(X\in C)P(Y\in D) \qquad\text{for every Borel }C,D\subseteq\mathbb R.

The quantifier “for every” is essential. A value event such as \(\{X=x\}\) is just one particular measurable-set event, obtained by taking \(C=\{x\}\). Independence of the variables requires the product rule not only for one pair of values, but across all possible measurable sets of their values.

Theorem: The random variables \(X\) and \(Y\) are independent if and only if the sigma-algebras they generate, denoted \(\sigma(X)\) and \(\sigma(Y)\), are independent. Here \(\sigma(X)\) is the collection of events determined by \(X\), namely \(\{X\in C\}\) for Borel sets \(C\), and likewise for \(\sigma(Y)\).

Proof. By definition, \(\sigma(X)\) consists of the events \(\{X\in C\}\) as \(C\) ranges over Borel sets, and \(\sigma(Y)\) consists of the events \(\{Y\in D\}\) as \(D\) ranges over Borel sets. Independence of these sigma-algebras means that every event in the first is independent of every event in the second. Substituting their specified forms gives exactly the all-Borel-sets definition of independence of \(X\) and \(Y\). The two conditions are therefore equivalent. \(\square\)

Finite-Valued Variables: Checking All Value Pairs

For finite-valued variables, the all-sets definition can be checked by examining each pair of values. The reason is that any event defined by a set of possible values is a disjoint union of the corresponding single-value events.

Theorem: Suppose \(X\) takes values in a finite set \(\{x_1,\ldots,x_r\}\) and \(Y\) takes values in a finite set \(\{y_1,\ldots,y_s\}\). Then \(X\) and \(Y\) are independent if and only if

P(X=x_i,\ Y=y_j)=P(X=x_i)P(Y=y_j) \quad\text{for every }i,j.

Proof. If \(X\) and \(Y\) are independent, take \(C=\{x_i\}\) and \(D=\{y_j\}\) in the definition to obtain the displayed equalities. Conversely, suppose all those equalities hold. For any Borel sets \(C,D\), let \(I=\{i:x_i\in C\}\) and \(J=\{j:y_j\in D\}\). The events in each union below are disjoint, so finite additivity gives

\begin{aligned} P(X\in C,\ Y\in D) &=\sum_{i\in I}\sum_{j\in J}P(X=x_i,\ Y=y_j)\\ &=\sum_{i\in I}\sum_{j\in J}P(X=x_i)P(Y=y_j)\\ &=\left(\sum_{i\in I}P(X=x_i)\right) \left(\sum_{j\in J}P(Y=y_j)\right)\\ &=P(X\in C)P(Y\in D). \end{aligned}

Thus the product rule holds for every Borel \(C,D\), which is independence of \(X\) and \(Y\). \(\square\)

Worked Example: Independent Indicators From Two Coin Tosses

Toss a fair coin twice. Let \(X=1\) if the first toss is heads and \(X=0\) otherwise; define \(Y\) in the same way using the second toss. Show that \(X\) and \(Y\) are independent.

The four equally likely outcomes give \((X,Y)=(1,1),(1,0),(0,1),(0,0)\), each with probability \(1/4\). Also, \(P(X=1)=P(X=0)=P(Y=1)=P(Y=0)=1/2\). For each of the four pairs \(x,y\in\{0,1\}\), \(P(X=x,Y=y)=1/4=(1/2)(1/2)=P(X=x)P(Y=y)\).

Both variables take only the two values 0 and 1. The finite-valued theorem therefore shows that the product rule holds for every pair of Borel sets, not merely for the four single-value events. Hence \(X\) and \(Y\) are independent.

One Independent Pair Does Not Establish Independence

A single equality for a particular pair of value events can hold even when the variables are dependent. The following construction checks one such equality and then exhibits a different pair that fails the product rule.

Worked Example: A Pair of Values That Hides Dependence

Choose uniformly from eight outcomes labelled \((i,k)\), where \(i\in\{1,2,3,4\}\) and \(k\in\{1,2\}\). Define \(X=i\). Set \(Y=0\) on outcomes \((1,1),(2,1),(2,2),(4,1)\), and set \(Y=1\) on the other four outcomes.

Each value of \(X\) occurs twice, so \(P(X=i)=2/8=1/4\) for every \(i\). There are four outcomes with \(Y=0\), so \(P(Y=0)=4/8=1/2\). The event \(\{X=1,Y=0\}\) contains only \((1,1)\), and thus has probability \(1/8\). This equals \(P(X=1)P(Y=0)=(1/4)(1/2)=1/8\).

That one matching equality does not prove independence. For \(X=2\), both outcomes have \(Y=0\), so \(P(X=2,Y=0)=2/8=1/4\). But \(P(X=2)P(Y=0)=(1/4)(1/2)=1/8\). Since these probabilities differ, \(X\) and \(Y\) are dependent.

Common Mistake and Why Independence Matters

Wrong reasoning: “The events \(\{X=x\}\) and \(\{Y=y\}\) satisfy the product rule, so \(X\) and \(Y\) are independent.” This checks only one pair of sets. Correction: independence of variables requires the product rule for every pair of Borel sets; for finite-valued variables, that means checking every pair of possible values, not just one.

Independence is a strong structural assumption: it says that events described using one variable and events described using the other combine by multiplication, no matter which measurable ranges are chosen. The event definition makes that claim exact and also clarifies its limits. Observing one coincidental product equality—or one particular outcome—does not establish the all-sets property.

Key takeaway: Two events are independent when their intersection probability is the product of their probabilities. Two random variables are independent only when this event-level condition holds for every pair of measurable sets of values.

Check Your Understanding

Use the all-sets definition whenever the question concerns independence of random variables.

For two events \(A,B\), state the equation that defines independence.
If \(A\) and \(B\) are independent, what is \(P(A^c\cap B)\) in terms of \(P(A)\) and \(P(B)\)?
In the definition of independence of \(X\) and \(Y\), what sets range over \(C\) and \(D\)?
Why does checking all value pairs suffice for finite-valued \(X\) and \(Y\)?
In the eight-outcome example, which value pair demonstrates that \(X\) and \(Y\) are dependent?