Why Independence Is About Relationships
Two random variables can describe the same chance experiment in different ways. For example, one variable might record the result of a first coin toss while another records the result of a second toss. To say these variables are independent is to make a precise claim: knowing the value of one does not change the probabilities for the other. This claim concerns probabilities of combinations of values, not just the separate distributions of the variables.
The distinction matters because knowing the probability distribution of each variable separately is not enough to determine how they relate. Two variables can each have the same marginal probabilities while being either independent or strongly dependent. Their joint probabilities reveal the connection.
A pair with zero probability is included in the definition. For instance, if \(X\) cannot equal 1 at the same time as \(Y\) equals 1, then \(P(X=1,Y=1)=0\). If each of those values has positive marginal probability, their product is positive, so the variables cannot be independent.
For discrete variables, the possible values are finite or countable. Values that have probability zero do not affect the condition: both sides of the factorization are zero whenever one of the marginal probabilities is zero. The key requirement is that the equality holds for every pair of values, not merely for a pair that seems representative.
What the Factorization Says
The joint probability \(P(X=x,Y=y)\) describes a combined event: \(X\) takes the value \(x\) and \(Y\) takes the value \(y\). The product \(P(X=x)P(Y=y)\) is what that joint probability would be if the two variables did not affect one another's value probabilities. Independence says that this product gives the correct joint probability for every pair.
If \(P(Y=y)>0\), the definition also gives a familiar conditional-probability interpretation. By the definition of conditional probability, $$ P(X=x\mid Y=y)=\frac{P(X=x,Y=y)}{P(Y=y)}. $$ When \(X\) and \(Y\) are independent, substitute the factorization into this expression: $$ P(X=x\mid Y=y) =\frac{P(X=x)P(Y=y)}{P(Y=y)} =P(X=x). $$ Thus, learning that \(Y=y\) does not change the probability that \(X=x\). The positive-probability condition matters: conditional probability given an event of probability zero is not defined by this formula.
Worked Example: Indicators From Two Fair Coin Tosses
Suppose two fair coins are tossed, with the four ordered outcomes HH, HT, TH, and TT equally likely. Define \(X=1\) if the first toss is heads and \(X=0\) if it is tails. Define \(Y=1\) if the second toss is heads and \(Y=0\) if it is tails. Determine whether \(X\) and \(Y\) are independent.
The ordered pairs of variable values corresponding to the four outcomes are \((1,1)\), \((1,0)\), \((0,1)\), and \((0,0)\). Each pair has probability \(1/4\). For either variable, the probability of 1 is \(1/2\), and the probability of 0 is \(1/2\).
Check the factorization for each possible pair. For example, $$ P(X=1,Y=1)=\frac14 \quad\text{and}\quad P(X=1)P(Y=1)=\frac12\cdot\frac12=\frac14. $$ For the other pairs, the joint probability is also \(1/4\), and the product of the corresponding marginals is also \((1/2)(1/2)=1/4\). The equality holds for all four pairs, so \(X\) and \(Y\) are independent.
In context, knowing the first toss was heads does not change the probability that the second toss was heads: it remains \(1/2\). The fact that the experiment has four individual outcomes is not itself the definition of independence; the conclusion follows from the factorization for the values of \(X\) and \(Y\).
From Values to Events
Independence is not limited to single values. An event described using \(X\) might include several of its values, such as \(X\in A\), where \(A\) is a set. Similarly, let \(Y\in B\) describe a set of values of \(Y\). If the variables are independent, then these two events are independent: $$ P(X\in A,Y\in B)=P(X\in A)P(Y\in B). $$ This follows directly by adding the joint probabilities for all value pairs in the two sets.
The event \(X\in A,Y\in B\) consists of all pairs \((x,y)\) with \(x\in A\) and \(y\in B\).
Because the possible value pairs are disjoint, $$ P(X\in A,Y\in B)=\sum_{x\in A}\sum_{y\in B}P(X=x,Y=y). $$
If \(X\) and \(Y\) are independent, replace each joint probability with its product of marginals: $$ \sum_{x\in A}\sum_{y\in B}P(X=x)P(Y=y) =\left(\sum_{x\in A}P(X=x)\right)\left(\sum_{y\in B}P(Y=y)\right). $$
The first sum is \(P(X\in A)\), and the second is \(P(Y\in B)\). Their product gives the claimed event-level factorization.
This also explains why independence is a statement about the entire distributions, rather than a single numerical comparison. The value-by-value factorization entails the corresponding factorization for any events formed from the two variables. Conversely, checking only one choice of \(A\) and \(B\) is not enough to establish independence.
Worked Example: Dependence From One Coin Toss
Toss one fair coin. Let \(X=1\) if the result is heads and \(X=0\) if it is tails. Let \(Y=1\) if the result is tails and \(Y=0\) if it is heads. Are \(X\) and \(Y\) independent?
Each variable has probability \(1/2\) of being 1. However, the event \(X=1,Y=1\) cannot occur: one toss cannot be both heads and tails. Therefore, $$ P(X=1,Y=1)=0, \qquad P(X=1)P(Y=1)=\frac12\cdot\frac12=\frac14. $$ The joint probability is not the product of the marginals, so \(X\) and \(Y\) are dependent.
The same conclusion can be seen by conditioning. If \(Y=1\), then the toss was tails and \(X=1\) is impossible, so \(P(X=1\mid Y=1)=0\), not \(P(X=1)=1/2\). Here one variable's value determines the other's.
Variables and Individual Outcomes Are Different Things
A random variable is a rule that assigns a numerical value to each outcome of an experiment. An individual outcome is one possible result of the experiment itself, such as a particular die face or an ordered pair of coin results. Independence is not a property of two individual outcomes. It is a relationship between random variables, or between events, expressed through probabilities.
It is common to say that two trials or two events are independent. In that usage, the claim is that the events associated with those trials satisfy a probability factorization. For random variables, the corresponding claim is that their value events satisfy the factorization for every pair of values. Merely having two variables—or two descriptions of an experiment—does not make them independent.
Worked Example: Independent Variables From One Die Roll
A fair four-sided die has outcomes 1, 2, 3, and 4, each with probability \(1/4\). Define \(X=1\) for outcomes 1 or 2 and \(X=0\) for outcomes 3 or 4. Define \(Y=1\) for outcomes 1 or 3 and \(Y=0\) for outcomes 2 or 4. Determine whether \(X\) and \(Y\) are independent.
The mapping from die outcome to variable values is:
| Die outcome | \(X\) | \(Y\) | Probability |
|---|---|---|---|
| 1 | 1 | 1 | \(1/4\) |
| 2 | 1 | 0 | \(1/4\) |
| 3 | 0 | 1 | \(1/4\) |
| 4 | 0 | 0 | \(1/4\) |
Each variable equals 1 for two of the four equally likely outcomes, so \(P(X=1)=P(Y=1)=1/2\). Each equals 0 with probability \(1/2\) as well. For every pair \((x,y)\) in \(\{0,1\}\times\{0,1\}\), exactly one die outcome produces that pair, so \(P(X=x,Y=y)=1/4\). The product of the corresponding marginal probabilities is also \(1/2\cdot1/2=1/4\) for every pair. Thus \(X\) and \(Y\) are independent.
Both variables are functions of the same die outcome, yet they are independent. Sharing an underlying experiment does not by itself imply dependence. The defining question is whether the joint value probabilities factor into the marginals.
Common Mistakes and Why Independence Matters
- Assuming that separate distributions establish independence. Knowing \(P(X=x)\) and \(P(Y=y)\) alone does not give \(P(X=x,Y=y)\). The joint probabilities must satisfy the factorization.
- Checking only one pair of values. A match for one pair does not show that all pairs factor. Independence requires the equality for every possible value of both variables.
- Calling outcomes independent. An outcome is a possible result, not a random variable. Ask instead which variables or events are being compared and whether their probabilities factor.
- Assuming variables from the same experiment must be dependent. The die example shows that this is not generally true; the joint and marginal probabilities decide.
- Confusing “cannot happen together” with independence. If two positive-probability value events are mutually exclusive, their joint probability is zero while the product of their probabilities is positive. They are dependent, not independent.
The factorization definition is useful because it turns the informal idea of “no change in probabilities when information is learned” into a precise rule. It also separates two questions that are easy to conflate: what each variable does on its own, and how the variables behave together. Once that relationship is specified, probabilities for events involving both variables can be built from their separate distributions when independence holds.
Check Your Understanding
Answer each question using the joint-and-marginal definition of independence.
- Suppose \(P(X=2)=0.4\), \(P(Y=1)=0.5\), and \(P(X=2,Y=1)=0.2\). Does this pair of values satisfy the independence factorization? Does that single match prove the variables independent?
- If \(P(X=0)=0.3\), \(P(Y=1)=0.6\), and \(P(X=0,Y=1)=0.25\), are the variables independent?
- Why does independence imply \(P(X\in A,Y\in B)=P(X\in A)P(Y\in B)\) for sets of values \(A\) and \(B\)?
- Can two variables that are functions of the same experiment be independent? Explain using the four-sided die example.
- In one coin toss, \(X\) indicates heads and \(Y\) indicates tails. Identify a pair of values that disproves independence and show the unequal probabilities.