Pairwise Independence Is Not Mutual Independence

Two Different Independence Requirements

Independence is often checked two variables at a time. For a family of three or more random variables, however, that does not generally settle whether the whole family is independent. The distinction is not a technicality: pairwise independence determines each two-variable marginal law, while mutual independence requires a factorization involving all variables at once.

Let $X_1,\ldots,X_n$ be random variables on a probability space $(\Omega,\mathcal F,\mathbb P)$. Independence of two variables means that their generated sigma-algebras are independent. Equivalently, for all measurable sets $A$ and $B$ in their respective state spaces, the probability that both variables fall in the specified sets is the product of the two probabilities.

Definition: A finite family $X_1,\ldots,X_n$ is pairwise independent if $X_i$ and $X_j$ are independent for every distinct pair $i,j$. It is mutually independent if, for every choice of measurable sets $A_1,\ldots,A_n$, $$ \mathbb P(X_1\in A_1,\ldots,X_n\in A_n) = \prod_{i=1}^n \mathbb P(X_i\in A_i). $$

Mutual independence implies pairwise independence: in the mutual factorization, choose the whole state space for every variable except the two under consideration. The reverse implication is false. The examples below show the failure directly, including the precise joint probability that pairwise checks do not constrain.

A Basic Parity Construction

Worked Example: Three Pairwise Independent Bits

Take the four-point probability space $\Omega=\{(0,0),(0,1),(1,0),(1,1)\}$, with each outcome having probability $1/4$. Define $X(a,b)=a$, $Y(a,b)=b$, and let $Z(a,b)$ be the remainder when $a+b$ is divided by $2$. The resulting values are:

$(X,Y,Z)$	Probability
$(0,0,0)$	$1/4$
$(0,1,1)$	$1/4$
$(1,0,1)$	$1/4$
$(1,1,0)$	$1/4$

Each variable takes each value $0$ and $1$ with probability $1/2$. For the pair $(X,Y)$, all four possible pairs occur once, so each has probability $1/4$, equal to $\mathbb P(X=x)\mathbb P(Y=y)$. For $(X,Z)$, the four listed outcomes give the pairs $(0,0),(0,1),(1,1),(1,0)$, again once each. For $(Y,Z)$, they give $(0,0),(1,1),(0,1),(1,0)$, also once each. Thus all three pairs are independent.

But the event $\{X=1,Y=1,Z=1\}$ is empty: the only outcome with $X=Y=1$ has $Z=0$. Its probability is $0$, whereas the product of the three marginal probabilities is $1/8$. The required three-variable factorization fails, so the variables are not mutually independent.

This example exposes the missing condition. Pairwise independence verifies all the two-coordinate probabilities, but it does not force the probability of a specified triple of values to equal the product of its three marginal probabilities.

What Factorization Means for Finite-Valued Variables

When variables take values in finite sets, mutual independence can be tested using their joint mass function. This makes the distinction especially concrete: pairwise independence factors each two-coordinate mass function, while mutual independence factors the entire joint mass function.

Theorem: Suppose $X_1,\ldots,X_n$ take values in finite sets $S_1,\ldots,S_n$. They are mutually independent if and only if, for every $x_i\in S_i$, $$ \mathbb P(X_1=x_1,\ldots,X_n=x_n) = \prod_{i=1}^n \mathbb P(X_i=x_i). $$

Proof. If the variables are mutually independent, apply the definition with $A_i=\{x_i\}$ for each $i$. This gives the displayed point-probability factorization. Conversely, suppose that factorization holds for every tuple of values. For arbitrary subsets $A_i\subseteq S_i$, finite additivity gives $$ \begin{aligned} \mathbb P(X_1\in A_1,\ldots,X_n\in A_n) &=\sum_{x_1\in A_1}\cdots\sum_{x_n\in A_n} \mathbb P(X_1=x_1,\ldots,X_n=x_n)\\ &=\sum_{x_1\in A_1}\cdots\sum_{x_n\in A_n} \prod_{i=1}^n\mathbb P(X_i=x_i)\\ &=\prod_{i=1}^n\left(\sum_{x_i\in A_i}\mathbb P(X_i=x_i)\right)\\ &=\prod_{i=1}^n\mathbb P(X_i\in A_i). \end{aligned} $$ The second equality uses the assumed pointwise factorization; the third distributes the finite sums over the product. This is mutual independence. $\square$

For the parity example, the joint mass at $(1,1,1)$ is $0$, while the product of the marginal masses is $(1/2)^3=1/8$. The theorem confirms that this one failed pointwise factorization is enough to rule out mutual independence.

Pairwise Independence Without Zero-Probability Outcomes

The failure is not caused only by a joint outcome being impossible. A small change to the probabilities gives a pairwise independent family for which every triple has positive probability, but the full joint mass still does not factor.

Worked Example: A Positive Joint Mass That Does Not Factor

On $\Omega=\{0,1\}^3$, assign probability $3/16$ to each triple whose coordinate sum is even, and probability $1/16$ to each triple whose coordinate sum is odd. There are four triples of each kind, so the total probability is $4(3/16)+4(1/16)=1$.

Fix any values of two coordinates. Of the two possible values of the remaining coordinate, exactly one makes the sum even and the other makes it odd. Their probabilities add to $3/16+1/16=1/4$. Thus every pair of coordinates is uniform on its four possible value-pairs. In particular, each coordinate has marginal probabilities $1/2$ and $1/2$, and each pair has mass $1/4=(1/2)(1/2)$ at every pair of values. The three coordinates are pairwise independent.

However, the probability of $(0,0,0)$ is $3/16$, since its sum is even. The product of the three marginal probabilities at this point is $1/8=2/16$, not $3/16$. The full joint mass does not factor, so the variables are not mutually independent. All eight triple outcomes nevertheless have positive probability.

This example is useful when checking intuition: positivity of every joint cell does not make pairwise factorization sufficient. The issue is whether the joint probabilities have the required product values, not merely whether outcomes are possible.

A Block Can Reveal the Dependence

Pairwise independence also does not say that one variable is independent of a block formed from the others. In the parity construction, $Z$ is determined by the pair $(X,Y)$. Although $Z$ is independent of $X$ and independent of $Y$ separately, it is not independent of the random vector $(X,Y)$.

Worked Example: A Pair Is Informative About the Third Variable

Use the four equally likely outcomes and variables from the parity example. The event $\{Z=0\}$ is exactly the event that $X=Y$. In particular, when $(X,Y)=(0,0)$, the value of $Z$ is certainly $0$. Hence $$ \mathbb P\big((X,Y)=(0,0),\,Z=0\big)=\frac14. $$ If the random vector $(X,Y)$ and $Z$ were independent, this probability would instead equal $$ \mathbb P((X,Y)=(0,0))\mathbb P(Z=0) =\frac14\cdot\frac12 =\frac18. $$ The two quantities differ, so the block and the third variable are dependent. The pairwise checks do not test this block-level factorization.

The same construction can be written with signs. Put $U=(-1)^X$, $V=(-1)^Y$, and $W=(-1)^Z$. Then $W=UV$; each sign is equally likely to be $1$ or $-1$, and each pair is independent. Yet $UVW=1$ at every outcome, so $\mathbb E[UVW]=1$, while $\mathbb E[U]\mathbb E[V]\mathbb E[W]=0$. This is another way to see that a joint factorization can fail even when all pairwise factorizations hold.

How to Check Independence Carefully

For a finite family, a reliable check separates the pairwise and mutual questions rather than treating them as interchangeable. First establish each marginal law. Then check every two-variable joint law against the product of its marginals. To establish mutual independence, also check the full joint law; for finite-valued variables, the theorem above says it is enough to check every point of the joint state space.

Find the marginal probabilities.
Compute the probability of each value of each variable.

Test every pair.
For each two-variable value-pair, compare its joint probability with the product of the corresponding marginals.

Test the full joint law.
Compare joint point probabilities with products of all the marginal probabilities. A single mismatch disproves mutual independence.

The implication from mutual to pairwise independence is useful, but it must not be reversed without further evidence. Pairwise checks answer a collection of two-variable questions; mutual independence asks for simultaneous factorization across the entire family. In the parity examples, the missing information is exactly the joint behavior of all three coordinates.

Check Your Understanding

Use the finite examples and definitions above to answer these questions.

In the four-outcome parity construction, what is $\mathbb P(X=1,Y=1,Z=1)$, and what would it equal under mutual independence?
Why does checking all four possible value-pairs establish independence for each pair of binary variables in the parity example?
In the weighted example, verify the probability of $(0,0,0)$ against the product of its marginal probabilities.
Why does positive probability for every triple not guarantee mutual independence?
In the block example, what event shows that $(X,Y)$ and $Z$ are dependent?

\((X,Y,Z)\)	Probability
\((0,0,0)\)	\(1/4\)
\((0,1,1)\)	\(1/4\)
\((1,0,1)\)	\(1/4\)
\((1,1,0)\)	\(1/4\)