Common-Cause and Collider Pitfalls

Why Conditioning Can Change Dependence

Conditional independence is not a property that necessarily survives conditioning on an arbitrary variable or event. A shared cause can make two variables associated before conditioning, even when they are independent once that cause is known. Conversely, conditioning on a common effect of two variables can make them associated when they were independent beforehand.

These patterns are useful warnings, not mechanical rules. The actual dependence depends on the joint law and on what is being conditioned on. The calculations below make the distinction precise.

Conditional Independence

In a finite model, this says that within each value of \(Z\) having positive probability, the conditional joint probabilities factor into the corresponding conditional marginal probabilities. Conditional independence does not say that \(X\) and \(Y\) are independent without conditioning. Nor does unconditional independence guarantee independence after conditioning.

A Common Cause Can Create Marginal Dependence

A common-cause pattern has a variable \(Z\) influencing both \(X\) and \(Y\). If \(Z\) is unobserved or ignored, the mixture over its possible values can associate \(X\) and \(Y\). Once \(Z\) is given, that association may disappear.

The example is an instance of a general covariance identity. It separates dependence within groups defined by \(Z\) from dependence due to differences between those groups.

Proof. Write \(m_X=\mathbb{E}[X\mid\sigma(Z)]\) and \(m_Y=\mathbb{E}[Y\mid\sigma(Z)]\). By the definition of conditional covariance and the tower property, $$ \mathbb{E}[\operatorname{Cov}(X,Y\mid Z)] =\mathbb{E}[XY]-\mathbb{E}[m_Xm_Y]. $$ Also, \(\mathbb{E}[m_X]=\mathbb{E}[X]\) and \(\mathbb{E}[m_Y]=\mathbb{E}[Y]\), so $$ \operatorname{Cov}(m_X,m_Y) =\mathbb{E}[m_Xm_Y]-\mathbb{E}[X]\mathbb{E}[Y]. $$ Adding these two equalities yields \(\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]=\operatorname{Cov}(X,Y)\), as required. \(\square\)

If \(X\) and \(Y\) are conditionally independent given \(Z\), then their conditional covariance is zero almost surely, provided they are square-integrable. The theorem then says their marginal covariance is the covariance of their conditional means. In the binary example, those means both increase with \(Z\), so their covariance is positive. Conditional independence therefore does not, by itself, imply marginal independence.

A Collider Can Create Dependence

A collider pattern has two variables feeding into a common effect. Conditioning on that effect can change the relative plausibility of the possible values of its causes. This can associate the causes, despite their being independent before selection.

For this particular “either cause” rule, the induced association is negative. That sign is not a universal feature of collider conditioning: it depends on how the common effect is defined.

Proof. Put \(d=\mathbb{P}(C=1)=p+q-pq\), which is positive. Since \(C=1\) whenever \(X=1\), and likewise for \(Y\), independence gives $$ \mathbb{P}(X=1\mid C=1)=\frac{p}{d},\qquad \mathbb{P}(Y=1\mid C=1)=\frac{q}{d},\qquad \mathbb{P}(X=1,Y=1\mid C=1)=\frac{pq}{d}. $$ Subtracting the product of the first two conditional probabilities from the third gives $$ \operatorname{Cov}(X,Y\mid C=1) =\frac{pq}{d}-\frac{pq}{d^2} =\frac{pq(d-1)}{d^2}. $$ Since \(d-1=-(1-p)(1-q)\), this is the claimed expression. All factors \(p,q,1-p,1-q\) are positive, so the covariance is strictly negative. \(\square\)

The Collider Mechanism Does Not Fix the Sign

These examples also show why the word “collider” is a useful description of a structure, not a complete calculation. The effect may encode “at least one,” “both,” “exactly one,” equality, or a noisy combination. Those rules select different subsets or weight different outcomes differently. One must calculate the relevant conditional law before asserting the direction or strength of the association.

Diagnosing Common Claims

Several tempting claims fail because they omit what is being conditioned on and how the variables are generated:

• “Conditional independence implies independence.” The common-cause example disproves this: the conditional law factors at each value of \(Z\), but its mixture does not factor.
• “Independent variables remain independent after conditioning.” The collider examples disprove this: \(X\) and \(Y\) begin independent, but conditioning on their common effect makes them dependent.
• “A collider always induces negative dependence.” The equality example gives positive conditional covariance. The sign depends on the collider mechanism.
• “Conditioning always removes dependence caused by a common cause.” In the common-cause example it does remove the association, but that conclusion follows from the specified conditional factorization; it is not guaranteed for every model in which a variable influences two others.

In practice, specify the joint law or the conditional kernels, identify the event or variable being conditioned on, and compare the conditional joint law with the product of its conditional marginals. A structural story can suggest which calculations to make; it cannot replace them. In particular, do not confuse conditioning on a common cause with conditioning on a common effect. These operations can have opposite consequences.