Why Conditional Independence Constrains Conditional Models
A collection of conditional models can look individually reasonable while imposing incompatible requirements on the same underlying distribution. Conditional independence gives a precise way to expose some of those requirements. If a joint distribution says that two variables are independent after conditioning on a third, then the conditional distribution of either variable must remain unchanged as the other variable varies, within each conditioning stratum.
In MICE, this matters because a conditional model is specified for each variable to be imputed. Compatibility asks whether those conditionals could all be derived from one joint distribution. A conditional-independence claim can give a useful check: if the specified conditionals violate a restriction that the claim requires, they cannot all come from a joint distribution with that independence structure. Passing this particular check is not, by itself, proof of compatibility.
The definition has an equivalent conditional-distribution form: for values with positive conditioning probability, \(P(X=x\mid Y=y,Z=z)=P(X=x\mid Z=z)\), and likewise \(P(Y=y\mid X=x,Z=z)=P(Y=y\mid Z=z)\). In words, once \(Z\) is known, learning \(Y\) gives no further information about \(X\), and learning \(X\) gives no further information about \(Y\). These are distributional statements: they concern the whole conditional distribution, not just its mean.
Deriving the Restrictions
The equivalence follows directly from the definition. Suppose \(X\perp Y\mid Z\), and consider a stratum \(Z=z\) in which \(P(Y=y\mid Z=z)>0\). Then
The same argument with \(X\) and \(Y\) exchanged gives the other equality. Conversely, if \(P(X=x\mid Y=y,Z=z)=P(X=x\mid Z=z)\) for all relevant \(x,y,z\), multiplying both sides by \(P(Y=y\mid Z=z)\) gives the factorization in the definition. The positive-probability qualification matters: a conditional probability given an event of probability zero is not determined by the ordinary ratio definition.
These equalities are practical compatibility constraints. For example, if a proposed conditional model for \(X\) includes \(Y\), but a claimed joint structure says \(X\perp Y\mid Z\), then its distribution for \(X\) must not change with \(Y\) once \(Z\) is fixed. A model may be parameterized in many ways, but its implied probabilities still have to obey that invariance.
Worked Example: Checking Independence Within Each Stratum
Consider a hypothetical probability table of 100 individuals. \(X\) and \(Y\) are binary, and the table gives counts within each level of \(Z\). Check whether the table supports \(X\perp Y\mid Z\).
| Stratum | \(X=0,Y=0\) | \(X=0,Y=1\) | \(X=1,Y=0\) | \(X=1,Y=1\) | Total |
|---|---|---|---|---|---|
| \(Z=0\) | 18 | 12 | 12 | 8 | 50 |
| \(Z=1\) | 8 | 12 | 12 | 18 | 50 |
For \(Z=0\), the conditional probability of \(X=1\) is \((12+8)/50=0.4\). Among those with \(Y=0\), it is \(12/(18+12)=0.4\); among those with \(Y=1\), it is \(8/(12+8)=0.4\). Thus the conditional distribution of \(X\) does not change across the observed values of \(Y\) in this stratum.
For \(Z=1\), \(P(X=1\mid Z=1)=(12+18)/50=0.6\). The corresponding probabilities within the two \(Y\) groups are \(12/(8+12)=0.6\) and \(18/(12+18)=0.6\). The same invariance holds in this stratum.
The joint probabilities also factor in each stratum. For \(Z=0\), \(P(X=1\mid Z=0)=0.4\) and \(P(Y=1\mid Z=0)=(12+8)/50=0.4\); their product is \(0.16\), equal to \(P(X=1,Y=1\mid Z=0)=8/50=0.16\). For \(Z=1\), the two marginal probabilities are \(0.6\) and \(0.6\), whose product \(0.36\) equals \(18/50=0.36\). The other cells factor as well. The table therefore satisfies conditional independence within both strata. This is a check on the table’s probabilities, not a claim that a finite sample proves exact independence in a population.
Using the Restriction to Check Specified Conditionals
When conditionals are specified as probability models rather than tabulated counts, check whether the distribution for one variable is invariant to the other variable at fixed values of the conditioning variables. This check is most direct when all relevant conditioning combinations have positive probability. If some combinations are impossible or absent, the data cannot check restrictions for those combinations without additional assumptions.
Worked Example: Two Conditionals That Share an Independence Structure
Suppose a hypothetical specification says \(X\perp Y\mid Z\), with \(X\) and \(Y\) binary. It specifies \(P(X=1\mid Y,Z)\) as \(0.2\) for either value of \(Y\) when \(Z=0\), and \(0.7\) for either value when \(Z=1\). It specifies \(P(Y=1\mid X,Z)\) as \(0.3\) for either value of \(X\) when \(Z=0\), and \(0.6\) for either value when \(Z=1\). Both conditionals satisfy the invariance required by the claimed independence. We can show that these particular specifications do coexist by constructing a joint distribution.
Choose \(P(Z=0)=P(Z=1)=0.5\). Within each stratum, take \(X\) and \(Y\) to be independent with the specified probabilities. For \(Z=0\), the four conditional cell probabilities are \(P(X=1,Y=1\mid Z=0)=0.2(0.3)=0.06\), \(P(X=1,Y=0\mid Z=0)=0.2(0.7)=0.14\), \(P(X=0,Y=1\mid Z=0)=0.8(0.3)=0.24\), and \(P(X=0,Y=0\mid Z=0)=0.8(0.7)=0.56\). They sum to \(1.00\).
For \(Z=1\), the corresponding probabilities are \(0.7(0.6)=0.42\), \(0.7(0.4)=0.28\), \(0.3(0.6)=0.18\), and \(0.3(0.4)=0.12\), which also sum to \(1.00\). Multiplying each stratum-specific cell probability by \(P(Z=z)=0.5\) gives joint probabilities for \((X,Y,Z)\) that sum to \(0.5+0.5=1\). Summing those joint probabilities over \(Y\) recovers the specified conditional distribution of \(X\) in each stratum; summing over \(X\) recovers the specified conditional distribution of \(Y\). Thus these conditionals do have a joint distribution with the asserted conditional independence.
The construction works because both conditionals reduce to distributions depending only on \(Z\), and the within-stratum joint distribution can be formed as their product. The example verifies this particular pair of models; it does not establish a general rule that any conditionals satisfying one simple invariance check will be compatible.
When a Claimed Independence Is Contradicted
A failed invariance check is decisive for the specific independence claim, provided the values being compared are in the relevant support. If a conditional model assigns different distributions to \(X\) at different values of \(Y\), while holding \(Z\) fixed, then it cannot be the conditional distribution from a joint distribution satisfying \(X\perp Y\mid Z\). This conclusion does not require a test statistic: it follows from the definition.
Worked Example: A Direct Violation of Conditional Independence
Suppose a hypothetical model claims \(X\perp Y\mid Z\), and all four combinations of \(Y\) and \(Z\) considered below have positive probability. At \(Z=0\), it specifies \(P(X=1\mid Y=0,Z=0)=0.2\) but \(P(X=1\mid Y=1,Z=0)=0.5\). The two conditional probabilities differ by \(0.5-0.2=0.3\), or 30 percentage points.
If the claimed independence held, both probabilities would have to equal \(P(X=1\mid Z=0)\). That would require \(0.2=0.5\), a contradiction. Therefore, no joint distribution can both generate these specified conditional probabilities and satisfy \(X\perp Y\mid Z\) on this support. The conclusion is about the joint distribution and the stated independence claim; it does not establish that the conditional model is unusable under some different joint structure.
Assumptions, Diagnostics, and Limits
A conditional-independence check is meaningful only when the variables and conditioning sets refer to the same joint probability model. State the proposed relationship precisely: for example, \(X\perp Y\mid Z\), rather than the less specific claim that \(X\) and \(Y\) are “unrelated.” Then check the complete conditional distributions at fixed \(Z\), not merely a few summaries. For discrete variables, this means comparing probabilities across the \(Y\) categories. For continuous variables, it means comparing conditional distributions or densities; equality of conditional means alone need not establish equality of distributions.
Several diagnostic cautions follow:
- Check support. The equality restrictions apply where the conditioning events have positive probability. If a combination cannot occur, its conditional probability is not identified by ordinary conditioning.
- Separate sample variation from model restrictions. In a finite data set, estimated proportions can differ by chance even when a population independence assumption is reasonable. Conversely, similar sample proportions do not prove exact population independence.
- Do not infer independence from a zero coefficient alone. A zero regression coefficient may express a restriction on a conditional mean, not on the full conditional distribution. Additional distributional assumptions are needed to turn some coefficient restrictions into independence claims.
- Check the conditioning set carefully. Independence given \(Z\) is not the same as marginal independence. Averaging over \(Z\) can create an association between \(X\) and \(Y\), even if they are independent within every stratum.
There are useful alternatives and complements to this check. Direct algebra on specified conditional probabilities is appropriate when the models are fully known, as in the examples above. A joint probability table makes the factorization visible for discrete variables. In continuous models, distributional comparisons or a stated joint model may be needed. Checking only a correlation, a regression coefficient, or one moment is generally weaker than checking the full conditional-distribution restriction. A joint model can provide a way to derive and compare its conditionals, but the conditional-independence check remains a targeted diagnostic rather than a complete audit of every compatibility requirement.
The central limitation is logical: a contradiction with an implication of a proposed joint structure rules that structure out; absence of a contradiction does not prove that all specified conditionals can be generated by one joint distribution. Conditional independence supplies necessary equalities for the claimed structure. Other restrictions may still matter. This distinction is particularly important in MICE: a stable sequence of updates or a plausible-looking conditional model does not turn a failed independence restriction into a compatible joint specification.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- If \(X\perp Y\mid Z\), what equality must hold between \(P(X=x\mid Y=y,Z=z)\) and \(P(X=x\mid Z=z)\) when the conditioning events have positive probability?
- At \(Z=0\), a model gives \(P(X=1\mid Y=0,Z=0)=0.35\) and \(P(X=1\mid Y=1,Z=0)=0.35\). Does this single comparison establish \(X\perp Y\mid Z\)? Explain.
- A data table has \(P(X=1\mid Z=0)=0.4\) and \(P(Y=1\mid Z=0)=0.5\). Under conditional independence, what is \(P(X=1,Y=1\mid Z=0)\)?
- Why can a zero difference between two fitted conditional means be insufficient to establish conditional independence?
Solutions. 1. The required equality is \(P(X=x\mid Y=y,Z=z)=P(X=x\mid Z=z)\) for every relevant \(x,y,z\). 2. No. It checks one value of \(X\) in one stratum; the full conditional distributions must be invariant across all supported \(Y\) values and strata. 3. Conditional independence gives \(0.4(0.5)=0.20\). 4. Equal means do not generally imply equal distributions: conditional distributions can have the same mean but different variances or shapes. Independence restricts the full conditional distribution, not only its mean.