Conditional Independence In Shared-Parameter Factorizations

Why Condition on a Shared Effect?

In a longitudinal study, outcomes and response indicators can be related because they share an underlying subject characteristic. For example, a latent tendency toward poor health could affect both a participant’s outcomes and the probability of responding at later visits. A shared-parameter model represents this idea by conditioning on a latent random effect. Given that effect, the outcome vector and the response-pattern vector may be independent; after the effect is averaged out, they can nevertheless be associated.

Let $Y=(Y_1,\ldots,Y_J)$ denote the outcome vector, $R=(R_1,\ldots,R_J)$ the response-indicator vector, and $U$ a latent random effect. Here $R_j=1$ means the outcome at visit $j$ is observed, and $R_j=0$ means it is missing. The effect $U$ is shared in the sense that it may influence the conditional distribution of both vectors.

Definition: Conditional independence of $Y$ and $R$ given $U$, written $Y\perp R\mid U$, means that for almost every value $u$ in the distribution of $U$, their conditional joint distribution factors as $$ f(y,r\mid u)=f_Y(y\mid u)f_R(r\mid u). $$ This factorization concerns the two vectors as wholes. It does not assert independence among the components of $Y$, nor independence among the components of $R$.

The densities in this expression may be probability mass functions when variables are discrete. More generally, the equality is understood for the appropriate conditional distributions. The key assumption is the factorization conditional on the same $U$, not a factorization of the marginal distributions.

From Conditional Independence to the Joint Law

Under the conditional-independence assumption, the joint law of the outcomes, response indicators, and random effect can be written as

f(y,r,u)=f_Y(y\mid u)f_R(r\mid u)f_U(u).

This follows from the chain rule $f(y,r,u)=f(y,r\mid u)f_U(u)$, followed by the conditional factorization in the definition. To obtain the marginal joint law of $Y$ and $R$, integrate or sum over the latent effect:

f(y,r)=\int f_Y(y\mid u)f_R(r\mid u)f_U(u)\,du.

The integral averages products of conditional distributions. It is generally not equal to the product of their separate averages:

\int f_Y(y\mid u)f_R(r\mid u)f_U(u)\,du \ne \left\{\int f_Y(y\mid u)f_U(u)\,du\right\} \left\{\int f_R(r\mid u)f_U(u)\,du\right\}.

The two sides are equal under marginal independence, but conditional independence alone does not require that equality. When the conditional distributions of both vectors vary with the same $U$, mixing over $U$ can create marginal association.

Worked Example: Conditional Independence but Marginal Association

Suppose $U$ takes values 0 and 1 with probability $0.5$ each. Given $U$, binary outcome $Y$ and response indicator $R$ are independent. Let $P(Y=1\mid U=0)=0.2$, $P(Y=1\mid U=1)=0.8$, $P(R=1\mid U=0)=0.1$, and $P(R=1\mid U=1)=0.7$. Find the marginal probabilities and assess independence.

The conditional-independence assumption applies separately at each value of $U$. Thus the probability of $Y=1,R=1$ is

$$ P(Y=1,R=1) =0.5(0.2)(0.1)+0.5(0.8)(0.7) =0.01+0.28=0.29. $$

The marginal probabilities are $P(Y=1)=0.5(0.2)+0.5(0.8)=0.50$ and $P(R=1)=0.5(0.1)+0.5(0.7)=0.40$. If $Y$ and $R$ were marginally independent, their joint probability of both being 1 would be $0.50(0.40)=0.20$, not 0.29. They are therefore marginally dependent.

For clarity, their marginal $2\times2$ probabilities are $P(1,1)=0.29$, $P(1,0)=0.21$, $P(0,1)=0.11$, and $P(0,0)=0.39$. These sum to 1. The shared effect makes high values of $Y$ and $R$ more likely together, even though the variables are independent within each level of $U$.

Covariance and the Binary Case

For scalar variables with finite second moments, the law of total covariance separates conditional association from association between conditional means:

\operatorname{Cov}(Y,R) = E\{\operatorname{Cov}(Y,R\mid U)\} + \operatorname{Cov}\{E(Y\mid U),E(R\mid U)\}.

Conditional independence implies $\operatorname{Cov}(Y,R\mid U)=0$ whenever these conditional covariances exist. Consequently,

\operatorname{Cov}(Y,R) = \operatorname{Cov}\{E(Y\mid U),E(R\mid U)\}.

This is a useful justification for the dependence seen in the example: the conditional means of $Y$ and $R$ both increase with $U$, so their covariance is positive. In fact, using the example’s numbers, their conditional means are $(0.2,0.8)$ for $Y$ and $(0.1,0.7)$ for $R$. Their mean product is $0.5(0.02)+0.5(0.56)=0.29$, while the product of their averages is $0.50(0.40)=0.20$; the covariance is $0.09$.

For binary $Y$ and $R$, marginal independence has an especially simple relationship to covariance. Since $E(YR)=P(Y=1,R=1)$, zero covariance is equivalent to $P(Y=1,R=1)=P(Y=1)P(R=1)$. For binary variables, this equality implies the factorization of all four joint probabilities, so it is equivalent to marginal independence. Therefore, under conditional independence given $U$, zero covariance of the conditional means is sufficient for marginal independence when both variables are binary.

Worked Example: Binary Variables Can Be Marginally Independent

Let $U$ again be equally likely to be 0 or 1. Given $U$, binary variables $Y$ and $R$ are independent. Suppose $P(Y=1\mid U=0)=0.2$, $P(Y=1\mid U=1)=0.8$, and $P(R=1\mid U=u)=0.4$ for either value $u$.

The marginal probabilities are $P(Y=1)=0.5(0.2)+0.5(0.8)=0.50$ and $P(R=1)=0.40$. The joint probability is

$$ P(Y=1,R=1) =0.5(0.2)(0.4)+0.5(0.8)(0.4) =0.04+0.16=0.20. $$

This equals $P(Y=1)P(R=1)=0.50(0.40)=0.20$. The other three joint probabilities are $P(1,0)=0.30$, $P(0,1)=0.20$, and $P(0,0)=0.30$, which also factor as products of the marginals. Thus $Y$ and $R$ are marginally independent in this example, despite the variation in the conditional distribution of $Y$ across $U$. The conditional mean of $R$ is constant, making the covariance of the conditional means zero.

Why Zero Covariance Is Not a General Independence Test

The binary result does not extend to all outcome types. For nonbinary variables, zero covariance does not in general imply independence. Conditional independence given $U$ does not change that fact: zero covariance of the conditional means can make the marginal covariance zero while other features of the conditional distributions still depend on $U$.

Worked Example: Zero Covariance but Nonbinary Dependence

Let $U$ be equally likely to be 0 or 1. Given $U=0$, let $Y$ be 0 or 2 with equal probabilities and set $R=0$. Given $U=1$, set $Y=1$ and $R=1$. Given $U$, $R$ is fixed, so $Y$ and $R$ are conditionally independent.

The conditional means of $Y$ are 1 at both values of $U$, while those of $R$ are 0 and 1. Therefore, $\operatorname{Cov}\{E(Y\mid U),E(R\mid U)\}=0$, and the law of total covariance gives $\operatorname{Cov}(Y,R)=0$. Nevertheless, $P(Y=1)=0.5$, $P(R=0)=0.5$, and $P(Y=1,R=0)=0$, whereas independence would require $0.5(0.5)=0.25$. Thus $Y$ and $R$ are dependent.

The dependence is carried by more than the conditional means: the conditional distribution of $Y$ changes with $U$, and $R$ reveals which level of $U$ occurred. This illustrates why covariance alone cannot characterize independence for nonbinary outcomes.

Vector Factorization: What It Does and Does Not Say

When $Y$ and $R$ are vectors, conditional independence is a statement about the full vectors. It permits dependence among visits within $Y$, and dependence among response indicators within $R$. For example, a subject-level effect may make all outcomes tend to be high together, while a response process may make later indicators more likely to be missing after an earlier missed visit. Neither form of within-vector dependence contradicts $Y\perp R\mid U$.

Worked Example: Factorizing Whole Vectors

Suppose $U$ is equally likely to be 0 or 1, and $Y=(Y_1,Y_2)$, $R=(R_1,R_2)$ are conditionally independent as vectors. At $U=0$, let $P(Y=(1,1)\mid U=0)=0.6$ and $P(R=(1,0)\mid U=0)=0.1$. At $U=1$, let the corresponding probabilities be 0.2 and 0.7. The remaining probability in each vector distribution can be assigned to other vector values.

The joint probability of these two specified vector events is

P\{Y=(1,1),R=(1,0)\} =0.5(0.6)(0.1)+0.5(0.2)(0.7) =0.03+0.07=0.10.

The separate marginal event probabilities are $P\{Y=(1,1)\}=0.5(0.6)+0.5(0.2)=0.40$ and $P\{R=(1,0)\}=0.5(0.1)+0.5(0.7)=0.40$. Their product is $0.16$, not $0.10$, so the vectors are not marginally independent. The model factorizes the conditional vector distribution at each $u$; it does not permit replacing the vectors by separate visit-specific factors without additional assumptions.

Assumptions, Diagnostics, and Alternatives

The shared-parameter factorization is a model assumption, not a consequence of having introduced a latent effect. It requires that the chosen $U$ be sufficient to remove conditional dependence between the full outcome and response vectors. It also requires a specified distribution for $U$ and valid conditional distributions $f_Y(y\mid u)$ and $f_R(r\mid u)$. If there remains a dependence between $Y$ and $R$ after conditioning on $U$, the product factorization is not justified.

Because $U$ is latent, the conditional-independence claim is generally not checked by simply stratifying observed data on $U$. Model assessment instead depends on the implications of the fitted model for observed data, the plausibility of the latent-effect structure, sensitivity to alternative specifications, and whether the model can reproduce relevant features of the data. A model can fit observed summaries adequately without making its latent conditional-independence assumption empirically verifiable.

The main contrast is between conditional and marginal factorizations. The shared-parameter formulation factors $Y$ and $R$ given $U$, then mixes over $U$; a marginal-independence model would factor their marginal distributions directly, which is a stronger and different restriction. Selection and pattern-mixture factorizations organize the joint law using different conditional distributions, but changing the factorization does not by itself establish conditional independence given a shared effect. The earlier material on the selection and pattern-mixture factorizations provides those alternative organizations.

A practical failure mode is to interpret a conditional-independence assumption as a statement that response is unrelated to outcomes. It is not: in general, $Y$ and $R$ are associated marginally, and the response mechanism can depend on the latent effect that also predicts outcomes. Nor does the factorization alone say that missing outcomes can be ignored. The relationship between the latent effect, observed outcomes, and response process must be considered as part of the substantive model.

Key takeaway: $Y\perp R\mid U$ gives a product of conditional vector distributions. Integrating over a shared $U$ generally does not preserve that product. For binary $Y$ and $R$, zero marginal covariance is equivalent to independence; for nonbinary variables, it is not.

Check Your Understanding

Answer each question, then compare with the solutions below.

If $Y\perp R\mid U$, must the components $Y_1$ and $Y_2$ be conditionally independent given $U$?
For binary $Y$ and $R$, suppose $Y\perp R\mid U$ and $\operatorname{Cov}\{E(Y\mid U),E(R\mid U)\}=0$. What follows about their marginal relationship?
Why can a shared random effect induce marginal association even when the conditional joint law factors?
Does zero covariance imply independence for arbitrary nonbinary $Y$ and $R$?

Solutions. (1) No. Conditional independence is between the vectors $Y$ and $R$; it imposes no independence among components within $Y$. (2) The law of total covariance gives $\operatorname{Cov}(Y,R)=0$. For binary variables, this is equivalent to marginal independence. (3) The marginal joint law averages the products $f_Y(y\mid u)f_R(r\mid u)$ over the same $U$; that average generally differs from the product of the separately averaged distributions. (4) No. For nonbinary variables, zero covariance does not generally imply independence, as the nonbinary example demonstrates.