Why Conditional Independence Does Not Mean Marginal Independence
Repeated outcomes from the same person, patients treated at the same center, or measurements from the same school may resemble one another. A GLMM can represent this dependence by assigning each cluster a latent random effect. Observations in a cluster share that effect, so their outcomes can be dependent in the population even when they are independent after conditioning on the effect.
The distinction is between conditional and marginal dependence. Conditional statements hold after the random effect is fixed. Marginal statements average over its distribution. GEE takes a different route: it specifies a marginal mean model and uses a working correlation to represent within-cluster association, without requiring a latent random effect to generate that association.
Here \(g\) is the link function, \(\beta\) is a vector of fixed effects, and \(z_{ij}\) describes how the random effects enter the linear predictor. Clusters are typically assumed independent of one another, conditional on the study design. The random-effects distribution, such as a mean-zero normal distribution with a specified covariance matrix, is also part of the model. The outcomes’ conditional distributions must be appropriate for their type, such as Bernoulli for a binary response or normal for a continuous response.
Deriving the Dependence
The law of total covariance separates dependence into an average conditional covariance and covariance between the conditional means. Write \(\mu_{ij}(b_i)=E(Y_{ij}\mid b_i)\), with covariates held fixed. For two different observations \(j\) and \(k\) in the same cluster,
Under conditional independence, the first term is zero. Thus the marginal covariance is the covariance, across the random-effects distribution, of the two conditional means. The marginal variance of one outcome has an additional component:
These identities explain the mechanism: if the same cluster effect tends to raise both conditional means, outcomes from that cluster tend to be positively associated. But random effects do not guarantee positive covariance in every model. The way the random effects enter at each observation matters.
Worked Examples
Worked Example: Gaussian Random Intercept
Suppose a hypothetical study records a continuous response at two occasions for each person. For illustration, assume the conditional model \(Y_{ij}=\beta_0+\beta_1x_{ij}+b_i+\varepsilon_{ij}\), where \(b_i\sim N(0,4)\) and the independent errors satisfy \(\varepsilon_{ij}\sim N(0,9)\). The random intercept and errors are mutually independent. The fixed effects describe the conditional mean, while \(b_i\) shifts both observations for person \(i\).
For two distinct occasions in the same person, the shared term is \(b_i\), while the occasion-specific errors are independent. Therefore,
For either occasion, the marginal variance is \(4+9=13\). Hence the marginal correlation is \(4/\sqrt{13\cdot13}=4/13=0.3077\). In this model the random intercept contributes the shared variation, and the residual variance contributes variation specific to each occasion. A hypothetical pair of responses such as 8 and 11 for one person would not, by itself, determine this correlation; the calculation follows from the stipulated model variances and independence assumptions.
Worked Example: Dependence in a Binary Random-Intercept Model
Consider two hypothetical binary responses in each cluster. Let \(P(Y_{ij}=1\mid b_i)=\operatorname{logit}^{-1}(b_i)\), and suppose \(b_i\) equals \(-1\) or \(1\), each with probability \(0.5\). Assume the two outcomes are conditionally independent given \(b_i\). This simple two-point random-effects distribution makes the averaging explicit; it is used here to illustrate the mechanism.
The conditional success probabilities are \(\operatorname{logit}^{-1}(-1)=0.2689\) and \(\operatorname{logit}^{-1}(1)=0.7311\). The marginal success probability for either observation is
Since the outcomes are conditionally independent, their marginal covariance is the variance of the shared conditional probability:
Each marginal outcome is Bernoulli with success probability \(0.5\), so its variance is \(0.5(1-0.5)=0.25\). The marginal correlation is \(0.05339/0.25=0.2136\). A cluster with a high value of \(b_i\) has higher success probabilities for both responses, producing positive association after the random effect is averaged out.
Worked Example: A Random Slope Can Produce Negative Covariance
Suppose a hypothetical continuous response follows \(Y_{ij}=\beta_0+\beta_1t_{ij}+b_{0i}+b_{1i}t_{ij}+\varepsilon_{ij}\). Let \(b_{0i}\) and \(b_{1i}\) be independent \(N(0,1)\) random effects, with independent \(N(0,1)\) errors. Consider two observations at times \(t=-2\) and \(t=2\) in the same cluster.
The random part of the conditional mean at time \(t\) is \(b_{0i}+t b_{1i}\). Therefore, the covariance between the two conditional means is
The residual errors are independent, so the marginal covariance of the two responses is also \(-3\). At either time, the marginal variance is \(1+4+1=6\): one unit from the random intercept, four from the random slope, and one from the residual error. Thus the marginal correlation is \(-3/\sqrt{6\cdot6}=-0.5000\). The shared random effects still induce dependence, but the slope effect moves the two conditional means in opposite directions at these times.
When a Random-Intercept Covariance Is Positive
For a random-intercept model, suppose the conditional mean at each of two observations is a nondecreasing function of the same scalar random intercept \(b_i\). Their conditional means then move in the same direction as \(b_i\) changes, so their covariance is nonnegative. One justification uses an independent copy \(B_i'\) of \(b_i\). For nondecreasing functions \(f\) and \(h\),
The inequality holds because the two differences always have the same sign. In particular, the inverse-logit function is increasing, so a logistic random-intercept model with the same intercept entering both observations has nonnegative covariance under conditional independence. This is a statement about a shared scalar intercept; the random-slope example shows why it should not be generalized to every random-effects structure.
How This Differs From GEE Working Correlation
A GLMM and GEE can both account for clustered outcomes, but they describe dependence differently. In a GLMM, the random effect is a latent cluster-level variable with a probability distribution. The conditional outcome model and random-effects distribution together imply a marginal covariance. In GEE, the mean is specified at the population level, and a working correlation is used to represent within-cluster association for estimating that mean model. A working correlation is not, by itself, a claim that a particular latent variable generated the dependence.
| Feature | GLMM random effects | GEE working correlation |
|---|---|---|
| Primary representation | Latent cluster effects enter a conditional model | Association among observations is represented directly in a working correlation |
| Source of modeled dependence | Shared random effects induce dependence after averaging | A specified working structure represents within-cluster association |
| Mean being modeled | Conditional mean given random effects; marginal means follow by averaging | Population-averaged mean |
| Interpretation | Includes cluster-specific effects conditional on the random effects | Targets population-averaged effects |
The distinction matters when the scientific question is about cluster-specific effects or the distribution of cluster differences: the latent-effects formulation makes those quantities part of the model. If the goal is a population-averaged mean and an explicit latent distribution is not needed, GEE provides a direct alternative. Neither representation automatically repairs a misspecified mean model, and a GLMM’s implied covariance can be wrong if its random-effects structure is inappropriate.
Assumptions, Diagnostics, and Common Pitfalls
The covariance derivation depends on the model’s conditional-independence assumption for the two outcomes. If residual dependence remains after conditioning on the included random effects and covariates, then the conditional covariance term in the law of total covariance is not zero. The model may need a richer dependence representation; a shared intercept alone does not account for every possible temporal or spatial pattern.
- Check whether the random-effects structure is plausible. Cluster-level patterns over time may indicate that a random slope, rather than only a random intercept, is needed. The chosen structure should reflect how observations are expected to vary together.
- Check residual patterns after fitting. Persistent within-cluster patterns not accounted for by the fitted model are a warning that conditional independence or the mean structure may be inadequate. Residual diagnostics should be interpreted with the response type and model in mind.
- Remember that distributional assumptions matter. The assumed random-effects distribution helps determine the implied marginal means and dependence. A convenient distribution is not proof that the latent cluster effects follow it.
A common mistake is to say, “The outcomes are conditionally independent, so they are independent.” The correction is that conditional independence does not generally imply marginal independence: averaging over a shared random effect can create covariance. Another mistake is to claim that a random intercept always gives the same positive correlation for every pair of observations. Even with a random intercept, marginal variances and correlations can depend on the outcome distribution and covariates; with random slopes, covariance can change sign across measurement occasions.
Check Your Understanding
Work through each question, then compare your reasoning with the solutions below.
- In a Gaussian random-intercept model, the random-intercept variance is 2 and the independent residual variance is 6. What is the within-cluster covariance and correlation between two observations?
- In a conditionally independent binary random-intercept model, both conditional success probabilities increase with the same scalar random intercept. What can be said about the sign of the marginal covariance, and why?
- For a Gaussian random-intercept and random-slope model with independent random effects of variance 1 each and residual variance 1, calculate the covariance and correlation at times \(t=-1\) and \(t=1\).
- In one sentence, distinguish the dependence representation in a GLMM from GEE working correlation.
Solutions.
- The covariance is the random-intercept variance, \(2\). Each marginal variance is \(2+6=8\), so the correlation is \(2/\sqrt{8\cdot8}=2/8=0.2500\).
- The covariance is nonnegative. Both conditional means are nondecreasing functions of the same random intercept, so their covariance across the random-effects distribution is nonnegative; conditional independence makes this the marginal covariance.
- The covariance of the random parts is \(1+(-1)(1)=0\). Each marginal variance is \(1+1+1=3\), so the correlation is \(0/\sqrt{3\cdot3}=0\). Shared effects do not guarantee nonzero covariance at every pair of times.
- A GLMM uses latent random effects whose averaging induces marginal dependence, whereas GEE specifies a population-averaged mean and represents association using a working correlation.