Marginalization, Latent Variables, And Spurious Graph Edges

Why Marginalizing Changes a Graph

A Gaussian graphical model describes conditional relationships through zeros in a precision matrix. But a study may record only some of the variables in the model. If other variables are unobserved and are integrated out, the distribution of the recorded variables remains Gaussian, yet its precision matrix can have a different pattern of zeros. A sparse graph containing latent nodes can therefore correspond to a much denser graph among the observed variables.

The distinction matters when interpreting a fitted graph. An edge in the graph of observed variables represents conditional dependence given the other observed variables. It does not, by itself, establish a direct edge between those variables in a larger model that includes latent variables.

Joint and Marginal Precision Matrices

Let \(X\in\mathbb{R}^p\) be the observed vector and \(Z\in\mathbb{R}^q\) the latent vector. Suppose the centered joint vector \((X^\mathsf{T},Z^\mathsf{T})^\mathsf{T}\) is multivariate Gaussian with positive-definite precision matrix

where \(A\) is \(p\times p\), \(C\) is \(q\times q\), and \(B\) is \(p\times q\). Positive definiteness of \(\Omega\) implies that its principal block \(C\) is positive definite, so \(C^{-1}\) exists.

The block \(A\) has a different interpretation from the observed marginal precision. In the joint Gaussian density, terms depending on \(x\), with \(z\) held fixed, have quadratic part \(x^\mathsf{T}Ax\). Thus \(A\) is the conditional precision of \(X\) given \(Z\). It need not be the precision matrix of \(X\) after \(Z\) has been integrated out.

The Marginal Precision Is a Schur Complement

Proof. Up to its normalizing constant, the joint density has exponent

Complete the square in \(z\):

Integrating over \(z\) integrates a shifted Gaussian kernel with precision \(C\). Its integral is a finite constant independent of \(x\), because \(C\) is positive definite. The remaining density in \(x\) is therefore proportional to \(\exp\{-x^\mathsf{T}Sx/2\}\), where \(S=A-BC^{-1}B^\mathsf{T}\). To check that this is a valid positive-definite precision matrix, take any nonzero \(x\) and set \(z=-C^{-1}B^\mathsf{T}x\). Then

since the vector in the quadratic form is nonzero and \(\Omega\) is positive definite. Hence \(S\) is positive definite, and it is the marginal precision of \(X\). \(\square\)

For the graph, the key entrywise identity is

where \(b_i^\mathsf{T}\) is row \(i\) of \(B\). Thus, even if \(A_{ij}=0\), the marginal precision entry \(S_{ij}\) can be nonzero. Conversely, a nonzero \(A_{ij}\) can be canceled by the second term. Since \(C^{-1}\) is positive definite, \(b_i^\mathsf{T}C^{-1}b_j\) is an inner product under the \(C^{-1}\) metric; it may be zero, positive, or negative.

Induced Edges and Exact Cancellation

The matrix \(BC^{-1}B^\mathsf{T}\) has rank at most \(q\), the number of latent variables. Low rank does not mean sparse: even a rank-one matrix can have a nonzero entry for every pair of variables. Marginalization can therefore add many edges at once. But it does not necessarily add every possible edge: the formula for \(S_{ij}\) allows both zero inner products and exact cancellation.

Conditional and Marginal Graphs Answer Different Questions

The joint graph describes conditional independences among all variables when the other joint variables are conditioned on. The observed-variable graph describes conditional independences among recorded variables when conditioning only on the other recorded variables. Once latent variables are omitted from the conditioning set, they can transmit dependence between observed variables. This is why taking the observed nodes of a joint graph and simply deleting its latent nodes does not generally produce the observed-variable precision graph.

How to Interpret an Observed Edge

In an observed-variable precision graph, an edge means conditional dependence given the other observed variables under the fitted Gaussian model. It does not tell us whether the dependence arises from a direct interaction, an omitted common influence, or a combination of pathways. In particular, an edge induced by a latent variable is a real edge in the marginal graph, not a computational error; it simply has a different interpretation from an edge in a larger joint graph.

When latent variables are plausible, it is useful to state which graph is being discussed. A sparse joint precision matrix can have a dense observed marginal precision matrix. Conversely, exact cancellations can make a joint edge disappear from the marginal graph. Neither relationship can be inferred by merely inspecting the observed-node subgraph of the joint graph; the Schur complement determines the marginal pattern.

Common Mistake