Diagnostics For Fitted Gaussian Graphical Models

Why Diagnose a Fitted Graph?

A sparse precision estimate can produce a plausible-looking graph without adequately describing the data. An omitted nonlinear relationship, a heavy-tailed variable, or a small number of unusual observations can affect both the fitted conditional distributions and the selected edges. Diagnostics therefore ask several distinct questions: do the conditional models leave systematic residual patterns, does the fitted distribution predict new observations well, do selected edges recur under resampling, and does a particular observation substantially change the fit?

These checks are complementary, not interchangeable. A graph can predict well while some individual edges are unstable, or an edge can recur reliably even when the Gaussian distribution misses tail behavior. In high dimensions, residuals and edge frequencies are also affected by estimation and tuning. They are evidence for assessing a model, not automatic certificates that it is correct.

Conditional Residuals

Let \(X\in\mathbb{R}^p\) have a multivariate Gaussian distribution with mean \(\mu\) and positive-definite precision matrix \(\Omega\), so its covariance matrix is \(\Omega^{-1}\). The conditional distribution of one coordinate given all the others is particularly useful for checking a Gaussian graphical model.

Under the model with its true parameters, each \(r_{ij}\) has a standard normal marginal distribution. The residual checks whether the observed coordinate is surprising given the other coordinates according to the fitted conditional model. Plotting residuals against fitted conditional means, other variables, or time can reveal trends, changing variance, clusters, or extreme tails that a Gaussian conditional distribution does not capture.

For a fitted model, replace \(\mu,\Omega\) by estimates. Training residuals are not exactly standard normal: the same observations helped determine those estimates, and regularization adds further dependence. When feasible, calculate diagnostics on held-out observations or use cross-fitting, in which each observation is evaluated using a model fitted without it. Examine residuals across variables and observations rather than interpreting one large value in isolation.

Held-Out Log Likelihood

Residual plots are useful for finding particular patterns, while held-out likelihood summarizes predictive performance. Suppose a model fitted to training data gives mean \(\hat{\mu}\) and positive-definite precision estimate \(\hat{\Omega}\). Its log density for a held-out vector \(x\) is \[ \ell(x)=-\frac{p}{2}\log(2\pi)+\frac{1}{2}\log\det(\hat{\Omega})-\frac{1}{2}(x-\hat{\mu})^\mathsf{T}\hat{\Omega}(x-\hat{\mu}). \] For a held-out set, add these contributions or report their average. Larger average log likelihood means better predictive density on that set. All candidate models must be scored on the same held-out observations and with the same treatment of the mean and variables.

To verify the inequality, put \(U=p(X)/q(X)\) for \(X\sim q\), assuming \(p\) is positive wherever \(q\) is positive. Then \(E_q[U]\leq 1\), allowing for candidate probability outside the support of \(q\). Jensen's inequality for the concave logarithm gives \(E_q[\log U]\leq\log E_q[U]\leq 0\). Rearranging yields \(E_q[\log q(X)]-E_q[\log p(X)]\geq0\), which is the claimed result. Equality requires \(p=q\) \(q\)-almost everywhere. \(\square\)

This result concerns expected predictive score, not the score on every finite test set. A single held-out set can favor a worse model by chance. In high dimensions, the fitted precision must give a valid positive-definite density for this Gaussian score to be defined. Keep all fitting, scaling, and tuning decisions inside the training data; using held-out observations to choose the model makes the reported score optimistic.

Edge Stability and Observation Influence

An edge in a fitted graph is a selected nonzero off-diagonal precision entry. Its recurrence across resamples is a useful sensitivity diagnostic. For a chosen resampling procedure, define the selection frequency of edge \((j,k)\) as the fraction of resampled fits in which that edge is selected. The procedure should specify the subsampling scheme, fitting method, and how tuning is chosen. If tuning is repeated within each resample, the frequency reflects the full selection pipeline; if tuning is fixed, it reflects stability conditional on that choice.

A high frequency indicates that an edge is relatively insensitive to the particular resamples used, not that the edge is certainly a true conditional dependence. Frequencies can change with sample size, tuning, scaling, or the resampling design. Compare the frequencies of scientifically important edges and inspect whether the same broader neighborhoods recur, rather than treating a universal cutoff as a guarantee.

An observation is influential when removing it and refitting materially changes quantities of interest, such as selected edges, estimated precision entries, or held-out predictions. An unusually large conditional residual can prompt this check, but unusualness alone does not establish influence. Conversely, a point may affect covariance structure without having an extreme residual under a model that has already been pulled toward it. A practical analysis records the change from the full fit to a leave-one-out fit and checks whether the change is large relative to ordinary resampling variation.

Common Pitfall and Practical Use

Wrong reasoning: “Every fitted conditional residual should be an independent standard normal, so I can flag every large training residual against a standard normal cutoff and trust the remaining graph.” This fails twice. Estimated training residuals are not independent standard normal observations, and checking many coordinates creates many chances for an apparently extreme value. In addition, a residual check does not establish that selected edges are stable or that new observations receive good predictive scores.

Correction: use held-out or cross-fitted residuals where possible, look for systematic patterns across observations and variables, and interpret extremes with the number of checks and the fitting procedure in mind. Pair those checks with held-out log likelihood, resampling-based edge stability, and targeted deletion refits for potentially influential cases. These diagnostics answer different questions and should be reported as such.