Goodness-of-Fit Diagnostics for Network Models

Why Network Fit Needs More Than an Edge List

A fitted graphical model summarizes patterns of dependence among features, but a plausible-looking network is not by itself evidence that the model describes the data well. A sparse graph can omit important dependencies; a chosen conditional distribution can have the wrong shape; and a model may perform poorly on samples not used to estimate it. Goodness-of-fit diagnostics examine these possibilities from different angles.

This tutorial focuses on three complementary checks: conditional residuals, held-out likelihood, and whether dependencies implied by the fitted graph are consistent with the data. They diagnose a model as a whole or its conditional components. They do not establish that an edge is causal, and they do not replace uncertainty assessment for selected edges.

Assumptions and What Is Being Checked

Let \(X=(X_1,\ldots,X_p)\) denote the features measured on one sample. A graphical model specifies a joint distribution, or a collection of conditional distributions, for these features. In an undirected graph, a missing edge often encodes a conditional-independence claim, subject to the model’s Markov assumptions. In a directed acyclic graph, the joint distribution is factored into node distributions conditional on their parents. Diagnostics should therefore be tied to the actual model factorization and variable types, rather than to a generic assumption that all omics measurements are Gaussian.

The interpretations below assume that samples used for validation represent the target population and are independent sampling units, or that the validation split respects any dependence structure. For example, repeated measurements from one patient should not be split across training and validation sets as though they were independent patients. Preprocessing, including scaling and imputation, must be learned from training data and applied to validation data without using validation outcomes. Otherwise, held-out performance can be optimistic.

Parameters are estimated, so fitted residuals are not exactly distributed as if the true parameters were known. In-sample residuals can also look artificially small because the model was chosen to fit them. Use them to locate patterns, but give more weight to independent validation where feasible. A diagnostic flags a possible mismatch; it does not identify a unique explanation for that mismatch.

Conditional Residuals

For a continuous feature \(X_j\), suppose the fitted conditional model has mean \(\hat{\mu}_j(x_{-j})\) and standard deviation \(\hat{\sigma}_j(x_{-j})\), where \(x_{-j}\) denotes the other variables or predictors used for that conditional model. The raw residual is the observed value minus the fitted conditional mean. A standardized conditional residual expresses this discrepancy in conditional standard-deviation units.

If the conditional distribution is correctly specified and parameters are known, a Gaussian conditional model implies \(e_{ij}\sim N(0,1)\). With estimated parameters, this is an approximate reference, not an exact finite-sample guarantee. Inspect residuals for shifts, changing spread, outliers, skewness, or patterns against fitted values and predictors. A large residual is a clue to examine the observation and model; it is not, on its own, proof that a particular edge should be added.

For discrete features, subtracting a conditional mean does not generally produce residuals with a common, easily interpreted distribution. One option is a randomized probability integral transform. If the fitted conditional cumulative distribution is \(F_j\), define \(F_j(y^-)\) as the probability of values strictly below the observed \(y\), and draw \(V\sim\mathrm{Uniform}(0,1)\). Then set \(U=F_j(y^- \mid x_{-j})+V P(X_j=y\mid x_{-j})\), and transform it to \(z=\Phi^{-1}(U)\), where \(\Phi\) is the standard normal cumulative distribution function. Under a correctly specified conditional distribution with known parameters, \(U\) is uniform and \(z\) is standard normal. Randomization means repeated diagnostic displays can differ; it does not remove uncertainty from parameter estimation.

A Gaussian Justification for Residual Checks

In a multivariate Gaussian model, the conditional-mean residual for one feature should be uncorrelated with the variables used to predict it. This gives a specific reason to look for residual patterns against those variables. Let \(X\) have covariance matrix \(\Sigma\), and partition it into \(X_j\) and \(X_{-j}\). Under the Gaussian model, the conditional mean is linear with coefficient vector \(\beta=\Sigma_{-j,-j}^{-1}\Sigma_{-j,j}\), assuming the predictor covariance matrix is invertible.

The final equality follows by substituting \(\beta^\mathsf{T}=\Sigma_{j,-j}\Sigma_{-j,-j}^{-1}\). Because \(R_j\) and \(X_{-j}\) are jointly Gaussian, zero covariance implies independence. Thus systematic residual dependence on predictors is inconsistent with this Gaussian conditional specification. In non-Gaussian models, zero correlation alone is not enough to establish independence; inspect conditional distributions and use diagnostics suited to the variable type.

Held-Out Log Likelihood

A predictive score asks whether a fitted model assigns high probability or density to observations not used to fit it. For held-out samples \(x_1,\ldots,x_m\), the average log likelihood is \(\frac{1}{m}\sum_{i=1}^m\log \hat{p}(x_i)\), where \(\hat{p}\) is the fitted joint probability mass function or density. A larger value is better for the same held-out observations and response representation. Continuous log densities can be positive or negative and depend on measurement units, so comparisons require the same variables, units, and scoring convention.

If only nodewise conditional models are fitted, one may instead sum or average the conditional log scores, often called a pseudolikelihood score. It is a score for the collection of conditionals, not necessarily the joint log likelihood of a normalized joint model. Compare models using the same nodes and held-out samples. Do not compare a joint likelihood for one model with a conditional pseudolikelihood for another as though they were identical quantities.

Log score rewards assigning probability to what occurred and penalizes assigning very low probability to an observed outcome. It is a proper scoring rule: in expectation, the true probability distribution maximizes expected log score among candidate distributions, provided the expectation is taken under that true distribution. A single finite validation set can nevertheless favor a model by chance. Use an appropriate split or cross-validation plan, and report the variability of performance when the available sample size permits.

Checking Dependencies Implied by the Graph

A network encodes statements about which variables should remain conditionally related after accounting for specified variables. One direct diagnostic is to examine whether a relationship that the fitted graph says should disappear is still visible after conditioning. For example, in a Gaussian undirected chain \(X_1-X_2-X_3\) with no \(X_1-X_3\) edge, the graph implies \(X_1\perp X_3\mid X_2\), under the model’s Markov assumptions. In a directed graph, conditional-independence claims must instead follow from its factorization and d-separation; simply seeing no direct arrow is not sufficient.

For the Gaussian chain, fit the relevant conditional means and calculate residuals for \(X_1\) and \(X_3\) after accounting for \(X_2\). A residual association can indicate that the claimed conditional independence is not adequately represented. The check is diagnostic rather than definitive: finite-sample fluctuation, an incorrect functional form, an unmeasured common cause, or measurement error can all produce an apparent residual relationship.

Putting the Diagnostics Together

Different diagnostics reveal different failures. Conditional residuals localize misfit to a node or conditional distribution. Held-out likelihood summarizes predictive performance across the scored outcomes, but does not by itself explain which component is poor. Checks of implied dependencies target particular conditional-independence claims, but a residual association may reflect a misspecified conditional form rather than a missing edge. A useful assessment uses more than one check and interprets them in the model’s context.

Common failure modes include leakage of validation information into preprocessing or tuning, treating repeated samples from one person as independent, and using Gaussian residual rules for a count or binary feature. Another mistake is to interpret a better held-out score as proof that every edge is correct. Predictive fit evaluates distributions on scored observations; it does not uniquely identify a network structure, and association in a fitted network is not a causal conclusion.