Residual Diagnostics for NMF Fits

Why Inspect NMF Residuals?

A low NMF objective does not by itself show that a reconstruction captures the relevant structure in every feature, sample, or omics modality. A total error can conceal a group of poorly reconstructed features, one unusual sample, or a consistent pattern left in a particular block. Residual diagnostics make these patterns visible. They help assess where a fitted reconstruction is inadequate; they do not establish that the model is correct or that a residual pattern has a particular biological cause.

For an omics block, let \(X^{(m)}\) be a nonnegative matrix with \(p_m\) features in rows and \(n\) aligned samples in columns. A rank-\(K\) NMF reconstruction is \(\widehat X^{(m)}=W^{(m)}H\), where \(W^{(m)}\) is a nonnegative feature-by-component matrix and \(H\) is a nonnegative component-by-sample matrix. In a joint fit, the shared \(H\) links samples across blocks, although some models may also include block-specific terms. The diagnostic object is the difference between the observed matrix and the reconstruction on the scale being examined.

These are descriptive residuals, not automatically standardized errors with a known reference distribution. Their interpretation depends on the data scale, preprocessing, missingness pattern, and objective used to fit the model. If squared-error loss was used, squared residuals directly contribute to the fitting objective. With another beta-divergence loss, signed differences can still be inspected, but they are not the loss contributions and may not reflect the fitting criterion well.

Feature-Wise and Sample-Wise Summaries

A residual table is often too large to inspect cell by cell. For a complete, equally weighted matrix fitted with squared-error loss, a feature-wise sum of squared residuals identifies features contributing most to the total error. Dividing by the number of observed samples gives a mean squared residual; its square root is a root mean squared residual (RMSE), in the original units. Sample-wise summaries are defined in the same way across features.

Here \(n_i\) is the number of observed entries for feature \(i\), and \(p_{mj}\) is the number observed for sample \(j\). For complete matrices these are \(n\) and \(p_m\). With missing entries, calculate summaries only over observed entries and report the denominators; otherwise samples or features with more missing values can appear artificially well fitted. A sum of squares measures total error and grows with the number of observations, while an RMSE is an average-scale summary. Neither removes the effect of feature-specific variability or measurement noise.

Comparing Omics Blocks and Examining Residual Dependence

Raw residual sums of squares should not be used alone to decide which modality fits worse. Their size depends on the block's scale and number of entries. One descriptive comparison is the relative Frobenius error, the square root of the residual sum of squares divided by the observed sum of squares. It is dimensionless, but it can be unstable when the observed sum of squares is close to zero and is not a substitute for checking the preprocessing and scientific meaning of each block.

Residual dependence can reveal coordinated structure the factorization has not captured. For example, residuals for two features can move together across samples, or two modality residual profiles can align across matched samples. A simple descriptive measure is cosine similarity, the residual dot product divided by the product of Euclidean lengths. It measures directional alignment without centering; it is not Pearson correlation and does not test independence. Pairwise residual summaries are most interpretable when the features or samples are meaningfully matched and the scales have been considered.

Derivation, Diagnostics, and Limits

The feature-wise and sample-wise squared-error summaries are two ways to partition the same total squared error. For a complete matrix, finite sums can be regrouped by rows or columns:

This identity is useful as a calculation check. It also explains why a single total cannot reveal where error is concentrated: the total remains unchanged whether its contributions are spread across all entries or concentrated in one feature or sample.

A practical review combines summaries with the signed residual matrix. Plot residuals against fitted values, inspect feature and sample summaries, and look for contiguous or repeated patterns when features have a meaningful order. In a joint fit, examine each block separately before interpreting a pooled summary. Patterns to investigate include:

• Feature-wise bias: residuals for one feature repeatedly have the same sign, suggesting a systematic mismatch for that feature.
• Sample-wise concentration: a sample has unusually large residuals across many features, which may reflect unusual biology, data quality, or preprocessing rather than a single cause.
• Structured dependence: groups of residuals move together, or an ordered feature region shows a pattern, suggesting structure not represented by the current reconstruction.
• Block-specific error: one modality has a larger scale-aware error or a distinctive residual pattern, which may indicate a modality-specific mismatch or an unsuitable comparison scale.

Before assigning meaning to any pattern, check that samples are aligned correctly across blocks, residuals are calculated on the intended scale, feature transformations are appropriate, and missing entries are excluded consistently. For count matrices, a raw difference can be hard to interpret when variability depends on the mean; the diagnostic should be considered alongside the count objective and the data's scale. If the fit used a non-squared beta-divergence, inspect the corresponding objective contributions as well as signed differences. If sparse outlier corrections were included, clarify whether the diagnostic residual is computed before or after those corrections: the two answer different questions about the factor reconstruction and the model's full fitted representation.

Residuals are not independent observations simply because they are displayed in a matrix. The factors were estimated from the data, which constrains the residuals and can induce dependence. Large residuals are not automatically outliers, and small residuals do not guarantee a good model: a high-dimensional reconstruction can fit noise, while a large residual can reflect a legitimate feature with a different scale. Residual diagnostics are descriptive checks, not a replacement for assessing rank, preprocessing, or scientific plausibility.

Common Mistake and Alternatives

Other useful views answer different questions. Absolute residuals emphasize magnitude but hide whether the model over- or under-reconstructs. Squared residuals emphasize large deviations and match a squared-error objective. Relative residuals can compare scale but become unstable near zero. For a non-squared fitting objective, its per-entry divergence contributions are closer to the criterion actually optimized, but they do not provide signed direction. These diagnostics complement one another; they should not be treated as interchangeable rankings.