Deriving Block Weights for Joint NMF

Why Joint NMF Needs Block Weights

Joint NMF uses a common sample-score matrix to represent several omics blocks. Even when every block is fitted with a Frobenius-norm reconstruction loss, simply adding the losses does not make the modalities contribute comparably. A block with more entries, larger numerical values, or larger residuals can dominate the shared scores. Block weights make this trade-off explicit.

The purpose here is to derive weights from a chosen target: how much should each block's relative squared reconstruction error count? This is one principled choice, not a universal default. It differs from weighting by measurement precision or by the number of entries, and its interpretation depends on the data scales and the modeling goal.

The blocks must refer to the same samples in the same order for a shared \(H\) to have this interpretation. The matrices and factors are nonnegative, and the rank \(r\) is chosen in advance for this objective. The squared-error loss is appropriate when squared deviations are a meaningful measure of reconstruction discrepancy. A Gaussian-error motivation is possible, but only with additional assumptions; the weighted objective alone does not establish that the residuals are independent Gaussian errors or that \(\lambda_m\) are inverse variances.

Deriving Weights for Relative Reconstruction Error

For a nonzero block, define its relative squared reconstruction error as the squared residual norm divided by the squared norm of the observed block. This dimensionless quantity measures residual size relative to the block's overall scale. Let \(\alpha_m\) specify the desired importance of that relative error. To keep every block active under the objective's positive-weight convention, require \(\alpha_m>0\) for all \(m\), with \(\sum_{m=1}^M\alpha_m=1\). Also require \(\|X^{(m)}\|_F>0\) for every block.

Match the weighted Frobenius objective to this target term by term. The coefficient multiplying the residual norm squared in \(J_{\mathrm{rel}}\) must be \(\alpha_m/\|X^{(m)}\|_F^2\). Therefore the block weight is:

This is an identity, not an approximation: with these fixed weights, each block's contribution to the objective is exactly one half of its assigned importance times its relative squared error. Equal \(\alpha_m\) values give equal importance to relative errors, not equal importance to absolute residual sums. Other positive \(\alpha_m\) values encode a deliberate priority, and must be chosen before interpreting the fitted solution.

Weights also determine how each modality's residual affects the common score matrix. Differentiating the objective with respect to \(H\), while holding the \(W^{(m)}\) fixed, gives a sum of block-specific gradient contributions:

Thus, changing a block weight changes its contribution to the optimization of the shared scores. It does not merely change the printed value of the loss. The derivative follows by expanding each squared Frobenius norm as a trace, differentiating the quadratic in \(H\), and using that the derivative of \(\frac{1}{2}\|W^{(m)}H-X^{(m)}\|_F^2\) is \((W^{(m)})^\mathsf{T}(W^{(m)}H-X^{(m)})\). Nonnegativity constraints still apply to both factors; this gradient expression does not itself specify a constrained optimization algorithm.

Scaling Choices and Their Consequences

Unweighted reconstruction, \(\lambda_m=1\), gives each block a contribution proportional to its absolute squared residual norm. It can be reasonable when absolute discrepancies are comparable across blocks and their units and scales are meaningfully commensurate. But a large block norm may arise from having many features, larger measurements, or both. Then raw Frobenius loss may emphasize that block even if its relative fit is no better.

Relative-error weighting divides by the total squared norm of each block. It does not divide by the number of entries. A different choice, \(\lambda_m=1/(p_m n)\), weights the mean squared residual per entry equally across blocks. That can address block-size differences but does not adjust for different measurement scales. A precision-based choice instead uses weights motivated by residual variability: under a suitable independent Gaussian residual model with common variance within block \(m\), inverse-variance weights are proportional to \(1/\sigma_m^2\). This has a statistical interpretation only if the error model is defensible and variance estimates are appropriate. It is not generally the same as normalizing by \(\|X^{(m)}\|_F^2\).

A useful invariance clarifies the role of normalization. If every entry of block \(m\) is multiplied by a positive constant \(c\), its squared norm and squared residual norm both multiply by \(c^2\), provided its reconstruction is scaled consistently. Its relative squared error is unchanged. Its corresponding \(\lambda_m\) becomes \(\lambda_m/c^2\), so the weighted residual contribution remains unchanged. Raw unweighted loss does not have this property: changing units can alter the relative influence of a block on the shared factors.

Assumptions, Diagnostics, and Limitations

Before applying relative-error weights, verify that the blocks contain the same samples in a matched order, are represented on scales appropriate for a nonnegative factorization, and have nonzero Frobenius norms. The importances should be strictly positive for the formula above and for the objective convention \(\lambda_m>0\). If a block is identically zero, its relative error is undefined; remove it as uninformative for this criterion or define another objective rather than dividing by zero. If an importance is set to zero in an alternative formulation that permits \(\lambda_m\geq0\), that block is excluded from the criterion and no longer influences the fit.

Check the actual weighted contributions \(\frac{1}{2}\lambda_m\|X^{(m)}-W^{(m)}H\|_F^2\), not only the raw residual norms. Also check each relative error and the selected weights. A very small observed-block norm can produce a very large weight for a given importance, making the objective sensitive to modest absolute residuals. A large weight is not automatically a numerical error, but it warrants checking the block scale, preprocessing, and intended meaning of relative error.

The choice \(\lambda_m=\alpha_m/\|X^{(m)}\|_F^2\) has limitations. A single block-wide norm can conceal feature-specific scales, sparsity, or noise differences. Squared error is sensitive to large residuals, and equal relative-error importance may not be scientifically appropriate. Weights and importances are modeling decisions, not facts inferred by the formula. Report them, explain their rationale, and examine whether conclusions are sensitive to reasonable alternatives. Weight selection is distinct from choosing a rank or deciding how to represent shared and modality-specific structure.