View-Specific Residual Variance

What View-Specific Residual Variance Represents

A latent factor model summarizes patterns shared across samples and, in a multi-omics setting, may relate those patterns across assays. Even when the factors capture meaningful structure, they do not generally reproduce every observed value. The remaining variation is represented by a residual term. Its variance is view-specific because different assays can have different measurement scales, noise levels, and amounts of variation left unexplained by the factors.

The residual variance is not automatically “technical noise.” It collects variation not represented by the fitted factors, which can include measurement error, biological heterogeneity outside the model, omitted structure, and model misspecification. Its size therefore informs how completely the factors describe a view, but does not by itself identify why variation was left over.

Definition: For a continuous, centered omics view $m$, let $Y^{(m)}$ be its $n \times p_m$ data matrix, $Z$ the $n \times K$ matrix of sample scores, and $W^{(m)}$ the $p_m \times K$ loading matrix. A Gaussian factor model is $$ Y^{(m)} = Z(W^{(m)})^\mathsf{T} + E^{(m)}. $$ A scalar-residual specification assumes that, conditional on the scores, the entries of $E^{(m)}$ are independent with mean zero and variance $\psi_m$, often with a Gaussian distribution. The parameter $\psi_m$ is the view-specific residual variance. A more flexible diagonal specification uses a separate variance $\psi_{jm}$ for feature $j$ in view $m$.

The scalar specification is parsimonious: it assigns the same residual variance to every feature within a view, though the factor-explained variance can differ by feature. The diagonal specification allows features to have different residual variances, while still treating their residuals as uncorrelated in the model. Neither specification says that all views have the same residual variance.

For the variance calculations below, assume the scores are centered with covariance matrix $I_K$, the $K \times K$ identity matrix, and that residuals have mean zero and are independent of the scores. These assumptions give a particularly direct interpretation of the loadings. They are a working model for continuous observations, not a universal description of every assay.

Variance Decomposition Within a View

For feature $j$ in view $m$, write its value in sample $i$ as $y_{ij}^{(m)} = (w_j^{(m)})^\mathsf{T}z_i + \epsilon_{ij}^{(m)}$, where $w_j^{(m)}$ is the feature’s $K$-element loading vector and $\epsilon_{ij}^{(m)}$ is its residual. The factor term is the part represented by the latent scores; the residual is the part the model does not represent.

Because the residual is mean zero and independent of the scores, its covariance with the factor term is zero. The variance of a sum is the sum of the two variances plus twice their covariance. Thus, under the stated assumptions, the variance of a feature has the following decomposition:

\begin{aligned} \operatorname{Var}(y_{ij}^{(m)}) &= \operatorname{Var}\big((w_j^{(m)})^\mathsf{T}z_i\big) + \operatorname{Var}(\epsilon_{ij}^{(m)}) \\ &= (w_j^{(m)})^\mathsf{T} I_K w_j^{(m)} + \psi_m \\ &= \sum_{k=1}^{K}(w_{jk}^{(m)})^2 + \psi_m. \end{aligned}

The factor-explained variance for this feature is therefore $\sum_k(w_{jk}^{(m)})^2$, while $\psi_m$ is its residual variance under the scalar specification. With feature-specific residual variances, replace $\psi_m$ by $\psi_{jm}$. This decomposition is a model-based variance statement; it does not mean the observed residual for any one sample equals its residual variance.

A view-level summary averages over its features. Under the scalar specification, the average factor-explained variance per feature is $\lVert W^{(m)}\rVert_F^2/p_m$, where the squared Frobenius norm is the sum of squared loading entries. The average total variance is this quantity plus $\psi_m$. A corresponding fraction of variance attributed to factors is:

\text{Factor-explained fraction in view }m = \frac{\lVert W^{(m)}\rVert_F^2/p_m} {\lVert W^{(m)}\rVert_F^2/p_m+\psi_m}.

This fraction is interpretable only relative to the model and the chosen feature scales. For a diagonal residual model, the average residual variance is $\sum_{j=1}^{p_m}\psi_{jm}/p_m$, which can be used in the same ratio. Neither ratio should be read as a direct measure of the view’s biological importance.

Worked Example: Decomposing Variance for One Feature

Suppose a hypothetical continuous assay has one latent factor with variance 1. A feature has loading $0.6$ and scalar residual variance $\psi_m=0.64$. Assume the score and residual are independent, the residual has mean zero, and the feature is centered.

The factor-explained variance is the squared loading times the score variance: $0.6^2 \times 1 = 0.36$. The total model variance is therefore $0.36+0.64=1.00$. The factor-explained fraction is $0.36/1.00=0.36$, or 36%; the remaining 64% is residual variance under this model.

The calculation checks the relevant assumptions by using a unit-variance score and an independent, mean-zero residual, as stipulated. If the score variance were not 1, the factor contribution would instead be $0.6^2\operatorname{Var}(z_i)$. The loading alone does not determine a variance contribution without the score scale.

Comparing Residual Variance Across Assays

A larger $\psi_m$ means more residual variation in the measurement units of view $m$, given the fitted factors and the assumed residual structure. It does not automatically mean that the factors are less useful in that view. An assay with a large total variance can have a larger residual variance in absolute units and still have a larger factor-explained fraction. Conversely, a small residual variance can simply reflect a small measurement scale.

When features are centered and standardized to unit variance, absolute residual variances become more comparable across features, though the standardization changes the scale on which the model is interpreted. When features remain on their original scales, report residual variance in those units squared and use a within-view fraction for a scale-relative summary. In either case, the preprocessing choices matter: changing units or scaling features changes the numerical loading and residual variance values.

Worked Example: Comparing Factor Fractions Across Two Views

Consider two hypothetical continuous views, each modeled with scores having covariance $I_K$. In view A, the average squared loading contribution per feature is $1.04$, and $\psi_A=0.26$. In view B, the average squared loading contribution is $0.10$, and $\psi_B=0.10$. Assume the scalar-residual model is adequate within each view.

For view A, the total average variance is $1.04+0.26=1.30$, so the factor-explained fraction is $1.04/1.30=0.800$, or 80.0%. For view B, the total average variance is $0.10+0.10=0.20$, so its factor-explained fraction is $0.10/0.20=0.500$, or 50.0%.

View A has a larger average factor contribution in absolute units and a larger factor-explained fraction. These results support the limited conclusion that the fitted factors account for a greater share of the model-based variation in view A. They do not establish that view A is more biologically important: the assays could have different scales, units, reliability, and feature composition. The comparison assumes that each view’s specified residual variance represents its residual variation reasonably well.

Estimating and Checking the Residual Variance

If the score matrix and loadings are treated as fixed and known, a Gaussian scalar-residual model estimates $\psi_m$ by averaging squared residuals across all $n p_m$ entries. This is the maximum-likelihood estimate conditional on those fixed fitted means. In an actual factor analysis, scores and loadings are estimated too, so this simple conditional formula is not a general degrees-of-freedom-corrected estimator for the full fit. Estimation uncertainty and the fitting procedure must be accounted for when making inferential claims.

\widehat{\psi}_m^{\,\mathrm{cond}} = \frac{1}{n p_m} \sum_{i=1}^{n}\sum_{j=1}^{p_m} \left(y_{ij}^{(m)}-z_i^\mathsf{T}w_j^{(m)}\right)^2

Worked Example: Calculating a Conditional Residual-Variance Estimate

For a hypothetical view with $n=3$ samples and $p_m=2$ features, suppose the residuals after subtracting the specified factor predictions are $\begin{pmatrix}0.2&-0.1\\-0.2&0.3\\0&-0.2\end{pmatrix}$. Treat the predictions as fixed for this conditional calculation.

The sum of squared residuals is $0.2^2+(-0.1)^2+(-0.2)^2+0.3^2+0^2+(-0.2)^2 = 0.04+0.01+0.04+0.09+0+0.04=0.22$. There are $3\times2=6$ residuals, so the conditional estimate is $0.22/6=0.03667$ in squared measurement units.

This calculation checks the model’s residuals against the stipulated predictions and uses the Gaussian conditional likelihood’s denominator, $n p_m$. It does not subtract degrees of freedom for estimating the latent factors. If the predictions were fitted from these same observations, interpreting $0.03667$ as an unbiased estimate of the full model’s residual variance would require additional justification.

Diagnostics should ask whether the residual specification is credible, not merely whether the estimated variance is small. Useful checks include residuals versus fitted values, residual spread by feature, and residual distributions. Patterns, changing spread, strong outliers, or residual correlations can indicate that the scalar independent Gaussian residual assumption is inadequate. A diagonal residual model can accommodate unequal feature-wise variances, but it does not remove residual dependence between features.

A high residual variance can indicate weak factor coverage, but can also reflect high measurement variability or a mismatch between the observation model and the data. A very low estimate may signal an overfit or a scale/preprocessing issue, rather than exceptionally complete biological explanation. Compare residual summaries with the total variation on the same scale and inspect feature-level patterns before drawing conclusions.

Model Choices and Limits of Interpretation

Residual specification	What it represents	Main limitation
One scalar variance per view	Common residual variance across features within an assay	Can misrepresent features with substantially different residual spreads
Diagonal feature-specific variances	Different residual variances by feature, with residuals still uncorrelated in the model	Does not capture residual correlations between features
Structured or correlated residual model	Residual dependencies beyond the shared factors	Requires more parameters and stronger modeling choices

The scalar Gaussian model is most appropriate as a simple observation model for continuous data when residuals are approximately centered, independent as specified, and similarly variable within a view. Feature-specific residual variances relax the equal-spread assumption. Structured residual models may be needed when diagnostics show dependencies not represented by the factors, but greater flexibility also makes estimation more demanding.

For count and binary observations, the variance is tied to the chosen likelihood and its mean, rather than necessarily being a single Gaussian residual variance. A Gaussian residual variance should not be assigned to such observations as though their observation model were continuous Gaussian. The variance decomposition above applies to the stated continuous model; the variance interpretation for other likelihoods must follow their own observation model.

A common mistake is to claim that the assay with the smallest residual variance is the assay best explained by the factors. That reasoning compares absolute values without accounting for total variance and units. The correction is to report residual variance in context and, when useful, compare a factor-explained fraction calculated on a clearly stated scale. Even a larger fraction means better relative reconstruction under the model, not necessarily greater scientific or biological importance.

Key takeaway: View-specific residual variance quantifies variation not represented by the latent factors, under a specified observation model. Interpret its magnitude relative to total variation within that view, its feature scales, and the adequacy of the residual assumptions.

Check Your Understanding

Work through each question, then compare with the solutions.

A feature has loading $0.5$ on a single factor with variance 1 and independent residual variance $0.75$. What are its factor-explained fraction and residual fraction?
View C has average factor-explained variance $2.0$ and residual variance $2.0$. View D has average factor-explained variance $0.2$ and residual variance $0.1$. Which has the larger factor-explained fraction, and what does that comparison not establish?
Why is the sum of squared residuals divided by $n p_m$ not automatically an unbiased full-model estimate when scores and loadings were estimated from the same data?
What assumption is questionable if residual spread increases systematically with fitted values? Would switching to diagonal feature-specific variances necessarily resolve the problem?

Solutions. (1) The factor variance is $0.5^2\times1=0.25$, and total variance is $0.25+0.75=1.00$. The factor fraction is $0.25/1.00=25\%$; the residual fraction is 75%. (2) View C has fraction $2.0/(2.0+2.0)=0.500$; view D has $0.2/(0.2+0.1)=0.667$. D has the larger fraction, but this does not establish that D is more biologically important, nor does it remove dependence on preprocessing and scale. (3) Estimating scores and loadings uses the observations and affects the residuals; the conditional denominator does not account for that fitting, so unbiasedness requires a separate justification. (4) The constant-variance assumption is questionable. Feature-specific variances can address different spreads across features, but do not necessarily fix a spread pattern that changes with fitted values within features.