Observation Models For Continuous Omics

From Latent Scores to Continuous Observations

In a factor model, latent scores describe sample-level positions, while loadings describe how those positions relate to measured features. For continuous omics measurements, an observation model specifies how a measurement varies around the value predicted by those scores and loadings. That specification matters: the latent part describes structured signal, while the residual part describes variation not captured by the model.

Let \(Y_{ivp}\) be the continuous measurement for sample \(i\), modality \(v\), and feature \(p\). Let \(z_i\) be the vector of latent scores for sample \(i\), and let \(\lambda_{vp}\) be the loading vector for feature \(p\) in modality \(v\). Write \(m_{vp}\) for the feature's intercept. The linear predictor for that measurement is \(m_{vp}+\lambda_{vp}^{T}z_i\). The observation model adds a residual error to this predictor.

The conditional statement is important. The mean and variance specified here describe the observation once the latent score is given. The model does not, by itself, specify a probability distribution for \(z_i\), and therefore does not determine an unconditional distribution for \(Y_{ivp}\). A distribution for the scores would be needed for that additional step.

The assumptions can be separated. First, the conditional mean is linear in the scores, with an intercept that does not vary across samples. Second, the residual has conditional mean zero, so the model's predictor is the conditional mean. Third, the residual variance is positive and does not depend on the sample or its score in this baseline specification. Finally, residuals are Gaussian and conditionally independent. Independence means that, after conditioning on the scores, one residual does not provide information about another residual under the model. It does not mean that the observed measurements themselves must be independent.

Why the Gaussian Density Has This Form

The model expresses each observation as a fixed conditional mean plus a Gaussian residual. If \(\varepsilon_{ivp}\mid z_i\sim N(0,\psi_{vp})\), adding \(m_{vp}+\lambda_{vp}^{T}z_i\) shifts the residual distribution without changing its variance. This gives the stated conditional distribution. Its density at an observed value \(y_{ivp}\) is

The difference in square brackets is the residual \(e_{ivp}=y_{ivp}-(m_{vp}+\lambda_{vp}^{T}z_i)\). The density is measured per unit of the measurement scale; it is not a probability that the observation equals one exact value. Probabilities apply to intervals and are obtained by integrating the density over those intervals.

Under conditional independence, the likelihood for all observed entries is the product of their conditional densities. For a particular feature \(p\) in modality \(v\), suppose there are \(n_{vp}\) observations and treat their conditional means as fixed. The part of the log likelihood involving \(\psi_{vp}\) is

To justify the result, write \(S=\sum_i e_{ivp}^{\,2}\). Differentiating the log likelihood with respect to \(\psi_{vp}\) gives \(-n_{vp}/(2\psi_{vp})+S/(2\psi_{vp}^{2})\). When \(S>0\), setting this derivative to zero yields \(\psi_{vp}=S/n_{vp}\). The derivative is positive when \(0<\psi_{vp}<S/n_{vp}\) and negative when \(\psi_{vp}>S/n_{vp}\), so this stationary point maximizes the likelihood. When \(S=0\), the log likelihood is \(-n_{vp}\log(2\pi\psi_{vp})/2\), which increases without bound as \(\psi_{vp}\) approaches zero from above. Since zero is not an allowed variance, there is no finite maximum.

Choosing the Residual Variance Structure

Allowing \(\psi_{vp}\) to differ by feature is a flexible baseline: a high-noise feature can have greater residual variance than a low-noise feature. A simpler model may instead use a single variance \(\psi_v\) for every feature in modality \(v\), or one variance shared across several modalities. These restrictions can reduce the number of variance parameters, but they assert that the relevant features have the same residual scale. That assumption should be checked rather than inferred from convenience.

A variance is expressed in squared units. If a measurement is in log-expression units, its residual variance is in squared log-expression units. Multiplying all measurements by a constant changes the variance by the square of that constant. Consequently, the variance assumptions depend on the measurement scale and any transformations or standardization used before fitting.

Dependence, Diagnostics, and Failure Modes

Conditional independence is a substantive restriction. It says that the latent scores and the linear predictors have accounted for the modeled dependence, leaving no residual association between entries. Residual correlation may remain when, for example, features share technical artifacts, samples share a batch effect not represented in the predictor, or the factor structure fails to capture a systematic pattern. A diagonal residual variance model cannot represent such remaining dependence.

A useful diagnostic is to examine residuals after fitting the model, preferably scaled by their estimated standard deviations. Patterns in residuals versus fitted values or sample-level covariates can indicate a misspecified conditional mean or a variance that changes with the fitted value or covariate. Feature-wise residual distributions can reveal skewness, heavy tails, or extreme observations that make the Gaussian approximation questionable. Residual correlations across features or modalities can signal remaining dependence. These checks are evidence about model adequacy, not proofs: with few samples, a lack of visible pattern may simply reflect low power.

The principal alternatives address different failures. A feature-specific diagonal variance allows different noise levels but still assumes residual independence. A shared variance is more parsimonious but imposes equal noise scales. A residual covariance model can represent dependence between features, at the cost of additional parameters and the need for enough observations to estimate them. If residuals are markedly skewed or heavy-tailed, a non-Gaussian error model may be more appropriate. These alternatives change the observation assumptions; they do not automatically correct an incorrect conditional mean.

Model Choice in Multi-Omics Analysis

A continuous Gaussian observation model is useful when measurements, after any scientifically justified preprocessing, can reasonably be treated as continuous with approximately symmetric residual deviations around a linear latent predictor. Its main appeal is a transparent separation between modeled signal and residual noise, together with a tractable likelihood. Its limitations are equally important: Gaussianity can be sensitive to outliers, constant variance can fail across a wide measurement range, and conditional independence can fail even when the mean structure looks plausible.

Diagnostics should be interpreted in context. For example, a variance pattern across features might reflect different measurement precision rather than a failure of the mean predictor. Residual association may indicate omitted sample structure, or it may motivate a residual dependence model. Preprocessing can improve the approximation, but transformations alter the scale and meaning of the measurements. The appropriate model is therefore determined by the measurement process and the residual evidence, not by the fact that the observations are numeric.