A Generative Model For Multi-Omics Factor Analysis

Why Specify a Generative Model?

A multi-omics factor analysis represents several measurements on the same samples using latent score vectors and feature-specific loadings. A generative model makes this representation precise: it states how the latent quantities and residual noise produce each observed view. That matters because a decomposition is not just a convenient way to summarize a matrix. Its assumptions determine which patterns can be attributed to shared sample variation, which are view-specific, and what the residuals are expected to contain.

Assume there are $n$ aligned samples and $M$ data views. View $m$ measures $p_m$ features on those samples. Let $K$ be the number of shared latent factors, and let $L_m$ be the number of view-specific factors for view $m$. The model below is a linear latent-factor model. It describes the conditional mean structure without requiring a particular continuous, count, or binary observation distribution.

Definition: For view $m$, let $Y_m$ be the $n \times p_m$ matrix of observed measurements, with samples in rows and features in columns. Let $Z$ be the $n \times K$ matrix of shared sample scores, $U_m$ the $n \times L_m$ matrix of view-specific sample scores, $W_m$ the $p_m \times K$ shared loading matrix, and $V_m$ the $p_m \times L_m$ view-specific loading matrix. Let $\mu_m$ be a $p_m$-vector of feature means and $E_m$ an $n \times p_m$ residual matrix. The generative equation is $$ Y_m = \mathbf{1}_n\mu_m^T + ZW_m^T + U_mV_m^T + E_m,\qquad m=1,\ldots,M. $$

The dimensions make each term an $n \times p_m$ matrix: $\mathbf{1}_n\mu_m^T$ is the view’s intercept matrix, $ZW_m^T$ is its shared-factor contribution, and $U_mV_m^T$ is its view-specific contribution. The same $Z$ appears in every view, while each view has its own $W_m$, $U_m$, and $V_m$. A view-specific score in $U_m$ therefore contributes only to view $m$ in this model.

Generative Assumptions and Conditional Meaning

The equation specifies a mean structure; the assumptions specify how its components relate. A basic version assumes that the samples are independent draws from the modeled population, or are independent conditional on any covariates already included in the analysis. Shared scores have mean zero, and each view-specific score has mean zero. Residuals have mean zero and finite second moments. These centering conditions assign constant feature means to $\mu_m$, rather than allowing a nonzero average score to be exchanged with the intercept.

For each sample, the shared score $z_i$ is common across views. The view-specific scores $u_{im}$ are independent across views and independent of $z_i$. Residuals are independent of all scores and, in the basic model, independent across samples and views. Within a view, residual features need not be mutually independent unless that is separately assumed; their covariance can be represented by a feature covariance matrix. These are modeling assumptions, not consequences of writing a matrix factorization. In particular, correlated residuals across views or samples would violate this basic version.

No Gaussian distribution is required for the linear mean equation itself. To make a fully specified likelihood, one must additionally choose probability distributions for the scores and residuals. Those choices are separate from the structural statement that the conditional mean is linear in the latent scores. When the residual distribution has a density, the conditional independence assumption implies the following factorization for a sample:

p(y_{i1},\ldots,y_{iM}\mid z_i,u_{i1},\ldots,u_{iM}) = \prod_{m=1}^{M}p_m(y_{im}\mid z_i,u_{im}).

Here $y_{im}$ is the row vector of measurements for sample $i$ in view $m$. The factorization says that, once the latent scores are given, the views’ residual variation is independent in this model. It does not say the observed views are marginally independent: they share $z_i$.

Conditional mean: If the residuals have conditional mean zero, then $$ E(Y_m\mid Z,U_m)=\mathbf{1}_n\mu_m^T+ZW_m^T+U_mV_m^T. $$ Thus the factor terms describe the model’s systematic signal, and $E_m$ is the difference between the observed matrix and that signal.

For a single sample, use row vectors throughout: $y_{im}$ has length $p_m$, $z_i$ has length $K$, and $u_{im}$ has length $L_m$. The dimensionally consistent equation is

y_{im}=\mu_m^T+z_iW_m^T+u_{im}V_m^T+e_{im}.

This follows entry by entry. For feature $j$ of view $m$, the model states

y_{imj} = \mu_{mj} + \sum_{k=1}^{K} z_{ik}w_{mjk} + \sum_{\ell=1}^{L_m}u_{im\ell}v_{mj\ell} + e_{imj}.

Worked Examples

Worked Example: Checking the Dimensions

Suppose two hypothetical omics views are measured on $n=12$ aligned samples. View 1 has $p_1=5$ features and view 2 has $p_2=3$. There are $K=2$ shared factors, with $L_1=1$ and $L_2=3$ view-specific factors. Determine the dimensions of every term in each matrix equation.

For view 1, $Y_1$ is $12 \times 5$, $Z$ is $12 \times 2$, $W_1$ is $5 \times 2$, $U_1$ is $12 \times 1$, and $V_1$ is $5 \times 1$. Therefore $ZW_1^T$ is $(12 \times 2)(2 \times 5)=12 \times 5$, and $U_1V_1^T$ is $(12 \times 1)(1 \times 5)=12 \times 5$. The intercept and residual matrices are also $12 \times 5$.

For view 2, $W_2$ is $3 \times 2$, $U_2$ is $12 \times 3$, and $V_2$ is $3 \times 3$. Then $ZW_2^T$ is $(12 \times 2)(2 \times 3)=12 \times 3$, and $U_2V_2^T$ is $(12 \times 3)(3 \times 3)=12 \times 3$. Both match $Y_2$. The same $Z$ can enter both equations because it has one row per aligned sample; the number of features is allowed to differ by view.

Worked Example: Generating Measurements for One Sample

Consider hypothetical measurements for one sample in two views, with one shared factor and one view-specific factor in each view. In view 1, let $\mu_1=(10,20)^T$, $W_1=(3,-1)^T$, $V_1=(2,4)^T$, shared score $z_i=2$, view-specific score $u_{i1}=-1$, and residual row $e_{i1}=(1,-2)$. Calculate the resulting row of observations.

The row-vector equation gives $$ y_{i1}=(10,20)+2(3,-1)+(-1)(2,4)+(1,-2). $$ The shared contribution is $(6,-2)$, and the view-specific contribution is $(-2,-4)$. Adding componentwise gives $$ y_{i1}=(10+6-2+1,\;20-2-4-2)=(15,12). $$ The first observed feature is 15 units and the second is 12 units, in their respective measurement units. The calculation uses a feature-mean column vector only through its transpose in the row-vector equation.

For view 2, suppose there is one feature, $\mu_2=5$, $W_2=1.5$, $V_2=-2$, $u_{i2}=0$, and $e_{i2}=-0.5$, while the shared score remains $z_i=2$. Then $y_{i2}=5+2(1.5)+0(-2)-0.5=7.5$. The shared score is reused, but the view-specific score and loadings are those of view 2.

Worked Example: Separating Fitted Signal From Residual

For a hypothetical sample in a two-feature view, let $\mu_m=(1,0)^T$, $W_m=(2,1)^T$, $V_m=(1,0.5)^T$, $z_i=1$, and $u_{im}=-1$. Suppose the observed row is $y_{im}=(4,1)$. Find the fitted conditional mean and residual.

The fitted conditional mean is $$ \widehat{y}_{im}=(1,0)+1(2,1)+(-1)(1,0.5)=(2,0.5). $$ Subtracting this row from the observation gives $$ e_{im}=y_{im}-\widehat{y}_{im}=(4,1)-(2,0.5)=(2,0.5). $$ The residual is therefore 2 units for feature 1 and 0.5 units for feature 2. It is the part not represented by the specified intercept and factor contributions. A single residual vector does not establish that the model’s zero-mean or independence assumptions fail; those are population assumptions assessed using patterns across samples and views.

Justification and Model Checks

The conditional-mean result follows directly from the generative equation. Conditional on $Z$ and $U_m$, the intercept and loading terms are fixed. Taking conditional expectations of both sides and using $E(E_m\mid Z,U_m)=0$ gives

\begin{aligned} E(Y_m\mid Z,U_m) &=E(\mathbf{1}_n\mu_m^T+ZW_m^T+U_mV_m^T+E_m\mid Z,U_m)\\ &=\mathbf{1}_n\mu_m^T+ZW_m^T+U_mV_m^T. \end{aligned}

The justification depends on the zero conditional mean of the residuals, not on Gaussian noise. If residuals systematically depend on a score, then this expectation need not hold and the factor terms no longer have the stated conditional interpretation.

A useful diagnostic begins with residual matrices computed as $Y_m-\mathbf{1}_n\mu_m^T-ZW_m^T-U_mV_m^T$. Check whether residual means or patterns remain across samples, features, or views; whether a few observations dominate; and whether residuals exhibit dependence that the model assumes away. Also check alignment: row $i$ must refer to the same sample in every view. Misaligned rows can create an apparent lack of shared structure or a misleading shared pattern.

Interpret diagnostics with care. Estimated latent quantities are not known population values, and finite-sample residuals need not average exactly to zero. A pattern in residuals is a warning to investigate the model, not an automatic proof of one particular failure. Batch effects, unmodeled covariates, nonlinear relationships, or an inadequate number of factors can all leave structure behind. Strong residual dependence across views suggests that the assumption of conditionally independent view noise may be unsuitable.

Alternatives, Limitations, and a Common Mistake

Approach	What it represents	Key distinction from this model
Separate factor models by view	Latent structure within each view	Does not, by itself, require a common score matrix across views.
Factor analysis on concatenated features	Factors for one combined feature matrix	Can represent joint structure, but does not explicitly assign each factor contribution to shared versus view-specific scores.
Joint non-negative matrix factorization	Non-negative component representations, with coupling determined by the chosen objective	Uses non-negativity and an optimization objective rather than this model’s centered latent-score and residual formulation.

The shared-and-specific construction is useful when the scientific question distinguishes cross-view sample patterns from variation confined to one view. Its limitations are equally important. It is linear in the scores; it depends on the chosen number of factors; and its assumptions about score and residual dependence may be unrealistic. The equation alone does not guarantee that the factors are uniquely determined, nor that a fitted factor corresponds to a biological mechanism. Different parameter values can sometimes represent the same fitted signal.

Common mistake: “The same $Z$ appears in every view, so the observed views must be independent once they are centered.” This is incorrect: centering only removes the modeled intercept. The views share latent scores, and the factor contributions can connect their sample patterns. The model’s conditional independence statement applies only after conditioning on the shared and view-specific scores, under the residual-independence assumptions.

A second frequent error is to treat $U_m$ as shared because it has the same number of rows as $Z$. Its rows index the same samples, but $U_m$ is specific to view $m$, and it has no role in the equation for another view. Finally, dimensions must be checked before interpreting a product: with samples in rows, $ZW_m^T$ is the shared contribution, not $W_mZ^T$.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

If $Y_m$ is $20 \times 8$, $Z$ is $20 \times 3$, and $W_m$ is $8 \times 3$, what is the dimension of $ZW_m^T$?
State the conditional mean of $Y_m$ under the model and the assumption needed for that result.
In the one-sample row equation, why is the intercept written as $\mu_m^T$?
Does conditional independence of views imply that their observed measurements are marginally independent? Explain.
Name one residual pattern that would suggest investigating a model assumption, and explain why it is a warning rather than conclusive proof of failure.

Solutions. (1) $W_m^T$ is $3 \times 8$, so $ZW_m^T$ is $(20 \times 3)(3 \times 8)=20 \times 8$. (2) $E(Y_m\mid Z,U_m)=\mathbf{1}_n\mu_m^T+ZW_m^T+U_mV_m^T$; this requires the residual matrix to have conditional mean zero given the scores. (3) $\mu_m$ is a $p_m$-dimensional column vector, while $y_{im}$ is a row vector, so its mean contribution in that equation is $\mu_m^T$. (4) No. The views share $Z$, so their observations can be dependent marginally even when their residuals are independent conditional on the latent scores. (5) For example, residuals that remain correlated across views suggest checking the assumed residual independence. It is a warning because estimated scores, finite-sample variation, or other model misspecification can also produce apparent patterns.