Latent Factors for Shared Molecular Variation

Why Represent Molecular Variation With Latent Factors?

When expression, protein abundance, and metabolite measurements are collected from the same individuals, some variation may be coordinated across layers. A biological process could influence several measured traits, while other variation is specific to one assay or molecular layer. A factor model summarizes these patterns using a small number of unobserved variables, or latent factors, rather than treating every measured trait as independent.

This is a statistical representation of covariance, not a guarantee that a factor corresponds to a distinct biological pathway. A factor can reflect biology, technical artifacts, or a mixture. Its interpretation depends on the model, the measurements, and checks for alternative explanations.

Definition: A shared latent factor is an unobserved variable whose loadings allow it to contribute to measurements in multiple molecular layers. A layer-specific latent factor contributes only to measurements in its own layer. A loading describes how strongly a factor contributes to a measured trait.

A Model for Shared and Layer-Specific Variation

Let \(i\) index individuals and \(l\) index molecular layers. Write \(x_{il}\) for the \(p_l\)-dimensional vector of centered measurements from layer \(l\) in individual \(i\). A useful model is

x_{il}=\Lambda_l f_i+\Gamma_l g_{il}+\varepsilon_{il}.

Here \(f_i\) is a \(q\)-dimensional vector of shared factors, with loading matrix \(\Lambda_l\) for layer \(l\). The vector \(g_{il}\) contains \(r_l\) factors specific to layer \(l\), with loading matrix \(\Gamma_l\). The residual vector \(\varepsilon_{il}\) contains variation not represented by either set of factors. The dimensions \(p_l\), \(q\), and \(r_l\) denote, respectively, the number of measured traits in layer \(l\), shared factors, and layer-specific factors.

To state the covariance interpretation precisely, suppose the factors and residuals have mean zero, and assume

\operatorname{Cov}(f_i)=I,\qquad \operatorname{Cov}(g_{il})=I,\qquad \operatorname{Cov}(f_i,g_{il})=0.

Also assume that factors are uncorrelated with residuals, that residuals from distinct layers have zero covariance, and, crucially, that layer-specific factors from different layers are uncorrelated:

\operatorname{Cov}(g_{il},g_{im})=0 \quad\text{for }l\ne m,\qquad \operatorname{Cov}(\varepsilon_{il},\varepsilon_{im})=0 \quad\text{for }l\ne m.

Under these assumptions, the cross-layer covariance is due to the shared factors. Without the assumption about distinct layer-specific factors, those factors could themselves induce cross-layer covariance, even if each has identity covariance within its own layer.

What the Covariance Reveals

For two distinct layers \(l\) and \(m\), the model implies

\operatorname{Cov}(x_{il},x_{im}) =\Lambda_l\Lambda_m^{T}.

Within one layer, the covariance has additional terms from its layer-specific factors and residuals. If the residual covariance within layer \(l\) is \(\Psi_l\), then

\operatorname{Cov}(x_{il}) =\Lambda_l\Lambda_l^{T}+\Gamma_l\Gamma_l^{T}+\Psi_l.

A common specification takes \(\Psi_l\) to be diagonal, but this is a modeling choice rather than a general requirement. The cross-layer result follows by expanding the covariance of the model and applying the assumptions. In particular, the cross-layer terms involving \(g_{il}\) and \(g_{im}\) vanish only because those factors are assumed uncorrelated across distinct layers.

Worked Example: Cross-Layer Covariance From One Shared Factor

Suppose two layers each contain two centered traits. There is one shared factor with variance one, and the loading vectors are \(\Lambda_1=(2,1)^T\) and \(\Lambda_2=(3,-1)^T\). Assume that all the cross-layer independence and zero-covariance conditions above hold. Calculate the cross-layer covariance matrix.

With one shared factor, the covariance is the outer product of the two loading vectors:

\operatorname{Cov}(x_{i1},x_{i2}) =\Lambda_1\Lambda_2^T = \begin{pmatrix}2\\1\end{pmatrix} \begin{pmatrix}3&-1\end{pmatrix} = \begin{pmatrix}6&-2\\3&-1\end{pmatrix}.

For example, the covariance between trait 1 in layer 1 and trait 2 in layer 2 is \(-2\) in the measurement units implied by the centered traits. The negative sign comes from the negative loading of trait 2 in layer 2. Layer-specific factors do not add to this matrix because they are uncorrelated across layers under the stated assumptions.

Relating Genetic Associations to the Factors

Let \(D_i\) be a centered genetic variant dosage, measured in allele copies, and suppose its association with the shared factors is represented by

f_i=aD_i+u_i,\qquad E(u_i\mid D_i)=0.

The vector \(a\) gives the change in each shared factor per additional allele. If the layer-specific factors and residuals have conditional means of zero given \(D_i\), the corresponding vector of marginal trait-association slopes in layer \(l\) is \(\Lambda_l a\). This follows by substituting the factor regression into the measurement model and taking the conditional expectation given dosage.

E(x_{il}\mid D_i)=\Lambda_l aD_i \quad\Longrightarrow\quad \frac{\partial E(x_{il}\mid D_i)}{\partial D_i}=\Lambda_l a.

Genetic effects need not act only through shared factors. To represent a layer-specific association as well, one can write \(g_{il}=c_lD_i+v_{il}\), with \(E(v_{il}\mid D_i)=0\). The expected trait-association slope then becomes \(\Lambda_l a+\Gamma_l c_l\). Thus a shared-factor association is one component of an association pattern, not a default explanation for every genetic effect across layers.

Worked Example: Translating a Factor Association Into Trait Slopes

Suppose a dosage has an estimated association of \(0.4\) factor units per allele with one shared factor. A protein trait has loading \(1.5\), and an expression trait has loading \(0.5\) on that factor. Assume no layer-specific genetic association and conditional mean-zero residuals.

The predicted protein slope is the loading multiplied by the factor association:

1.5(0.4)=0.60 \quad\text{protein units per allele}.

The predicted expression slope is

0.5(0.4)=0.20 \quad\text{expression units per allele}.

These are model-implied slopes, not evidence by themselves that the variant causally changes either trait. They describe how the estimated association with the shared factor translates through the fitted loadings, conditional on the model and its assumptions.

Estimation, Identification, and Interpretation

Factor models can be fitted using likelihood-based methods or approximations such as principal-component approaches, depending on the data structure and model. In either case, the number of factors and the form of the residual covariance need to be specified or selected. Choices should be assessed using diagnostics and scientific context; a factor that explains much covariance is not automatically biologically meaningful.

The factors are generally not uniquely identified in their original coordinates. For any orthogonal matrix \(Q\), replacing \(f_i\) by \(Q^Tf_i\) and \(\Lambda_l\) by \(\Lambda_lQ\) leaves the fitted shared component unchanged:

(\Lambda_lQ)(Q^Tf_i)=\Lambda_l f_i.

Consequently, the sign, order, and orientation of individual factors can change across equivalent fits. A particular factor’s genetic association and loadings should be interpreted relative to the chosen parameterization. The fitted shared subspace and the resulting modeled covariance are often more stable targets than the label or biological name assigned to one factor.

Justification: Under the stated model, expand \(\operatorname{Cov}(x_{il},x_{im})\) into the covariances among \(f_i\), \(g_{il}\), \(g_{im}\), and the residuals. The shared-factor term is \(\Lambda_l\operatorname{Cov}(f_i)\Lambda_m^T=\Lambda_l\Lambda_m^T\). Cross terms involving different components vanish by the uncorrelatedness assumptions. The layer-specific term \(\Gamma_l\operatorname{Cov}(g_{il},g_{im})\Gamma_m^T\) vanishes only when the distinct layers’ specific factors are uncorrelated; the residual cross-layer term vanishes only when residuals from distinct layers have zero covariance.

Worked Example: A Correlated Layer-Specific Factor Breaks the Interpretation

Consider one measured trait in each of two layers. Suppose there is no shared factor, and each layer has a layer-specific factor with loading \(1\) and variance \(1\). If the two specific factors are identical across individuals, their covariance is \(1\). Assume residuals are uncorrelated across layers.

The cross-layer covariance contributed by the specific factors is

1\cdot \operatorname{Cov}(g_{i1},g_{i2})\cdot 1 =1\cdot1\cdot1=1.

Thus the observed cross-layer covariance is \(1\), even though no shared factor is present. Identity covariance within each layer and zero covariance with an absent shared factor do not prevent this result. The covariance can be attributed to shared factors only if the model also rules out correlation between layer-specific factors from distinct layers, or explicitly models that correlation.

When Technical Variation Looks Shared

A batch, plate, processing date, or other technical variable can affect measurements in several layers. If it is omitted, a factor model may represent that coordinated technical variation as a shared factor. The result may look biologically coherent simply because the technical variable influences many assays. Moreover, if genotype dosage is associated with batch—for example, through recruitment, ancestry, or sample handling patterns—a technical effect can create an apparent genetic association.

Worked Example: A Batch-Driven Genetic Association

Suppose a centered assay measurement follows \(X_i=2T_i+\varepsilon_i\), where \(T_i\) is a centered technical batch indicator, the batch effect is \(2\) assay units, and \(\operatorname{Cov}(D_i,\varepsilon_i)=0\). In a hypothetical sample, let \(\operatorname{Var}(D_i)=0.5\) dosage-squared and \(\operatorname{Cov}(D_i,T_i)=0.1\) dosage-times-batch units. There is no biological effect of dosage in this model.

The unadjusted regression slope of \(X_i\) on \(D_i\) is

\frac{\operatorname{Cov}(D_i,X_i)}{\operatorname{Var}(D_i)} = \frac{\operatorname{Cov}(D_i,2T_i+\varepsilon_i)}{0.5} = \frac{2(0.1)+0}{0.5} =0.4 \quad\text{assay units per allele}.

The apparent association of \(0.4\) units per allele arises because dosage and batch covary, not because dosage has a biological effect in the specified model. If the batch indicator is measured and the model is correctly adjusted for it, the batch contribution is separated from the dosage association provided \(E(\varepsilon_i\mid D_i,T_i)=0\). If batch is unknown or poorly measured, the source of the association may remain ambiguous.

Diagnostics and Common Pitfalls

Before interpreting a factor as shared biology, examine whether its scores or loadings align with known technical variables, sample quality, collection site, or processing order. Check whether the genetic association persists after appropriate technical adjustment and whether it is robust to reasonable factor-model choices. Adjustment itself needs care: a technical covariate can be entangled with a genuine biological signal, so blindly removing variation may remove part of the target signal as well.

Pitfall: assuming cross-layer covariance proves shared biology. Cross-layer covariance can also arise from correlated layer-specific factors, residual dependence, or a shared technical influence. State and assess the covariance assumptions.
Pitfall: naming a factor from its largest loading. Loadings describe covariance patterns under a parameterization; they do not establish a biological mechanism.
Pitfall: treating an associated factor as a causal mediator. A genetic association with a latent factor is an association with a model-based summary. It does not alone establish a causal path to measured traits.
Pitfall: treating equivalent factor rotations as different scientific findings. Signs and orientations can vary while the fitted shared component remains the same. Compare invariant predictions or subspaces where possible.

Key takeaway: A shared-factor model separates covariance into modeled components, but its interpretation depends on explicit assumptions. In particular, layer-specific factors from distinct layers must be uncorrelated for cross-layer covariance to be attributed solely to shared factors. Genetic associations and biological interpretations also require checks for technical structure.

Check Your Understanding

Answer each question by referring to the model assumptions and the units in context.

In the factor model, what additional assumption about \(g_{il}\) and \(g_{im}\) is needed to attribute cross-layer covariance solely to shared factors?
If a shared factor has a genetic association of \(0.3\) factor units per allele and a trait loading of \(-2\), what trait-association slope does the model imply, and in what units should it be reported?
Why can two models with different signs or orientations for their factors yield the same fitted molecular measurements?
Describe how a batch effect could produce an apparent association between genotype dosage and a molecular trait even if dosage has no biological effect in the model.
Why does an association with a shared latent factor not, by itself, establish that a variant causes a biological pathway?