Confounding Between Latent Factors And Covariates

Why Covariates Can Resemble Latent Factors

A latent factor summarizes coordinated variation across measurements. But coordination need not arise from an unmeasured biological process: it may track a measured covariate, such as age, treatment group, collection site, or assay batch. A factor that aligns with a covariate can still describe real variation in the data. The interpretive question is whether it captures variation associated with that covariate, whether additional structure remains beyond it, or both.

This distinction matters in multi-omics data because the same covariate may affect many features in one or more views. If the resulting pattern is strong and coordinated, a factor model may represent it with a score across samples and a loading pattern across features. That representation alone does not establish that the factor is a distinct biological process, a technical artifact, or a causal effect.

These are descriptive quantities from an ordinary least-squares regression of \(f\) on \(c\) with an intercept. They require no normality assumption for their algebraic calculation. If they are used for formal inference, the sampling design and uncertainty introduced by estimating factor scores must also be addressed; treating estimated scores as fixed observations can understate uncertainty. Centering makes the intercept disappear from the formulas. The slope’s units are factor-score units per covariate unit, while \(R^2\) is dimensionless.

A factor’s sign and scale can be changed without changing the underlying factor representation. Consequently, the sign of \(\hat{\beta}\) depends on the chosen score sign. The absolute correlation, or \(R^2\), does not change under a sign reversal or nonzero rescaling of the score.

Deriving a Score–Covariate Diagnostic

Suppose the factor score is described by a linear trend with the covariate plus a remainder. The least-squares slope is the value that minimizes the squared residuals:

To see why, expand the quantity being minimized as \(f^\mathsf{T}f-2b c^\mathsf{T}f+b^2c^\mathsf{T}c\). Differentiating with respect to \(b\) gives \(-2c^\mathsf{T}f+2b c^\mathsf{T}c\); setting this equal to zero yields the stated slope. The fitted score is \(\hat{\beta}c\), and the residual is \(r=f-\hat{\beta}c\). By construction, \(c^\mathsf{T}r=0\), so the fitted and residual vectors are orthogonal. Therefore:

This decomposition gives a precise descriptive meaning to \(R^2\): the proportion of the centered score’s sum of squares accounted for by its linear projection onto the covariate. A large value indicates strong linear alignment, not proof that the covariate caused the factor or that the factor has no other interpretation.

Alignment Can Appear in the Loading Pattern

A common factor-model representation of a centered \(n\)-by-\(p\) data matrix \(X\) is \(X\approx FL^\mathsf{T}+E\), where rows correspond to samples, columns to measured features, \(F\) contains sample scores, \(L\) contains feature loadings, and \(E\) contains residual variation. Consider a single centered covariate \(c\). If its association with features has a linear pattern \(\gamma\), then a component of the data may be written \(c\gamma^\mathsf{T}\). This component is rank one (if both vectors are nonzero), so a one-factor representation can capture it exactly by taking its score proportional to \(c\) and its loading proportional to \(\gamma\).

This is a geometric statement about the representation, not a claim that every covariate effect is exactly rank one. Real data can include nonlinear effects, interactions, noise, multiple covariates, and biological variation. Still, a covariate-associated pattern that is shared across many features can readily resemble a latent factor.

Diagnosing What Remains Beyond the Covariate

A useful diagnostic separates the score’s covariate projection from its residual, as in the derivation above. If the residual score \(r=f-\hat{\beta}c\) is negligible, then the observed score adds little sample-wise variation beyond the linear covariate trend. If \(r\) remains substantial and organized, the factor may also capture variation not explained by that trend. This does not establish that the remainder is a separate biological mechanism: unmeasured covariates, nonlinear relationships, measurement error, and model misspecification can all contribute.

Feature-level checks help connect score alignment to the data pattern. For each feature, inspect its association with the measured covariate and compare the resulting pattern across features with the factor’s loadings. If the covariate-associated feature pattern closely resembles the loading vector, that supports the interpretation that the factor captures covariate-associated variation. A mismatch suggests that the score–covariate association may be only partial, or that multiple sources of variation contribute to the factor. These comparisons are descriptive; feature-wise testing also raises multiplicity and uncertainty issues.

For several factors, one score at a time can be misleading. Factors may be correlated, and permissible rotations can redistribute associations among their scores and loadings. In that setting, inspect the association of the covariate with the span of the scores, as well as the individual score associations under the chosen factor solution. A conclusion that depends heavily on one arbitrary orientation is less robust than one that holds for the factor subspace. Factor sign and scale conventions do not, by themselves, resolve this issue.

Assumptions, Alternatives, and Failure Modes

The score slope and \(R^2\) are well-defined descriptive summaries when the centered covariate and score are nonzero and observed on the same samples. Their linear interpretation is limited: a nonlinear relationship can be strong even when \(R^2\) is modest, and a few influential samples can dominate a slope. Plots and group-wise summaries can reveal patterns hidden by a single number. If sample membership or missingness differs across features, check that comparisons use an appropriate common set of samples.

Several approaches answer related but different questions. Direct feature-wise regression on a covariate asks how individual measurements vary with it; comparing those associations with loadings asks whether the factor pattern resembles that feature-level pattern. Stratified plots can show whether a factor varies within covariate groups or mainly between them, though small groups limit what can be learned. A covariate-adjusted factor model incorporates covariates in the model itself and is a different modeling approach. These approaches are not interchangeable: each makes distinct choices about which variation to represent and how to handle uncertainty.

A key limitation is lack of independent variation. If a biological condition always occurs in one batch and never in another, their effects cannot be separated from the observed data alone. Likewise, a factor’s association with a measured covariate does not establish that the covariate caused the score pattern. The covariate might be a proxy for another variable, or both might reflect a shared cause. These questions require study-design knowledge as well as statistical summaries.

• Confusing alignment with nuisance. A strong correlation does not make a factor “only technical.” A measured covariate may describe meaningful biology.
• Calling the residual independent biology. Variation orthogonal to one measured covariate may still reflect unmeasured or nonlinear effects.
• Ignoring score orientation. A negative slope can become positive if the factor score sign is reversed; the absolute alignment is unchanged.
• Relying only on a small p-value. A test can indicate evidence of association without showing how much score variation is associated or whether the pattern is scientifically important.
• Overinterpreting a single factor in a multi-factor solution. Correlation or rotation among factors can make individual score associations depend on the chosen representation.