Why Identifiability Matters
A fitted factor model can reproduce the observed omics measurements well without uniquely determining what each latent factor represents. One decomposition may describe a pattern as shared across views; another may describe the same pattern as specific to one view. The distinction matters when factors are interpreted as cross-omics biology rather than merely as a compact representation of the data.
Identifiability asks whether different parameter values or latent structures imply different distributions for the observations. If two distinct structures imply exactly the same distribution, no amount of additional data from that same model and design can distinguish them. If they imply different distributions but are very similar, the structure may be identifiable in principle yet difficult to estimate in a finite sample.
The earlier material on rotational invariance and scale and sign constraints establishes that factor coordinates can change without changing the fitted observations. Scale and sign conventions can remove some of these ambiguities, but they do not guarantee that shared and specific structure is identifiable. The main questions here are whether the data contain cross-view information, whether the relevant loading matrices have adequate rank, and whether the model assumptions rule out alternative decompositions.
A Model and the Assumptions Behind Cross-View Information
Let \(x_v\) be the centered feature vector for view \(v\), with \(p_v\) features. A linear shared-and-specific model for \(V\) views is
Here \(f\) is a vector of \(k_s\) shared scores, \(g_v\) is a vector of \(k_v\) scores specific to view \(v\), \(A_v\) and \(C_v\) are their loading matrices, and \(\epsilon_v\) is residual noise. A shared score is the same latent random vector across views; a view-specific score contributes to only its own view. This is a population model: statements about covariance below refer to the covariance of the underlying distribution, not a sample covariance estimate.
For cross-view covariance to isolate shared structure, assume that the scores have mean zero, that \(f\) is uncorrelated with every \(g_v\), that distinct view-specific scores are uncorrelated across views, and that residuals are uncorrelated with the scores and with residuals in other views. Write \(\Sigma_f=\operatorname{Cov}(f)\), and assume it is positive definite when discussing all \(k_s\) shared directions. These covariance assumptions are enough for the following covariance identity:
This identity is not an independence claim. In particular, zero covariance does not imply independence in general. If a claim concerns the complete observed distribution—for example, that two alternative Gaussian component models imply the same distribution—assume the relevant components are jointly Gaussian or state their independence directly. Under joint Gaussianity, uncorrelated components are independent; Gaussian marginal distributions alone do not establish that fact.
What Cross-View Rank Can Identify
Set \(M_{vw}=\operatorname{Cov}(x_v,x_w)\). Since \(M_{vw}=A_v\Sigma_f A_w^T\), its rank cannot exceed \(k_s\). Under stronger conditions it equals \(k_s\). Specifically, suppose \(k_s\leq \min(p_v,p_w)\), both \(A_v\) and \(A_w\) have full column rank \(k_s\), and \(\Sigma_f\) is positive definite. Then \(M_{vw}\) has rank \(k_s\), and its column space equals the column space of \(A_v\).
The justification is direct. The matrix \(\Sigma_f A_w^T\) has \(k_s\) rows and full row rank because \(\Sigma_f\) is invertible and \(A_w\) has full column rank. Consequently, multiplying \(A_v\) by \(\Sigma_f A_w^T\) preserves the \(k_s\)-dimensional column space of \(A_v\):
Thus, in this idealized case, cross-view covariance identifies the shared loading subspace in each view and the number of shared covariance directions. It does not uniquely identify the columns of \(A_v\), the latent scores \(f\), or their labels. In particular, invertible coordinate changes of the shared factors can alter the loading columns while preserving the same model. If the factor covariance is fixed to the identity, orthogonal rotations still preserve the model.
When a loading matrix is rank deficient, when \(\Sigma_f\) is singular, or when a shared direction has zero loading in one of the views, the rank of \(M_{vw}\) can be smaller than the nominal number of shared factors. The data then reveal fewer cross-view directions than the model names. A low-rank cross-covariance is therefore evidence about the dimension of shared covariance, not automatic proof that every specified shared factor is separately identified.
Worked Example: Reading the Shared Subspace From Cross-View Covariance
Consider two hypothetical continuous views. Suppose their only cross-view dependence comes from one shared score \(f\), with \(\operatorname{Var}(f)=2\), and their loading vectors are \(A_1=(1,2)^T\) and \(A_2=(3,-1)^T\). Assume all view-specific terms and residuals are uncorrelated across views. The cross-view covariance is
The second row is twice the first, so the matrix has rank 1. Its column space is the span of \((1,2)^T\), the shared loading direction in view 1. Its row space is the span of \((3,-1)\), the shared loading direction in view 2. This supports one cross-view covariance direction. It does not determine a unique scale for the score and loadings: multiplying \(f\) by a nonzero constant and inversely rescaling both loading vectors can preserve the observations and their covariance. Nor does the calculation establish that a single named biological mechanism generated the pattern.
When Distinct Shared-Factor Decompositions Are Equivalent
A nominal factor count need not equal the number of distinguishable directions. Suppose a model specifies two shared scores but their loading columns are proportional in every view. The observations then depend only on one linear combination of the two scores. The orthogonal combination has no distinct loading pattern to reveal it. This is a structural problem, not a matter of choosing a more accurate estimation algorithm.
Worked Example: Two Named Factors, One Observable Direction
In two hypothetical views, suppose \(f_1\) and \(f_2\) are independent \(N(0,1)\) scores. Let the loading matrices have columns \(a,2a\) in view 1 and \(b,2b\) in view 2. Assume the remaining terms are independent of these scores and have the same distribution in both candidate models. The shared contribution in view 1 is
Define \(t=f_1+2f_2\). Because \(f_1\) and \(f_2\) are independent Gaussian variables, \(t\sim N(0,5)\). View 2 has shared contribution \(b t\) as well. Therefore the two-factor model gives exactly the same joint Gaussian shared contribution to the observations as a one-factor model with score \(t\), loadings \(a\) and \(b\), and score variance 5. Both views observe the same \(t\); the second named direction does not create a second loading direction. The factor count of two is not identified by these loading patterns, even though the model's shared signal is nonzero.
A second ambiguity arises when a component described as shared is present in only one view. If its loading is zero in all other views, the observations cannot distinguish it from a view-specific component in the one view where it appears. The following example uses independent Gaussian components so that the equivalence is at the level of the full observed distribution, not just covariance.
Worked Example: A One-View “Shared” Component Is Not Distinguishable
Suppose two scalar views have the hypothetical model \(x_1=z+u_1+\epsilon_1\) and \(x_2=\epsilon_2\), where \(z\sim N(0,2)\), \(u_1\sim N(0,3)\), and \(\epsilon_1,\epsilon_2\sim N(0,1)\) are mutually independent. Label \(z\) as shared but give it loading 1 in view 1 and loading 0 in view 2. Since the components are independent Gaussian variables, the observed vector is jointly Gaussian, with
An alternative model with no \(z\), a view-specific score \(u_1^*\sim N(0,5)\), and the same independent residuals gives \(x_1=u_1^*+\epsilon_1\), \(x_2=\epsilon_2\). It has the same variances, zero covariance, and joint Gaussian distribution. Thus the two models imply exactly the same distribution for the observations. Calling the variance-5 contribution “shared but silent in view 2” rather than “specific to view 1” is not empirically distinguishable here. Requiring a shared direction to load nontrivially in at least two views rules out this particular boundary case by definition, but cannot resolve all rank deficiencies or other ambiguities.
Identifying View-Specific Structure and Checking Assumptions
Once shared covariance has been identified, it may be tempting to subtract it from each within-view covariance and call what remains “the specific factors.” The subtraction can identify a residual covariance matrix under the model assumptions, but usually does not uniquely split that matrix into specific-factor covariance and residual noise. For a view, the within-view covariance has the form
where \(\Psi_v=\operatorname{Cov}(\epsilon_v)\). Even if the shared term is known, the remaining covariance can admit different decompositions into \(C_v\Sigma_{g_v}C_v^T\) and \(\Psi_v\). Restrictions such as a specified residual covariance structure, constraints on loadings, or distributional assumptions can help, but must be stated and justified. An unrestricted residual covariance can absorb any positive semidefinite factor contribution, so a specific-factor interpretation is not identified from covariance alone in that case.
The cross-view argument also relies on the assumption that view-specific scores and residuals do not create cross-view covariance. If a technical effect or an omitted shared source induces correlated residuals across views, then \(M_{vw}\) contains that contribution too. Its rank and column spaces may no longer describe only the intended shared factors. Moreover, finite-sample cross-covariance estimates can have small nonzero singular values due to noise. Population rank conditions do not by themselves provide a reliable finite-sample threshold for declaring a direction present.
- Check loading rank. A factor with zero or linearly dependent loading columns cannot provide a distinct observable direction.
- Check cross-view support. For a claimed shared direction, examine whether it contributes to more than one view under the model, rather than relying only on its label.
- Check the covariance assumptions. Cross-view residual dependence can be mistaken for shared latent structure.
- Separate model equivalence from estimation instability. Exact equivalence is structural; noisy or near-collinear patterns can instead make estimates highly variable while remaining distinguishable in principle.
How Identifiability Relates to Common Constraints
Scale and sign constraints choose conventions for representing a factor; they do not create information absent from the observations. For example, setting factor variances to one can fix scale, and a sign convention can choose between a loading vector and its negative. But if two loading columns are proportional, fixing their signs does not make their two directions independently observable. Likewise, a rotation constraint may select one representation from an equivalent family without proving that the selected coordinates correspond to uniquely determined biological quantities.
| Question | Information or condition needed | What it does not guarantee |
|---|---|---|
| How many shared covariance directions are supported? | Rank of cross-view covariance under assumptions excluding other cross-view dependence | A unique set of factor coordinates or a biological interpretation |
| Is the shared loading subspace identified? | Full-rank loadings and nonsingular shared-score covariance for the views being compared | Unique loading columns within that subspace |
| Can a component be called view-specific? | Model restrictions and evidence that distinguish its contribution from shared and residual variation | Unique factor-versus-noise decomposition with unrestricted residual covariance |
| Are scale, sign, or rotation ambiguities removed? | Explicit normalization or coordinate conventions | Resolution of rank deficiency or shared-versus-specific equivalence |
The main alternative to a cross-view covariance argument is to impose stronger model structure, such as sparsity, known feature memberships, or a restricted residual model. Such assumptions may make a decomposition identifiable or easier to estimate, but the conclusion then depends on those restrictions. A predictive criterion can compare how well candidate models represent or predict data, yet predictive success alone cannot distinguish parameterizations that imply the same observed distribution. These approaches answer related but different questions.
Common Mistakes and Practical Limits
- Equating the fitted factor count with the identifiable factor count. A model may request several factors even when its loading matrices have lower rank. Check which directions alter the observed distribution.
- Treating every cross-view association as a shared factor. The covariance identity requires that specific terms and residuals do not generate cross-view covariance. Correlated technical effects can violate this interpretation.
- Claiming independence from zero covariance. Zero covariance is not generally independence. Use joint Gaussianity, or state independence directly, before deriving an independent-component distribution.
- Assuming normalization proves uniqueness. Scale and sign conventions fix representation choices, not missing cross-view information or a rank-deficient loading structure.
- Calling a weakly estimated factor non-identifiable. Near-collinearity can make a direction unstable in finite data without making two models exactly distributionally equivalent. Exact non-identifiability and practical uncertainty should be described separately.
Identifiability is therefore best reported at the level supported by the model and data: for example, “one shared covariance direction is supported, and its loading subspace is identifiable under the stated full-rank assumptions,” rather than “the latent biological factor is uniquely identified.” The latter requires additional assumptions and substantive evidence beyond a low-rank cross-view covariance pattern.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- If a two-view cross-covariance matrix has population rank 1, what does that establish about the nominal shared factor count?
- Why do full-column-rank loading matrices and a positive-definite shared-score covariance matter for identifying the shared loading subspace?
- A proposed shared score loads on only one view. What ambiguity does this create?
- Does zero covariance between a shared score and a view-specific score prove independence? What additional condition would justify independence in a Gaussian argument?
- Can fixing factor variances and signs resolve two proportional loading columns? Explain.
Solutions. (1) It establishes that the cross-view covariance contains at most one linearly independent shared covariance direction under the assumptions that exclude other cross-view dependence. The nominal factor count could be larger if some directions are unobservable or rank deficient. (2) They ensure that the product \(A_v\Sigma_f A_w^T\) has the full shared rank and that its column space equals \(\operatorname{col}(A_v)\); without them, directions can be lost in the product. (3) Its contribution can be represented as view-specific in the one view, so the shared and specific labels are observationally indistinguishable unless further restrictions apply. (4) No. Zero covariance alone does not imply independence; joint Gaussianity of the relevant components, or a direct independence assumption, justifies that conclusion. (5) No. Scale and sign conventions do not create a second observable direction; proportional columns still span only a one-dimensional loading subspace.