Worked Example Separating Shared And Private Factors

Why Separate Shared and View-Specific Factors?

Two omics views measured on the same subjects can contain variation that moves together across views, as well as variation particular to one view. A factor model makes this distinction explicit: shared factors have scores common to the views, while private factors have view-specific scores. The distinction is useful, but it is not automatic. A factor should be called shared only relative to a model and its assumptions, and a fitted decomposition need not reproduce the generating structure exactly.

We work with two centered data matrices, \(X^{(1)}\) and \(X^{(2)}\), with the same \(n\) subjects in the same row order. The matrix \(X^{(v)}\) has \(p_v\) features for view \(v\). Centering means that each feature has mean zero across these subjects; in applications, covariate adjustment may also be needed, as in the earlier material on covariate-adjusted multi-omics factor models.

The model alone does not uniquely name or scale its factors. For example, replacing \(Z\) by \(ZQ\) and \(A_v\) by \(A_vQ^{-\mathsf{T}}\), for an invertible matrix \(Q\), leaves the shared fitted values unchanged. Sign changes and permutations are also possible. Thus, comparisons with a known generating structure require a fixed convention for score scaling and factor order, or a suitable alignment of the estimated solution. The worked example fixes both by using known scores and a specified order.

For the calculations below, use the empirical cross-product \(n^{-1}M^\mathsf{T}N\). Assume the scores are centered and have unit empirical variance, so \(Z^\mathsf{T}Z/n=I_r\) and \(U^{(v)\mathsf{T}}U^{(v)}/n=I_{q_v}\). For a clean separation within each view, also assume \(Z^\mathsf{T}U^{(v)}/n=0\), and that each residual is empirically uncorrelated with that view’s scores. These are transparent conditions for the example, not guarantees supplied by fitting a factor model.

A Simulated Dataset With Known Structure

Consider four hypothetical subjects and one shared factor. Use one private factor for each view. The score vectors are \(z=(1,1,-1,-1)^\mathsf{T}\), \(u^{(1)}=(1,-1,1,-1)^\mathsf{T}\), and \(u^{(2)}=(1,-1,-1,1)^\mathsf{T}\). Each is centered and has squared length four, so its empirical variance under the \(1/n\) convention is one. Their pairwise cross-products are zero. Generate two features in each view, without residual noise:

The row-vector notation in these expressions means that each score vector is multiplied by the displayed row of feature coefficients. Equivalently, the generating loadings are \(A_1=(2,1)^\mathsf{T}\), \(B_1=(1,-2)^\mathsf{T}\), \(A_2=(1.5,-1)^\mathsf{T}\), and \(B_2=(2,1)^\mathsf{T}\). The resulting data are:

What the Cross-View Product Does—and Does Not—Measure

A shared score can induce cross-view covariance because it contributes to both views. But taking a cross-product of the observed matrices does not, by itself, isolate that contribution. To see why, substitute the factor models into \(X^{(v)\mathsf{T}}X^{(w)}/n\), for \(v\ne w\). The complete expansion is

Consequently, the exact finite-sample equality \(X^{(v)\mathsf{T}}X^{(w)}/n=A_vA_w^\mathsf{T}\) requires more than zero cross-view residual cross-product. Under the unit score normalization, it holds if the cross-view private-score products \(U^{(v)\mathsf{T}}U^{(w)}/n\) and the cross-products between scores and residuals in both views are zero, as well as \(E^{(v)\mathsf{T}}E^{(w)}/n=0\). The score-residual conditions include \(Z^\mathsf{T}E^{(w)}/n\), \(U^{(v)\mathsf{T}}E^{(w)}/n\), \(E^{(v)\mathsf{T}}Z/n\), and \(E^{(v)\mathsf{T}}U^{(w)}/n\), along with the score cross-products shown in the expansion. Omitting any nonzero term can give an incorrect decomposition.

For a population cross-covariance result, use the same expansion with expectations in place of empirical cross-products. If residuals are uncorrelated with latent scores in both views, private scores are uncorrelated across views, and residuals have zero cross-view covariance, then only the shared term remains. Under unit score covariance, the population cross-covariance is \(A_vA_w^\mathsf{T}\). In the finite simulated dataset above, all score vectors are mutually orthogonal and the residuals are zero, so the exact equality applies.

Comparing Shared and Private Signal Contributions

A simple measure of the fitted contribution of a factor set to one view is its squared Frobenius norm divided by \(n\). Under the stated unit score scaling and orthogonality, the shared contribution in view \(v\) is \(\lVert ZA_v^\mathsf{T}\rVert_F^2/n=\lVert A_v\rVert_F^2\), and the private contribution is \(\lVert U^{(v)}B_v^\mathsf{T}\rVert_F^2/n=\lVert B_v\rVert_F^2\). This follows by expanding the squared norm and using \(Z^\mathsf{T}Z/n=I\) or \(U^{(v)\mathsf{T}}U^{(v)}/n=I\). The total fitted signal is the sum of these contributions only if the shared and private fitted components are orthogonal within the view. Otherwise their cross-term must also be included.

Diagnostics, Alternatives, and Limitations

A useful diagnostic is to inspect whether the fitted shared scores align across views and whether each view retains systematic structure in its residuals. Also check score cross-products, residual-score cross-products, residual cross-view covariance, and within-view shared-private cross-products if using an orthogonal decomposition. Large remaining structure in residuals, nearly collinear score directions, or a decomposition that changes markedly under resampling are warnings that the factors are not cleanly separated. The earlier material on factor stability and residual diagnostics develops these checks in broader settings.

The least-squares calculation above is appropriate because the scores are known and the model is linear. In practical applications, scores are usually unknown and must be estimated jointly with loadings; different algorithms or constraints can produce different but observationally similar solutions. An alternative is to analyze each view separately, which can describe view-specific structure but does not directly impose shared scores. Another is a joint model that constrains some scores or loadings to be common; it can improve integration but risks forcing unrelated patterns together. No option removes the need to examine assumptions and sensitivity.

The main pitfall is to see a nonzero cross-view product and conclude that it is entirely due to shared factors. That reasoning silently discards private-score and residual terms in the expansion. The correction is to state the cross-view orthogonality and residual-uncorrelation assumptions explicitly, or to report the cross-view product as a mixture of contributions rather than as a pure shared signal. Likewise, a small residual cross-product alone does not establish the needed score-residual conditions.