Why Scale and Sign Need Conventions
A latent factor is not observed directly. Its interpretation comes from how its sample scores vary and how its feature loadings describe that variation across omics measurements. Yet a fitted factor can often be represented by different numerical scores and loadings while giving exactly the same fitted data. In particular, multiplying all scores for a factor by a nonzero constant and dividing its loadings by that constant leaves their products unchanged. Reversing both signs also leaves the products unchanged.
These symmetries do not mean that fitted reconstructions are arbitrary: the reconstructed signal can be identical. They do mean that the numerical scale and orientation of a factor are not automatically comparable across fits. A clear normalization and sign convention gives scores and loadings a consistent representation. The convention must match the model: preserving a reconstruction is not always the same as preserving a specified probability distribution for the latent scores.
Model and Invariance
Let \(X^{(v)}\) be a centered data matrix for omics view \(v\), with \(n\) samples and \(p_v\) features. For a model with \(K\) shared factors, write its fitted signal as
Here \(Z\) is the \(n \times K\) matrix of sample scores, \(B_v\) is the \(p_v \times K\) loading matrix for view \(v\), and \(E_v\) is the residual matrix. The \(k\)th column of \(Z\) contains scores for factor \(k\); the corresponding \(k\)th column of each \(B_v\) contains the feature loadings for that factor in view \(v\). The fitted signal is \(ZB_v^{\mathsf T}\). This equation describes the reconstruction; a probabilistic model may additionally specify distributions for scores and residuals.
The transformation must be coordinated across views: the same score matrix \(Z\) is used in each view, so the corresponding column of every \(B_v\) must receive the reciprocal scaling. Changing only one view’s loading would generally change that view’s fitted signal. The transformation also assumes unrestricted scores and loadings under this reconstruction equation; additional constraints can make some transformations inadmissible.
Worked Example: Reciprocal Scaling Across Two Omics Views
Suppose a hypothetical two-factor fit has score matrix \(Z\) with first factor scores \((1,2)^{\mathsf T}\), and the first-factor loading vectors are \(B_1=(3,1)^{\mathsf T}\) for view 1 and \(B_2=(2,-1)^{\mathsf T}\) for view 2. Consider only factor 1 and multiply its scores by \(c=2\).
The transformed scores are \(Z_1^*=(2,4)^{\mathsf T}\). To preserve the signal, divide each view’s corresponding loading by 2: \(B_{1,1}^*=(1.5,0.5)^{\mathsf T}\) and \(B_{2,1}^*=(1,-0.5)^{\mathsf T}\).
For sample 1 in view 1, the original fitted feature vector is \((1)(3,1)=(3,1)\); the transformed vector is \((2)(1.5,0.5)=(3,1)\). For sample 2 in view 2, the original vector is \((2)(2,-1)=(4,-2)\); the transformed vector is \((4)(1,-0.5)=(4,-2)\). The reconstruction is unchanged in both views. Dividing the loadings in only view 1, for example, would not preserve view 2’s signal.
Normalization Choices
A normalization assigns a chosen scale to each factor and leaves the reciprocal scale in the paired quantity. Two common choices are to give the sample scores unit sample variance or to give the loading vector a chosen norm. Neither is universally preferable. The first makes score magnitudes comparable under a particular sample-variance convention; the second makes loading magnitudes comparable under the chosen feature scale and norm. In multi-omics work, feature preprocessing matters: a loading norm can be dominated by differences in feature units unless the views have been appropriately scaled.
For example, let \(z_k\) denote column \(k\) of \(Z\), and let \(\bar z_k\) be its sample mean. The sample variance is \(s_{z_k}^2=\sum_{i=1}^n(z_{ik}-\bar z_k)^2/(n-1)\). If \(s_{z_k}>0\), define \(z_k^*=z_k/s_{z_k}\), and multiply the corresponding loading column in every view by \(s_{z_k}\). The transformed scores have sample variance 1, while every fitted signal remains unchanged. This convention does not imply that the underlying population variance is exactly 1.
Alternatively, if the loading vector in a selected view has nonzero Euclidean norm \(r_k=\lVert b_{v,k}\rVert_2\), one can set \(b_{v,k}^*=b_{v,k}/r_k\) and \(z_k^*=r_k z_k\). This makes that view’s loading norm 1. But if the factor is shared, the other views’ loading columns must also be divided by \(r_k\); otherwise their reconstructions change. Normalizing a loading in one designated view is therefore a convention about the common factor scale, not a separate change to that view alone.
A zero score variance or zero loading norm makes the corresponding normalization undefined. Such a factor carries no variation in the quantity used as its scale anchor, so the analyst must not divide by zero or report that the normalization was achieved. This is a diagnostic issue to investigate rather than a numerical inconvenience to hide.
Worked Example: Setting a Factor’s Sample Score Variance to One
Consider hypothetical centered scores \(z_1=(-2,0,2)^{\mathsf T}\) for three samples. Their mean is 0 and their sample variance is
Suppose the first-factor loading vector in a view is \(b_{v,1}=(1,3)^{\mathsf T}\). The normalized scores are \(z_1^*=z_1/2=(-1,0,1)^{\mathsf T}\), whose sample variance is \((1+0+1)/2=1\). The corresponding loading vector becomes \(b_{v,1}^*=2b_{v,1}=(2,6)^{\mathsf T}\).
For the first sample, the original fitted vector is \((-2)(1,3)=(-2,-6)\); after normalization it is \((-1)(2,6)=(-2,-6)\). For the third sample, both representations give \((2,6)\). Thus unit sample variance is attained without changing the fitted signal. For a shared factor, the same multiplication by 2 applies to its loading column in every view.
Sign Conventions and Their Limits
A sign flip is the special case \(D_{kk}=-1\): replace \(z_k\) by \(-z_k\) and every corresponding loading column by its negative. Each product of a score and loading remains the same. A sign convention chooses one of these equivalent orientations so that two otherwise identical fits do not describe the same pattern with opposite signs.
One practical rule is to choose a designated anchor loading and require it to be positive. If the anchor is negative, multiply the factor’s scores and all its loading columns by \(-1\). This rule is simple, but it fails when the anchor is zero or very close to zero; a small estimation change can then reverse the chosen orientation. Another rule is to make the loading with largest absolute value positive, using a prespecified tie rule if needed. This avoids choosing an arbitrary single feature as anchor, but the identity of the largest loading can change between fits when magnitudes are close.
The rule must be applied consistently to all views for a shared factor. If it is based on an anchor in one view, it determines the sign of that factor’s loading columns in every other view as well. It is usually sensible to state the anchor, feature preprocessing, and tie handling when reporting results. A sign convention makes orientation reproducible; it does not establish that a positive loading has a causal or inherently beneficial meaning.
Worked Example: Applying a Sign Convention
Suppose a hypothetical estimated factor has score vector \(z_2=(1,-1,2)^{\mathsf T}\), and its loading columns in two views are \(b_{1,2}=(-0.6,0.2)^{\mathsf T}\) and \(b_{2,2}=(-0.1,0.5)^{\mathsf T}\). Use the first feature in view 1 as the anchor and require its loading to be positive.
The anchor is \(-0.6\), so flip the factor: \(z_2^*=(-1,1,-2)^{\mathsf T}\), \(b_{1,2}^*=(0.6,-0.2)^{\mathsf T}\), and \(b_{2,2}^*=(0.1,-0.5)^{\mathsf T}\). For sample 1, the original view 2 fitted vector is \(1(-0.1,0.5)=(-0.1,0.5)\); the transformed vector is \((-1)(0.1,-0.5)=(-0.1,0.5)\). The view 1 signal is preserved by the same calculation. The anchor is now positive, but the rule describes orientation only; it does not alter the fit or give the scores a new substantive interpretation.
Justification and Probabilistic Qualifications
The reconstruction invariance follows directly from matrix multiplication. Since \(D\) is diagonal, it is symmetric, so \((B_vD^{-1})^{\mathsf T}=D^{-1}B_v^{\mathsf T}\). Therefore,
The argument requires \(D\) to be invertible, which is why its diagonal entries must be nonzero. It establishes equality of fitted signals, and therefore equality of residual matrices \(X^{(v)}-ZB_v^{\mathsf T}\), for this reconstruction model.
A probabilistic score model adds an important condition. If the rows of \(Z\) are assumed to have a fixed distribution with covariance \(I_K\), arbitrary rescaling generally does not preserve that distribution: \(ZD\) has score covariance \(D^2\) when the original covariance is \(I_K\). Sign flips do preserve it, because a diagonal sign matrix \(D\) satisfies \(D D^{\mathsf T}=I_K\). A scale change can still be a valid reparameterization if the score covariance is transformed as well, but it is not a symmetry under a fixed unit-covariance constraint. Thus a scale-normalization rule must respect the actual likelihood, prior, and parameter constraints, not merely the fitted-product equation.
Comparisons, Diagnostics, and Common Mistakes
| Convention | What is fixed | Useful when | Watch for |
|---|---|---|---|
| Unit sample score variance | Sample variance of each score column | Comparing score magnitudes across factors or fits | Requires nonzero score variation; depends on the sample and variance convention |
| Unit loading norm | Norm of a specified loading column | Comparing loading patterns under consistent feature scaling | Requires a nonzero anchor-view loading norm; all views must be transformed consistently |
| Positive anchor loading | Sign of a selected feature loading | Giving a factor a reproducible orientation | Unstable or undefined if the anchor loading is near zero |
| Largest loading positive | Sign of the largest-magnitude loading | Avoiding dependence on one prespecified feature | Can be unstable when top magnitudes are tied or nearly tied |
When two fits appear to disagree, first check whether their factor columns are in the same order, scale, and orientation before interpreting differences in scores or loadings. For the scale-and-sign question, inspect score variances and loading norms under the stated feature preprocessing, verify that every view uses the same shared-factor transformation, and check whether a model constraint forbids the proposed rescaling. If a sign anchor is weak, report that the orientation is unstable rather than treating a sign change as evidence of a changed biological pattern.
A common mistake is to say, “The score doubled, so the factor’s contribution doubled.” That conclusion is wrong if the paired loading was halved: their product, and hence the fitted contribution, is unchanged. The correction is to interpret scores and loadings jointly under a stated convention. Another mistake is to normalize scores while leaving loadings untouched, or to flip a loading in one view only; either action generally changes the fitted signal. Finally, reconstruction invariance alone does not prove that a transformation respects a fixed latent-score distribution or other model constraints.
These conventions improve comparability, not every aspect of factor interpretation. The choice of score or loading scale is a reporting and parameterization decision whose meaning depends on feature units and model assumptions. If a factor has a zero or nearly zero normalization anchor, or if transformations violate a fixed probabilistic constraint, the chosen convention should be reconsidered rather than applied mechanically.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- If one factor’s scores are multiplied by 3, what must happen to its loading columns across all views to preserve the fitted signal?
- A score vector is \((-1,0,1)^{\mathsf T}\) for three samples. What is its sample variance, and what reciprocal loading adjustment preserves the signal if the scores are rescaled to have sample variance 1?
- Why does flipping a factor’s scores but not its loadings change the fitted signal?
- Does arbitrary positive rescaling preserve a model that fixes the latent-score covariance at \(I_K\)? Explain briefly.
- An anchor loading used to set factor orientation is exactly zero. What should the analyst conclude about that sign rule?
Solutions. 1. Divide the corresponding loading column in every view by 3. 2. The sample variance is \((1+0+1)/(3-1)=1\), so no rescaling is needed; if a score vector with standard deviation \(s\) were divided by \(s\), its loadings would be multiplied by \(s\). 3. The score-loading product changes sign, so its fitted contribution is negated rather than preserved. 4. No. Rescaling changes the score covariance from \(I_K\) to a different covariance; it requires changing that distributional constraint as well. 5. The rule cannot determine an orientation from that anchor, so use a different stated rule or report that the orientation is unresolved by the chosen anchor.