Resampling-Based Factor Stability

What Does Factor Stability Mean?

A factor solution is an estimated representation of subjects and measured features in a lower-dimensional latent space. It is useful only if the structure is reproducible enough for the intended interpretation. Yet “stable” needs a precise meaning: a solution may recover nearly the same latent subspace across samples while assigning different axes within that subspace to individual factors. Resampling subjects makes this distinction visible by asking how the fitted structure changes when the observed subjects are perturbed.

Consider a centered, possibly feature-standardized multi-omics data matrix \(Y\), with \(n\) subjects in rows and \(p\) features across the omics views in columns. A rank-\(K\) factor representation can be written as

where \(Z\) is an \(n \times K\) matrix of subject scores and \(L\) is a \(p \times K\) matrix of feature loadings. In a multi-view analysis, the columns of \(Y\) may be arranged in blocks, one per view, and the corresponding rows of \(L\) retain that block structure. This notation describes the shared factor representation; it does not require that every factor affect every view.

A resampling analysis does not establish that a factor is biologically real, causal, or free of technical effects. It measures sensitivity to the particular subject perturbation. It also does not automatically include uncertainty from changing the feature set, model family, prior, or preprocessing: those are held fixed unless deliberately included in the procedure.

Why a Subspace Can Be Stable While Factors Are Not

The product \(ZL^\mathsf{T}\) can remain unchanged under a change of latent coordinates. For any invertible \(K \times K\) matrix \(R\), replacing \(Z\) with \(ZR\) and \(L\) with \(LR^{-\mathsf{T}}\) preserves that product. Thus, without further constraints, the model’s \(K\)-dimensional span may be identified even when its particular factor axes are not. Sign changes and permutations are especially simple examples of this ambiguity; the earlier material on Matching Factors Under Sign And Permutation Ambiguity addresses those alignment issues.

For a loading matrix \(L\) of full column rank, the feature-space subspace is \(\operatorname{span}(L)\). Replacing \(L\) by \(LR^{-\mathsf{T}}\) changes its columns but not their span. A subspace comparison therefore asks whether the fitted \(K\)-dimensional feature directions are similar, without requiring each axis to reproduce the same labels or order.

An individual factor is more demanding to assess. After accounting for sign and permutation, a factor axis is reproducible only if the fitted model gives that axis a sufficiently distinct and consistent role. When two directions are nearly interchangeable, a resample can rotate them within a stable subspace. In that case, interpreting a low factor-by-factor match as evidence that the entire latent structure disappeared would be misleading.

Comparing Estimated Subspaces

Suppose the reference fit and a resampled fit both estimate \(K\) dimensions. Let \(Q_0\) and \(Q_b\) be \(p \times K\) matrices whose columns are orthonormal bases for their loading subspaces. Their projection matrices are \(P_0=Q_0Q_0^\mathsf{T}\) and \(P_b=Q_bQ_b^\mathsf{T}\). These projectors do not depend on which orthonormal basis is used for either subspace.

The principal angles \(\theta_1,\ldots,\theta_K\), each between zero and \(\pi/2\), quantify the separation between two equal-dimensional subspaces. The singular values of \(Q_0^\mathsf{T}Q_b\) are \(\cos(\theta_j)\). A convenient bounded summary is the root-mean-square sine of those angles:

Here \(\lVert \cdot \rVert_F\) is the square root of the sum of squared matrix entries. The distance is zero for identical subspaces and approaches one for orthogonal subspaces. It is a descriptive distance, not a p-value or a universal pass/fail threshold.

The equality follows from the trace of the squared projector difference. Since \(P_0\) and \(P_b\) are symmetric idempotent projectors of rank \(K\), their squared Frobenius distance is

This comparison requires the same subspace dimension and a common feature coordinate system. In multi-omics work, the feature ordering, transformations, and view scaling used to define that coordinate system must be held fixed across fits. Otherwise, a measured difference can reflect changing scales or coordinates rather than resampling instability.

Designing a Subject-Resampling Assessment

The resampling unit should match the unit that is considered independently sampled. For independent subjects with paired measurements across several omics views, resample subjects jointly: each selected subject brings all of their view measurements and relevant covariates into the resampled data set. Sampling each view independently would break the cross-view pairing and assess a different data-generating setup. If observations are clustered or repeatedly measured, the independent unit may instead be a cluster or subject-level trajectory; the resampling scheme must preserve that dependence.

A bootstrap samples \(n\) subjects with replacement from the observed \(n\), so a resampled data set can contain repeated subjects and omit others. Subsampling selects subjects without replacement, often using fewer than \(n\) at a time. These schemes perturb the data in different ways; the choice should be reported and justified for the inferential target. In either case, the model and preprocessing should be refitted consistently. If a fit fails to converge, that failure is part of the assessment and should be reported, not silently replaced by a favorable fit.

There are two useful targets when rank can vary. A conditional assessment fixes \(K\) and asks whether that particular dimension’s fitted structure is reproducible. An end-to-end assessment repeats the rank-selection rule and reports how often each rank is selected, alongside stability comparisons for resamples with a comparable dimension. Subspace distances are not directly comparable when the dimensions differ, so the dimension must be stated rather than hidden in a pooled summary.

Summarizing Resampling Results

A stability report should show the distribution, not only an average. Useful summaries include the median and range or selected quantiles of the subspace distance, the frequency of rank outcomes when rank is reselected, the fraction of attempted fits that converge, and factor-match summaries when axis-level stability is a target. If a threshold is used to label a factor “matched,” specify it before examining results and show the underlying similarity distribution as well. A single threshold can conceal borderline matches.

Diagnostics, Alternatives, and Limits

Several checks help distinguish genuine resampling sensitivity from implementation problems. Confirm that subjects, rather than individual feature rows, are the sampling units; verify that paired views remain paired; and check that all fits use the same feature order and transformations. Inspect convergence and the selected dimension in every fit. A change in rank, failed optimization, or a large change in view scaling can make a factor comparison uninterpretable unless it is explicitly addressed. For algorithms with random initialization, use a consistent fitting policy and distinguish variation due to subject resampling from variation due only to random starts.

Subspace distances are most interpretable when the dimension is fixed and estimates are well-defined. When one resample selects a different dimension, comparing projectors as though both were rank \(K\) is not appropriate. Individual-factor matches can also be ambiguous when directions are nearly tied or when the model permits substantial rotations. Report such ambiguity instead of forcing a confident factor label.

Subject resampling is one alternative to evaluating reproducibility. A held-out split compares fits across separate portions of the data, but may be inefficient in a small cohort and can depend strongly on the split. Model-based posterior uncertainty describes uncertainty under a specified probabilistic model and prior; it answers a different question from perturbing the empirical subject sample. None of these approaches by itself establishes biological validity. Resampling stability is conditional on the measured cohort, chosen features, model, preprocessing, and fitting protocol.

A common mistake is to say, “Factor 1 changed its loading pattern, so the factor model is unstable,” without checking whether the span of the factors remained similar. The correction is to separate the two questions: quantify subspace stability without depending on an arbitrary basis, then assess individual factors only after alignment and with attention to whether their axes are distinguishable. The reverse mistake is also possible: a stable subspace does not guarantee that each factor has a reproducible interpretation.