Stability To Feature Set Perturbations

Why Perturb the Feature Set?

A factor can appear interpretable because a particular group of measured features dominates its loading profile. The practical question is whether the factor would still be found if some features were unavailable, down-weighted through subsampling, or deliberately omitted. This is a different stability question from changing which subjects are analyzed: here the subjects are held fixed, while the measured columns are changed.

For multi-omics data, feature perturbations can be applied within each view, such as genes within a transcriptomic block or metabolites within a metabolomic block. A perturbation may randomly retain a fraction of features, omit one prespecified feature group, or remove features in a controlled proportion from each view. The design of the perturbation determines what kind of robustness is being assessed.

Set Up Comparable Refits

Let \(X\) be an \(n \times p\) matrix of transformed measurements for the same \(n\) subjects and \(p\) features. In a \(K\)-factor model, write \(X \approx FL^{T}+E\), where \(F\) contains subject scores, \(L\) contains feature loadings, and \(E\) contains residual variation. The loading for feature \(j\) on factor \(k\) is \(l_{jk}\). This notation is used here to describe the comparison; the particular estimation method can be Bayesian or otherwise.

Fit the prespecified model to the full feature set to obtain a reference loading matrix \(L\). For perturbation \(b\), retain a feature subset \(S_b\) and refit the same model using the same subjects, factor count, preprocessing choices, priors or penalties, and convergence criteria. Denote its loading matrix by \(L^{(b)}\). Rows for features absent from \(S_b\) do not exist in that refit, so comparisons must be restricted to shared features.

This design assumes that the retained features provide enough information to estimate the specified number of factors, that the model fits converge adequately, and that the loading scales are made comparable. It also assumes that the measurement definitions and preprocessing are consistent across fits. If standardization is part of preprocessing, for example, use a consistent transformation for each feature rather than allowing the feature subset to redefine the scale in an unintended way. A perturbation that leaves too few features in a view, changes the effective model, or causes non-convergence is not evidence of a stable or unstable factor; it is an unusable comparison.

Factor solutions may differ by sign or ordering, and some models allow broader rotational ambiguity. The earlier material on matching factors under sign and permutation ambiguity addresses this identification issue. For a simple comparison, match factors by maximizing total absolute loading similarity across candidate pairs, then flip signs to make matched loading profiles point in the same direction. If the model permits arbitrary rotations, factor-by-factor matching may be insufficient; align the loading subspaces on the shared features before interpreting individual factors.

Compare Loadings on Shared Features

For a reference factor \(k\) and its matched factor \(m\) in perturbation \(b\), let \(S_b\) be the retained features. A useful measure is the absolute cosine similarity of their restricted loading vectors:

When both restricted vectors are nonzero, \(C_{kb}\) lies between 0 and 1. A value near 1 indicates similar directions over the features available in that perturbation; a value near 0 indicates little directional similarity. Absolute value makes the measure invariant to a whole-factor sign flip. It does not mean that the loading magnitudes are equal, nor does it measure similarity on features that were omitted.

The formula is a normalized inner product. Dividing each vector by its Euclidean length gives unit-length vectors; their inner product is the cosine of the angle between them. The Cauchy–Schwarz inequality gives an absolute inner product no greater than the product of the lengths, so the normalized value cannot exceed 1. This also explains why a cosine can be high when both loading vectors are small in magnitude: it measures pattern direction, not factor strength.

Summarize Repeated Perturbations

A single feature subset may be unusually favorable or unfavorable. For repeated perturbations, predefine a similarity threshold \(\tau\) appropriate to the scientific use and report the distribution of \(C_{kb}\) values. One simple summary is the retention proportion: the fraction of usable perturbations in which factor \(k\) has a matched factor with similarity at least \(\tau\). It is a descriptive frequency under the chosen perturbation design, not a probability that the factor is biologically real.

where \(B\) is the number of usable refits and \(I(\cdot)\) equals 1 when its condition is true and 0 otherwise. State the threshold, the number and type of perturbations, and how many attempted refits were unusable. A median and range or quantiles of the similarities add information that a single retention proportion can hide.

Find Factors Dependent on a Narrow Feature Set

Random subsampling measures sensitivity to the chosen range of feature removals, but targeted omissions can expose dependence on a particular group. For a reference loading profile, the fraction of squared loading magnitude concentrated in a prespecified feature group \(G\) is

This descriptive concentration measure is between 0 and 1 when the denominator is positive. A large value means that much of the factor's loading energy is concentrated in \(G\); it does not by itself show that the factor will disappear if \(G\) is removed. A leave-group-out refit tests that directly by checking whether the factor is still estimable and whether a comparable loading pattern remains on the other features.

Design Choices, Diagnostics, and Alternatives

Choose perturbations to answer a defined question rather than deleting features indiscriminately. Random feature subsampling assesses sensitivity to broad feature loss. Leave-one-view-out or controlled within-view thinning assesses dependence on a data block, although removing an entire view changes the information available for joint factors. Leave-group-out perturbations test dependence on a specified pathway, assay family, or feature cluster. Report whether sampling was balanced across views: otherwise, a low-similarity factor may reflect the loss of one large block rather than general sensitivity to individual features.

Diagnostics matter because a low cosine need not indicate genuine feature dependence. Check convergence and loading scales in every refit; inspect whether factors split, merge, or exchange their order; and verify that enough shared features remain to support the comparison. Conversely, a high cosine on a very small shared set can be misleadingly reassuring. Report the number of shared features and, when relevant, their distribution across views. If no meaningful match exists, report that the factor was not recovered under the matching rule rather than forcing a match to the least dissimilar factor.

Feature-set perturbation complements, rather than replaces, subject resampling and bootstrap stability of loading profiles. Subject resampling asks whether the result changes with the sampled individuals; feature perturbation asks whether it changes with the measured variables. A fixed-score projection onto a reduced feature set is another possible sensitivity check, but it does not answer the same question as refitting: projected scores keep the original factor definition and cannot reveal how the estimated loading structure changes. Full refitting is more informative about loading robustness, but is computationally more demanding and may produce factors that are difficult to align.

There is no universal cutoff for a “stable” cosine. A threshold such as 0.80 is a decision rule, not a theorem, and should be justified by the intended interpretation. Feature correlations can also make a factor replaceable: dropping one feature may have little effect because correlated features remain, while dropping a whole correlated group may have a large effect. Finally, these analyses assess robustness to the perturbations actually performed. They do not establish causal mechanisms, biological validity, or stability under untested preprocessing and model choices.

Common Pitfall

A common mistake is to say, “The cosine is high, so the factor is stable,” after comparing only the features retained in one subset. That conclusion ignores the omitted features and may confuse directional similarity with unchanged magnitude or importance. The correction is to state the comparison domain explicitly, report coverage and the perturbation design, and combine repeated random subsets with targeted omissions when dependence on a particular feature group is the scientific concern.