Bootstrap Stability Of Loading Profiles

Why Assess Loading Stability?

A loading profile describes how measured features contribute to a latent factor. A large loading in one fitted model may look compelling, but it can depend on which subjects happened to be observed, on a few influential samples, or on weak separation between factors. Bootstrap stability analysis asks a targeted question: if subjects like these had been sampled again, would the same features still contribute in similar ways?

The focus here is uncertainty in the loading profiles, not uncertainty in subject scores. Because factor solutions can be reordered or have their signs reversed without changing the fitted representation, bootstrap loadings must be matched to a reference solution before their distributions are summarized. Those ambiguities are addressed operationally below; matching does not make a factor’s global sign biologically absolute.

Setup and Assumptions

Suppose \(n\) subjects have measurements on \(V\) matched omics views. View \(v\) has \(p_v\) features. Let \(\hat{\lambda}_{vjk}\) denote the loading of feature \(j\) in view \(v\) on reference factor \(k\), estimated from the original data. A bootstrap replicate resamples the subject rows and refits the same factor model. The same sampled subjects must be used across views so that the matched multi-omics structure is preserved.

The usual nonparametric bootstrap treats the observed subjects as independent draws from the population of interest. It approximates repeated-sample variation by sampling \(n\) subject rows with replacement from the observed rows. It also assumes the analysis is repeated consistently: same model specification, factor number, feature definitions, preprocessing rules, and loading scale. If transformations or normalization parameters are estimated from the data, they should generally be re-estimated within each replicate when they are part of the intended analysis pipeline.

A loading’s scale must be comparable across fits. For example, if features are standardized before fitting, apply the same standardization convention in every replicate; if the model rescales factors, use a consistent normalization before comparing loadings. Otherwise, apparent loading variability can be an artifact of changing units rather than sampling uncertainty.

Matching Factors and Summarizing Loadings

For each bootstrap fit, first match its factors to the reference factors, using the model’s established matching procedure (for example, a similarity-based assignment). Then align each matched factor’s sign. If the loading vectors for a matched factor in the reference and bootstrap fit are \(\hat{\boldsymbol{\lambda}}_k\) and \(\hat{\boldsymbol{\lambda}}_k^{*(b)}\), respectively, a common sign rule is to multiply the bootstrap factor and its corresponding scores by \(-1\) when their inner product is negative. This chooses the orientation most similar to the reference. It does not alter the fitted reconstruction.

For a fixed feature and matched factor, the bootstrap standard error is the sample standard deviation across aligned replicates. A percentile interval uses the corresponding empirical quantiles of those replicates. The sign-consistency proportion records the fraction of aligned estimates with the same sign as the reference loading. A magnitude summary, such as the median of the absolute loadings, can be useful when sign stability is low, but it must not be mistaken for evidence that the signed effect is stable.

The bootstrap standard error estimates the sampling variability of the loading estimator under the resampling scheme. It is not the standard deviation of measurement error for that feature, nor does it include every possible source of modeling uncertainty.

Magnitude Uncertainty and Sign Consistency

A percentile interval for a signed loading is formed from the lower and upper empirical quantiles of its aligned bootstrap estimates. For example, a central 95% percentile interval uses the 2.5th and 97.5th percentiles. If that interval lies entirely above zero, the bootstrap estimates support a consistently positive loading relative to the chosen reference orientation; if it lies entirely below zero, they support a consistently negative one. If it includes zero, the sign or magnitude may be unstable. This is a bootstrap uncertainty summary, not a statement that there is a 95% probability the fixed population loading lies in the realized interval.

A complementary summary is the sign-consistency proportion. For a positive reference loading, it is the fraction of aligned estimates greater than zero; for a negative reference loading, it is the fraction below zero. A proportion near one indicates that the aligned loading retains the reference sign in most resamples. A proportion near one-half indicates frequent sign changes. Any operational cutoff, such as 0.90 or 0.95, should be specified before screening features and reported as a descriptive stability rule, not as a universal significance threshold.

Here \(I[\cdot]\) equals one when the condition is true and zero otherwise. The sign summary is meaningful only after factor matching and orientation alignment. Without those steps, a factor’s arbitrary reversal could make a perfectly consistent profile appear to change sign.

From Individual Loadings to Reproducible Features

A factor profile is often interpreted through a set of prominent features rather than one loading at a time. A practical screen can combine magnitude and consistency: require a feature to have a sufficiently large reference loading or bootstrap magnitude, retain its sign in a prespecified fraction of replicates, and appear among the leading contributors often enough. These criteria should reflect the scientific purpose and the scale of the loadings. They are descriptive selection rules, not automatic tests of biological importance.

One useful measure is top-feature frequency: the fraction of bootstrap fits in which a feature falls among the \(r\) largest absolute loadings for a matched factor. This asks whether the feature remains prominent, even if its precise loading varies. A magnitude threshold and top-feature frequency answer different questions: the former concerns effect size, while the latter concerns rank within the profile.

Justification and Diagnostic Checks

The bootstrap standard error formula follows directly from treating the \(B\) replicate estimates as an empirical approximation to the sampling distribution. Its empirical mean is \(\bar{\lambda}^{*}\); the unbiased sample variance of those \(B\) values divides the squared-deviation sum by \(B-1\). Taking the square root returns variability on the same scale as the loading. This is why the calculation uses the replicate estimates, rather than the variation of raw feature measurements across subjects.

Before interpreting summaries, check whether the resampling analysis is behaving as intended:

• Factor matches: Inspect similarity between each bootstrap factor and its assigned reference factor. Poor matches or frequent ambiguous assignments suggest factor identity is not stable enough for a simple loading summary.
• Fit failures: Record nonconvergence, degenerate fits, or solutions with implausible loading scales. Dropping failures without reporting them can make stability look better than it is.
• Distribution shape: Plot or inspect aligned loading distributions. Skewness, heavy tails, or multiple modes can make a standard error alone misleading; report quantiles and sign frequency as well.
• Influential subjects: If a small number of resamples produce extreme loadings, inspect whether they repeatedly contain or omit particular subjects. Such behavior may reveal sensitivity rather than ordinary sampling noise.
• Resampling design: If subjects are clustered or sampled in strata, resampling independent subject rows may be inappropriate. The resampling unit and design should reflect how the data were collected.

These checks help distinguish uncertainty in a loading from failure to identify a comparable factor. If factor matching itself is unstable, reporting a narrow loading interval for a forced match can be falsely reassuring.

Alternatives and Limitations

Bootstrap summaries are useful because they can be applied to complicated estimators without deriving a sampling distribution for every loading. They differ from several alternatives. Repeated random starts assess sensitivity to optimization and initialization, not sampling variation; they are a useful convergence check but do not replace subject resampling. Analytical standard errors can be faster and more precise when their assumptions and derivations fit the model, but may be difficult to obtain for complex latent-factor estimators. Bayesian posterior summaries quantify uncertainty under a specified likelihood and prior; they are not bootstrap estimates and depend on the Bayesian model and prior choices.

The subject bootstrap also has limits. It estimates uncertainty conditional on the observed feature set, model structure, preprocessing pipeline, and factor-matching rule. It does not automatically account for choosing the factor number after inspecting the same data, uncertainty in which features were measured, or changes in prior and hyperparameter choices. In small samples or high-dimensional settings, resamples may omit influential patterns or produce unstable fits. More bootstrap replicates reduce Monte Carlo error in the summaries but cannot repair a poor model, inadequate sample, or non-identifiable factor.