Sensitivity To Prior And Hyperparameter Choices

Why Prior Sensitivity Matters

In a Bayesian factor model, the likelihood describes how the observed omics measurements relate to latent scores and loadings; the prior expresses constraints or preferences about those unknown quantities. A prior that concentrates loadings near zero, for example, can stabilize estimates in a high-dimensional view, but can also attenuate a real signal. Thus, an interpretable factor is not automatically evidence that the same interpretation would appear under another defensible prior.

Sensitivity analysis addresses a focused question: if the data, likelihood, factor rank, and fitting procedure are held fixed, how much do the conclusions change when prior strengths or hyperparameter values change? It is not a search for the one prior that produces a preferred biological story. It is a check on whether the interpretation is supported by the data across reasonable prior specifications.

Definition: A prior-sensitivity analysis refits the same Bayesian model to the same data under a prespecified set of defensible prior distributions or hyperparameter values, then compares aligned posterior summaries and substantive interpretations. A conclusion is prior-sensitive when a meaningful change in those choices changes the conclusion, not merely a low-order decimal in an estimate.

Model, Assumptions, and What Is Varied

For a concrete notation, let \(Y^{(v)}\) be the \(n\)-by-\(p_v\) measurement matrix for omics view \(v\), with subjects in rows and features in columns. A shared-factor Gaussian model can be written

Y^{(v)} = F\Lambda^{(v)T} + E^{(v)}, \qquad E^{(v)}_{ij} \mid F,\Lambda^{(v)},\sigma_{vj}^{2} \sim N(0,\sigma_{vj}^{2}).

Here \(F\) is an \(n\)-by-\(K\) matrix of subject scores, \(\Lambda^{(v)}\) is a \(p_v\)-by-\(K\) loading matrix, and \(K\) is the chosen number of factors. This formulation assumes that, conditional on the factor scores and loadings, residuals have mean zero and are independent across subjects and features as specified by the likelihood. If a particular model instead includes residual correlations or view-specific factors, those features must be held fixed during this prior comparison.

A prior specification might assign \(F_{ik}\sim N(0,1)\) and \(\lambda^{(v)}_{jk}\sim N(0,\tau_v^2)\). The value of \(\tau_v\) controls loading shrinkage: smaller values put more prior mass near zero. Other models may have global and feature-specific shrinkage parameters, whose hyperparameters determine the prior distribution of those scales. In a sensitivity analysis, change one planned prior choice at a time where feasible, or use a small grid of prespecified combinations. Keep the data transformations, likelihood, factor rank, and other modeling choices unchanged so that the comparison can be attributed to the prior settings.

Factor solutions also have symmetries. Permuting factor columns and changing a factor’s sign can leave the fitted data unchanged if the corresponding loadings are transformed in the same way. Some models allow broader rotations as well. Therefore, raw column-by-column comparisons can make essentially equivalent solutions look different. First use the sign and permutation matching appropriate to the model; use rotation alignment only when the model’s constraints make rotations equivalent. Then compare scores and loadings on a common scale and with the same feature set.

A Conjugate Calculation: How Shrinkage Changes a Loading

A one-loading calculation makes the mechanism transparent. Suppose factor scores are treated as fixed, and after projecting one standardized feature onto one score, the resulting estimate \(z\) has likelihood \(z\mid\lambda\sim N(\lambda,s_z^2)\). Assume \(s_z^2\) is known and the loading prior is \(\lambda\sim N(0,\tau^2)\). These assumptions give a normal posterior. The example below uses this conditional calculation to isolate the effect of the prior; a full factor-model fit also estimates scores and other parameters.

Result: Under the stated normal likelihood and normal prior, the posterior variance and mean are

V_{\text{post}} = \left(\frac{1}{s_z^2}+\frac{1}{\tau^2}\right)^{-1}, \qquad m_{\text{post}} = V_{\text{post}}\frac{z}{s_z^2}.

The posterior mean is a precision-weighted compromise between the data estimate \(z\) and the prior mean zero. Reducing \(\tau\) increases prior precision \(1/\tau^2\), moving the posterior mean toward zero.

To justify the result, multiply the likelihood density by the prior density and retain the terms involving \(\lambda\). The log of this product, up to an additive constant, is

-\frac{1}{2}\left\{\frac{(z-\lambda)^2}{s_z^2} +\frac{\lambda^2}{\tau^2}\right\}.

Expanding the squares, the coefficient of \(\lambda^2\) is \(-\frac12(1/s_z^2+1/\tau^2)\), and the coefficient of \(\lambda\) is \(z/s_z^2\). Completing the square gives a normal density with variance \(V_{\text{post}}\) and mean \(m_{\text{post}}\) as stated. This derivation depends on the specified Gaussian likelihood, known \(s_z^2\), and independent Gaussian prior; it is not a general formula for every factor-model prior.

Worked Example: A Loading Under Weak and Strong Shrinkage

Suppose a hypothetical standardized feature has projection estimate \(z=0.80\) on a fixed factor, with known standard error \(s_z=1/\sqrt{10}\). Compare two zero-centered normal priors, \(\tau=1.5\) and \(\tau=0.30\). For both calculations, the Gaussian likelihood and data are identical, so the assumptions of the conjugate result hold.

For \(\tau=1.5\), \(s_z^2=0.10\), and the posterior precision is \(1/0.10+1/1.5^2=10+0.4444=10.4444\). Thus

V_{\text{post}}=1/10.4444=0.09574,\qquad m_{\text{post}}=0.09574(0.80/0.10)=0.7659.

The posterior standard deviation is \(\sqrt{0.09574}=0.3094\). A normal 95% posterior interval is \(0.7659\pm1.96(0.3094)=(0.1595,1.372)\).

For \(\tau=0.30\), the posterior precision is \(10+1/0.30^2=21.1111\). Therefore

V_{\text{post}}=1/21.1111=0.04737,\qquad m_{\text{post}}=0.04737(8)=0.3789.

The posterior standard deviation is \(0.2176\), and the 95% posterior interval is \(0.3789\pm1.96(0.2176)=(-0.0476,0.8055)\). Stronger shrinkage has moved the mean substantially toward zero and narrowed the interval. These are posterior intervals conditional on the stated model and prior; they are not frequentist confidence intervals. The calculation demonstrates a possible prior effect, but does not establish that either prior is appropriate for a real analysis.

Align Before Comparing

After fitting the full model under each prior setting, align factors before assessing sensitivity. Matching should use information that reflects the factor’s role, such as loading-profile similarity across corresponding features, with a consistent sign convention. If factor order is ambiguous, choose the permutation that maximizes the prespecified matching criterion. Then apply the same sign change or permutation to the associated scores. Do not align scores independently of their loadings: a score column and its loading column jointly define a factor.

Useful comparisons include correlations between matched score vectors, cosine similarities or correlations between matched loading profiles, changes in posterior means or medians, and changes in posterior spread. Score scale and loading scale must be made comparable across fits; otherwise a rescaling can look like sensitivity even when the fitted product \(F\Lambda^{T}\) is essentially unchanged. A similarity measure is a diagnostic summary, not a universal pass/fail threshold. Set any thresholds in advance and interpret them in context.

Worked Example: Matching a Factor With a Sign and Order Change

Consider hypothetical score summaries from two fits on the same three subjects. In the first fit, a factor’s scores are \((-1,0,1)\). In the second fit, the corresponding raw factor column is \((-0.8,0.1,0.7)\), but it appears in the opposite column order from the first fit. The loading profile in the first fit is \((0.8,0.6,0)\); the matched profile in the second is \((0.6,0.8,0)\). Assume the same features and score scaling are used in both fits.

After matching the factor order, the score vectors have means zero. Their correlation is

r = \frac{(-1)(-0.8)+(0)(0.1)+(1)(0.7)} {\sqrt{[(-1)^2+0^2+1^2][(-0.8)^2+0.1^2+0.7^2]}} = \frac{1.5}{\sqrt{2(1.14)}}=0.9934.

The root mean squared score difference is \(\sqrt{[(0.2)^2+(-0.1)^2+(0.3)^2]/3}=0.2160\) score units. The loading cosine similarity is \((0.8)(0.6)+(0.6)(0.8)+(0)(0)=0.9600\), since both loading vectors have unit length. The example suggests a similar factor direction in the two fits, though the nonzero score differences and loading-profile differences should still be reported. Comparing the raw factor columns without matching their order could instead pair unrelated factors and produce a misleadingly poor comparison.

Interpreting Sensitivity and Diagnosing Problems

The practical interpretation should distinguish several kinds of change. A factor may remain recognizable while its loading magnitudes shrink; the same features may retain their direction and relative ordering even when their posterior means differ. Alternatively, a factor may disappear, split, merge, or no longer match across fits, so a biological label attached to it is not robust to the chosen prior. Report both the changes in numerical summaries and what they imply for the specific scientific claim.

Worked Example: A Stable Direction but a Prior-Dependent Magnitude

Suppose two hypothetical refits yield an aligned factor with positive posterior mean loadings for a feature in RNA and a matched feature in protein. Under a weaker prior, the means are \(0.72\) and \(0.58\), with posterior standard deviations \(0.20\) and \(0.18\). Under a stronger prior, the means are \(0.31\) and \(0.26\), with standard deviations \(0.12\) and \(0.11\). Assume the factor is matched on the same features and the score correlation between fits is \(0.94\).

The RNA loading decreases by \(0.72-0.31=0.41\), or \(0.41/0.72=0.569\), about 56.9% of its weaker-prior magnitude. The protein loading decreases by \(0.58-0.26=0.32\), or about 55.2%. Both posterior means remain positive, and the aligned scores remain correlated, so the direction of a cross-view pattern may be reasonably stable in these summaries. Its magnitude is clearly prior-sensitive. A defensible report would retain that distinction rather than claiming either that nothing changed or that the entire factor interpretation failed. These illustrative summaries are hypothetical model outputs, not evidence from an actual study.

Before interpreting differences between prior settings, check that each fit has adequately converged using the diagnostics appropriate to the sampler and model. Poor mixing, chain disagreement, or unstable estimates can imitate prior sensitivity; refitting or improving computation is needed before attributing differences to priors. Conversely, good convergence under each setting does not establish prior robustness. Convergence asks whether a specified posterior was explored adequately; sensitivity asks whether the posterior conclusions change across specifications.

A useful analysis also checks whether results are dominated by the prior. Strong shrinkage can pull weakly informed loadings toward zero, while a weakly regularizing prior can permit large, noisy loadings in high-dimensional settings. Look for large shifts in matched summaries, changes in the set or ordering of prominent features, and factors that cannot be matched reliably. A prior that produces implausible scales or is incompatible with the measurement scale is not a useful sensitivity alternative merely because it is different.

Choosing Comparisons and Recognizing Limits

Prior sensitivity is different from several related analyses. Resampling-based factor stability changes which subjects are observed and assesses dependence on the sample; prior sensitivity keeps the observed data fixed. Feature-set perturbation changes which measurements enter the model. Varying residual noise assumptions or factor rank changes the model or likelihood structure. These checks can complement one another, but they answer different questions. Changing several of these elements at once makes it difficult to identify why the fitted interpretation changed.

A practical comparison can use a reference prior and a small set of alternatives chosen before inspecting which one gives the most appealing result. Change the main shrinkage scale across a defensible range, or vary a hyperparameter that controls a hierarchical shrinkage distribution. Keep other choices fixed, fit each model with adequate computation, align the factors, and summarize changes in scores, loadings, and posterior spread. If a prior is learned from the data through a hyperprior or empirical Bayes procedure, specify that procedure and assess the relevant hyperprior or estimation choices rather than treating the resulting shrinkage level as fixed without qualification.

No finite grid proves robustness to every possible prior. Alignment may also be ambiguous when factors are weak, nearly tied, or substantially restructured. Similar factor scores do not guarantee identical feature-level interpretations, and similar loading profiles do not by themselves establish a biological mechanism. Conclusions remain conditional on the likelihood, preprocessing, factor rank, and observed features; prior sensitivity analysis does not remove those limitations.

Key takeaway: Keep the data and model structure fixed, vary defensible prior choices, and align the resulting factors before comparing them. State separately which aspects of the interpretation are stable and which depend on prior strength or hyperparameter values.

Check Your Understanding

Answer each question, then compare with the solutions below.

In a prior-sensitivity analysis, why should the likelihood, factor rank, and data transformations usually remain fixed?
In the normal loading example, what happens to posterior shrinkage toward zero when \(\tau\) decreases, and why?
Two fits contain the same factors but in a different order, with one factor’s sign reversed. What must be done before comparing its scores and loadings?
How does prior sensitivity differ from resampling-based factor stability?
If factor scores remain similar across priors but loading magnitudes change substantially, what is an appropriately qualified interpretation?

Solutions. (1) Holding these elements fixed makes differences more attributable to the prior settings rather than to a changed model or dataset. (2) Shrinkage increases because prior precision \(1/\tau^2\) rises, pulling the posterior mean closer to the prior mean of zero. (3) Match the factor permutation and sign, applying the same transformations jointly to its score and loading columns; then compare on a common scale. (4) Prior sensitivity changes the prior while retaining the observed data, whereas resampling-based stability changes the sampled subjects or resamples and asks whether the result depends on the sample. (5) The factor-score pattern may be stable, while the loading magnitudes—and therefore claims about effect strength or feature importance—are prior-sensitive. Report those two findings separately rather than treating one as proof of the other.