Shrinkage Of View-Specific Effects

Why Shrink View-Specific Effects?

A multi-omics factor model can represent patterns that vary together across assays and patterns that are confined to one view. A view-specific loading describes how strongly a feature in one assay responds to a factor particular to that view. If the data provide weak evidence for such a loading, an unregularized estimate may assign substantial private structure to noise. A prior centered at zero offers a way to temper that estimate while still allowing a loading to remain nonzero when the evidence is strong.

The aim is not to declare that private biology is absent. Rather, shrinkage expresses the modeling preference that a view-specific effect should need evidence to move away from zero. How strongly this preference acts depends on the prior scale, the amount of information in the data, and the scale constraints on the factor scores. The scale-and-sign constraints and identifiability of shared and specific factors are essential context: a loading's numerical size has meaning only relative to the scale of its factor.

A Conditional Model and Its Assumptions

To isolate the update for one private loading, suppose the shared contribution to feature \(j\) in view \(m\) has been accounted for. Write the resulting residual response as \(r_i\), and consider the conditional model

Here \(u_i\) is the view-specific score and \(\gamma\) is the loading being regularized. For the conjugate calculation below, treat the scores, residual responses, and residual variance \(\sigma^2\) as known, and assume independent Gaussian errors with common variance. The prior is \(\gamma\sim N(0,\tau^2)\), independently of the errors, with \(\tau^2>0\). In a fitted factor model these are conditional assumptions: scores, shared contributions, and variances are generally uncertain, so a joint Bayesian analysis must account for that uncertainty rather than treating plug-in values as known without qualification.

Define \(S_{uu}=\sum_i u_i^2\) and \(S_{ur}=\sum_i u_i r_i\). Provided \(S_{uu}>0\), the unregularized least-squares estimate is \(\hat{\gamma}=S_{ur}/S_{uu}\), with sampling variance \(v=\sigma^2/S_{uu}\) under this model. Thus the conditional likelihood for \(\hat{\gamma}\) is proportional to the density of \(N(\gamma,v)\). A score with little variation, or a noisy response, produces a large \(v\), so the data exert less influence relative to the prior.

The posterior mean is a weighted version of the unregularized estimate. Its multiplier \(\tau^2/(\tau^2+v)\) lies between zero and one: smaller prior variance or larger sampling variance yields stronger shrinkage toward zero. The posterior variance is not the squared difference between the posterior mean and the estimate; it quantifies remaining uncertainty under the model and prior.

Why the Posterior Update Shrinks

The result follows directly by combining the likelihood and prior exponents. Omitting terms that do not depend on \(\gamma\), the log posterior is

Let \(A=S_{uu}/\sigma^2+1/\tau^2\) and \(B=S_{ur}/\sigma^2\). Completing the square gives

Therefore the posterior is normal with mean \(B/A\) and variance \(1/A\), which are the stated expressions. This derivation also shows the role of precision: the posterior precision is the sum of likelihood precision \(S_{uu}/\sigma^2\) and prior precision \(1/\tau^2\). The prior contributes information centered at zero, not at a nonzero alternative value.

How Shrinkage Changes the Shared–Private Balance

For illustration, consider a linear two-view model with shared score vector \(z_i\), private score \(u_i^{(m)}\) for view \(m\), shared loading vector \(\lambda^{(m)}\), and private loading vector \(\gamma^{(m)}\):

If shared and private scores are independent, have identity covariance under the chosen scale constraints, and are independent of residual errors with covariance \(\Psi^{(m)}\), then the within-view covariance is

The shared term can generate covariance across views; the private term contributes within-view covariance, but not cross-view covariance under the stated independence assumptions. Shrinking private loadings toward zero therefore reduces the model's capacity to explain view-specific covariance. It does not mechanically transfer the same amount of variance to the shared component. In a joint refit, shared effects may explain patterns that recur across views, but only if the data and model support that explanation. Otherwise, the pattern may remain in residual variation or be missed.

Prior Strength, Alternatives, and Sensitivity

The prior variance \(\tau^2\) determines the degree of regularization. As \(\tau^2\) becomes large, prior precision approaches zero and the posterior mean approaches \(\hat{\gamma}\). As \(\tau^2\) approaches zero, the posterior mean approaches zero. In practice, one may use a common scale across loadings, feature- or view-specific scales, or hierarchical scales that allow some loadings to escape stronger global shrinkage. These choices encode different assumptions and should be described rather than treated as automatic fixes.

A normal prior yields a ridge-like penalty: its negative log density adds \(\gamma^2/(2\tau^2)\) to the objective, up to a constant. This is a continuous penalty, unlike a point-mass selection model. The earlier material on sparse loading priors addresses sparsity and feature selection; here the emphasis is how regularizing private effects alters the model's allocation of variation. A loading prior should not be confused with a prior on residual variance, which regulates a different part of the model.

Diagnostics and Failure Modes

Shrinkage is useful only if its assumptions and consequences are checked. Useful questions include whether private scores have adequate variation, whether the residual noise scale is plausible, whether the prior is calibrated to the loading scale, and whether substantial view-specific residual patterns remain after fitting. In a Bayesian fit, examine posterior distributions and compare conclusions under reasonable prior scales. Posterior predictive checks can also reveal whether the fitted model systematically misses variation within a view. These checks do not prove that a private factor is biologically meaningful, but they can flag a model that has shrunk too aggressively or assigned variation poorly.

• Over-shrinkage: A narrow prior can pull genuine private effects close to zero, leaving structured within-view variation unexplained. Reconsider the scale and inspect residual patterns rather than assuming all private structure is noise.
• Under-shrinkage: A very diffuse prior can leave weakly supported private loadings largely unchanged. This is a concern when the data provide little information, such as when score variation is limited.
• Scale mismatch: If score scaling changes but the prior scale does not, the implied regularization changes. Interpret \(\tau^2\) in the units induced by the score constraints.
• Confounded directions: If shared and private factors are not distinguishable under the model, shrinkage may encourage one allocation without resolving the underlying identifiability problem.
• Plug-in certainty: Treating estimated scores or residual variances as known can make a conditional update appear more certain than a joint analysis warrants.

Shrinkage can stabilize estimates, but it does not establish identifiability, correct misspecification, or prove a causal interpretation. Its main limitation is that conclusions can depend materially on the prior, particularly with weak information. It should be reported as part of the modeling assumptions, with units and score scaling made explicit.