Creating Shared And Modality-Specific Signals

Why Separate Shared and Modality-Specific Signals?

A multi-omics simulation can contain dependence between modalities without making every signal shared. Some subject-to-subject variation may affect several assays, while other variation may be present in only one modality. A useful simulation therefore makes those sources explicit: it lets a benchmark test whether an integration method distinguishes cross-modal structure from structure confined to a single block.

The central construction is to give each shared latent component the same subject scores in every modality, but allow its feature loadings to differ by modality. Give each modality-specific component its own subject scores and loadings, and include a separate noise term. The generated scores and loadings are known by construction. Retaining them makes it possible to compare a method's estimated structure with the precise signal that was simulated.

A Generative Model for Multiple Modalities

Suppose \(n\) subjects are measured on \(M\) modalities. Let \(X^{(m)}\) be the \(n \times p_m\) data matrix for modality \(m\), with the same subjects in the same row order across modalities. Use \(K_s\) shared latent components and \(K_m\) modality-specific components for modality \(m\). Define \(Z\) as the \(n \times K_s\) matrix of shared subject scores, \(U^{(m)}\) as the \(n \times K_m\) matrix of modality-specific scores, \(A^{(m)}\) as the \(p_m \times K_s\) shared feature loadings, and \(B^{(m)}\) as the \(p_m \times K_m\) modality-specific feature loadings.

Here \(E^{(m)}\) is the \(n \times p_m\) noise matrix. For a transparent baseline model, assume scores have mean zero and covariance identity: each score component has variance one, and distinct components have zero covariance. Assume the shared and modality-specific scores are uncorrelated with each other. Assume noise has mean zero, is uncorrelated with the scores, and is independent across modalities. For the covariance derivation below, assume noise features have covariance \(\Psi^{(m)}\) within modality \(m\); independent feature noise with variances \(\sigma^2_{jm}\) is one special case. These are modeling choices, not properties guaranteed by multi-omics data.

Under these assumptions, the mean signal is zero. A nonzero feature mean or a different noise distribution can be added when needed, but the decomposition above focuses on the latent signals. In particular, a shared score does not mean that two modalities have identical measurements: its loadings, and therefore its contribution to each feature, may differ across modalities.

What the Model Implies About Covariance

The model's covariance structure helps explain the meaning of the components. For a single subject, write its modality-\(m\) row as \(x^{(m)}\). Under the stated score assumptions, its within-modality covariance is

The shared scores contribute \(A^{(m)}(A^{(m)})^T\), the modality-specific scores contribute \(B^{(m)}(B^{(m)})^T\), and the noise contributes \(\Psi^{(m)}\). For two distinct modalities \(m\) and \(\ell\), the cross-covariance is

The modality-specific terms do not appear in this expression because their scores are uncorrelated with the scores in the other block, and the noise is independent across modalities. The cross-covariance is thus generated by the shared scores, though a particular entry can be zero or negative depending on the loadings.

For the within-modality result, expand the covariance of \(Z(A^{(m)})^T + U^{(m)}(B^{(m)})^T + E^{(m)}\). The covariance of the first term is \(A^{(m)}\operatorname{Cov}(Z_i)(A^{(m)})^T=A^{(m)}(A^{(m)})^T\), since \(\operatorname{Cov}(Z_i)=I\). The second term gives \(B^{(m)}(B^{(m)})^T\). All cross-terms vanish because the scores are uncorrelated with one another and with the noise. Adding the noise covariance gives the stated result. For two modalities, the same expansion leaves only the shared-score term: its covariance contribution is \(A^{(m)}\operatorname{Cov}(Z_i)(A^{(\ell)})^T=A^{(m)}(A^{(\ell)})^T\).

Building a Mixed Signal Simulation

To build a mixed simulation, choose the number of shared and modality-specific components, generate their subject scores, specify the loading matrices, and then add noise. The construction should preserve the identity of the subjects across modalities: row \(i\) in every \(X^{(m)}\) must use row \(i\) of \(Z\). A practical sequence is:

Known Truth, Identifiability, and Checks

A simulation only provides a useful recovery target if its truth is unambiguous and preserved. Store the generated score matrices, loading matrices, component-wise signal matrices, noise matrices, and final data matrices. If the data are centered or transformed after generation, record that operation as well: otherwise, the reference loadings and scores may no longer describe the analyzed data on the same scale.

Latent factor representations have sign and ordering ambiguities. For example, replacing one score column \(z\) by \(-z\) and its loading column \(a\) by \(-a\) leaves \(za^T\) unchanged. Components can also be reordered without changing the signal. In some models, rotations within a multi-component subspace can produce equivalent fitted signals. In a simulation, the generated arrays define the chosen reference representation, but a comparison with an estimated representation may need to account for sign, ordering, or rotation before judging component-level agreement. Fixing score variance to one and recording the exact generated parameters makes the convention clear; it does not by itself guarantee every fitted representation is uniquely identified.

Basic diagnostics can catch construction errors before a benchmark is run. Check that all modality matrices have the intended subject count and row ordering; that shared score columns are exactly the same across blocks; that modality-specific score matrices are distinct; and that dimensions in every matrix product agree. Also compare empirical score means, variances, and cross-correlations with their targets. With a finite simulated sample, empirical values need not equal the population targets exactly, so inspect deviations rather than expecting exact equality unless the construction explicitly enforces it.

A final structural check is to inspect the generated signal without noise. If a modality is meant to have no shared component, its shared loading matrix should be zero; if it is meant to have no modality-specific component, its specific loading matrix should be zero. A zero score matrix or zero loading matrix removes that component. For a known loading vector \(a\), one simple truth-based comparison is a sign-aligned squared loading error. If the true vector is \(a=(2,1)^T\) and an estimate after sign alignment is \(\hat a=(1.8,1.1)^T\), then the squared error is \((1.8-2)^2+(1.1-1)^2=0.04+0.01=0.05\). This arithmetic illustrates why retaining the numerical truth matters; the choice of a full recovery score depends on the benchmark question.

Alternatives, Limitations, and a Common Pitfall

A common mistake is to call a signal “shared” because two modalities have similar-looking loading vectors, or because their features are correlated. Similar loadings alone do not create shared subject-level variation if the scores are independent. Under this model, the defining feature is that the same subject score enters both modalities. Conversely, a shared score can produce different feature patterns, including different signs, through its modality-specific loadings.

The model is deliberately structured and therefore has limitations. Its clean covariance interpretation depends on the assumptions about scores and noise; correlated modality-specific scores or cross-modal noise would also induce cross-modal covariance. Fixed loadings and a chosen number of factors may not represent real biological complexity. The model also does not, by itself, specify a realistic marginal distribution for every feature, a sparse effect pattern, or a particular outcome mechanism. Those are distinct design choices. A sound benchmark states which features of reality this construction is intended to isolate and does not treat one signal architecture as universally representative.