Simulating Redundant and Correlated Omics Signals

Why Simulate Redundant Signals?

Two omics blocks can each predict an outcome because they measure much of the same underlying biology. Combining them may improve prediction by averaging measurement noise, but the benefit should shrink when the blocks are nearly duplicates. At the same time, correlated features can make a fitted model unstable: small changes in the training sample may substantially change its estimated coefficients even when its predictions change little.

A useful simulation therefore needs to control both what is shared and how strongly the blocks are correlated. The construction below creates two standardized block scores that contain a shared outcome-relevant signal and separate noise. It is deliberately simpler than a full assay model: its purpose is to make the source and amount of redundancy explicit before adding realistic features.

A Basic Generative Construction

For each simulated subject \(i\), generate mutually independent standard normal variables \(U_i\), \(E_{i1}\), and \(E_{i2}\). Let \(U_i\) be the shared biological signal, and let \(E_{i1}\) and \(E_{i2}\) represent block-specific variation. For a chosen \(q\) between 0 and 1, define the two block scores and a continuous outcome by

where \(\varepsilon_i\) is independent of the three latent variables, has mean 0 and variance \(\sigma^2\), and \(\beta\) controls the outcome effect of the shared signal. Each score has mean 0 and variance 1. The parameter \(q\) controls both the proportion of score variance due to the shared signal and the correlation between the two scores:

Thus \(q=0\) gives uncorrelated scores that contain no information about \(U_i\), while \(q=1\) makes the two scores identical to \(U_i\). For intermediate values, both blocks carry the same predictive component but also contain distinct noise. The assumptions are explicit: the latent variables are independent standard normals, the outcome noise is independent and homoscedastic, and the same data-generating parameters apply across subjects. These assumptions provide a tractable benchmark, not a claim that real omics measurements are normally distributed or generated by a single factor.

Expected Predictive Performance

The population coefficient of determination \(R^2\) for the best linear prediction of \(Y_i\) from a single score is the squared correlation between that score and the outcome. Independence and the variance assumptions above give \(\operatorname{Cov}(B_{ij},Y_i)=\beta\sqrt q\) and \(\operatorname{Var}(Y_i)=\beta^2+\sigma^2\). Therefore,

Now consider combining the blocks by averaging their scores, \(\bar B_i=(B_{i1}+B_{i2})/2\). This is a simple prediction-level integration rule; it is not a general prescription for combining raw features. Its covariance with \(Y_i\) remains \(\beta\sqrt q\), while its variance is \((1+q)/2\). The resulting population \(R^2\) is

The difference is

The inequality follows because \(0\leq q\leq1\), \(\beta^2\geq0\), and \(\beta^2+\sigma^2>0\). In this particular model, averaging cannot reduce population \(R^2\): the shared signal is retained while independent block-specific noise is reduced. The gain is zero at \(q=0\) and \(q=1\), and positive for \(0<q<1\) when \(\beta\ne0\). It is important not to generalize this result to arbitrary fitted integration methods, finite samples, or blocks with bias and unequal signal quality.

Making Features Within Each Block

To simulate a block with multiple measured features, treat \(B_{ij}\) as a block-level latent score and generate features conditionally on it. For feature \(k\) in block \(j\), one simple standardized construction is

where each \(\eta_{ijk}\) has mean 0 and variance 1 and is independent of the block scores and other feature errors. Then each \(X_{ijk}\) has variance 1, and \(\lambda_{jk}\) controls how strongly that feature measures its block score. For a feature in block 1 and a feature in block 2, their correlation is \(q\lambda_{1k}\lambda_{2\ell}\) under these assumptions. In a real benchmark, the model may instead use correlated within-block errors or non-normal marginal distributions, but those changes should be introduced deliberately rather than mistaken for shared predictive signal.

Designing and Checking a Simulation

A practical simulation should make the estimand and data-generating assumptions clear before methods are compared. A useful sequence is:

The assumptions behind the analytic calculations should also be checked in each simulated replicate when those calculations are used as a benchmark. For instance, in the basic model, an empirical block correlation noticeably different from \(q\) may reflect ordinary sampling variation, but it can also reveal a coding error, an unintended dependence among noise terms, or a mistaken scaling factor. Likewise, check that outcome noise was generated independently and that the empirical variance of \(Y_i\) is consistent with \(\beta^2+\sigma^2\).

Compare methods on the same held-out subjects and report the variability of performance across replicates, not only its average. A one-block model gives a baseline. Concatenating features allows a model to use both blocks jointly but can create unstable coefficient estimates when features are highly correlated. Averaging block-specific predictions is a late-integration alternative; a shared representation or a fitted joint model uses a different rule for combining information. These approaches need not have identical flexibility or tuning effort, so a performance difference alone does not isolate the effect of integration unless their comparison is appropriately controlled.

Diagnostics, Pitfalls, and Limits

Useful checks include empirical correlations between block scores, correlations between each block and the outcome, the variance of each generated feature, and the frequency with which a fitted method fails or produces extreme predictions. For fitted models, examine whether coefficient estimates vary greatly across replicates even when held-out prediction is fairly stable. That pattern is consistent with collinearity: individual coefficients can be weakly identified while a combined predictive direction is more stable. If held-out performance deteriorates as \(q\) rises, the cause is not established by correlation alone; check whether the method is overfitting, whether the comparison is fair, and whether the simulated mechanism actually matches the assumptions used to interpret the result.

A common mistake is to treat a high correlation between blocks as proof that they contain redundant predictive information. The extended example shows why that reasoning fails: shared nuisance variation can increase block correlation without predicting \(Y_i\). The correction is to inspect the data-generating equations and measure each block's relationship to the outcome, not to infer predictive overlap from block correlation alone.

The basic construction is intentionally limited. It assumes one shared latent score, linear outcome dependence, independent residuals, and equal standardized block quality. It does not represent every feature of molecular data, and its formula for the benefit of averaging does not apply automatically to arbitrary predictors. More complex outcome mechanisms or other sources of dependence can be studied, but changing several parts of the design at once makes results harder to interpret. Begin with a known data-generating mechanism, verify its implications, and add complexity in controlled steps.