Simulating Realistic Omics Noise Distributions

Why Noise Distributions Matter

A benchmark can give an integration strategy an artificial advantage if it supplies data that fit one measurement model unusually well. Gaussian noise is convenient, but omics measurements may instead be nonnegative counts with mean-dependent variability, or continuous measurements with occasional extreme deviations. These differences affect feature scales, outliers, and the information retained by transformations. They can therefore affect how early, intermediate, and late integration strategies behave, even when the underlying biological signal is unchanged.

The goal of a controlled simulation is not to claim that one distribution describes every assay. It is to specify plausible alternatives, hold other design features fixed, and see whether conclusions about an integration strategy persist. In particular, a comparison of noise distributions should not accidentally compare a low-noise scenario with a high-noise one.

A Conditional Data-Generating Setup

Let \(x_{ijm}\) denote a continuous measurement for subject \(i\), feature \(j\), and omics block \(m\). Write its noise-free value as \(\mu_{jm}+\eta_{ijm}\), where \(\mu_{jm}\) is a feature baseline and \(\eta_{ijm}\) is the signal specified by the simulation design. For this tutorial, treat these values as fixed when generating measurement noise. This separates the noise mechanism from the signal structure.

A simple reference model assumes that, conditional on the noise-free values, errors are independent and identically distributed as \(N(0,\sigma^2)\). This entails zero-mean errors, a common variance, and Gaussian tails. Independence is a conditional simulation assumption, not a consequence of using a normal distribution. If the intended setting has correlated errors across features or subjects, that dependence must be simulated explicitly.

The Gaussian model is useful as a baseline because \(\sigma^2\) directly controls the variance around the specified signal. It is not automatically appropriate for every block. For example, a count assay has integer-valued observations and typically has variance related to its mean. A continuous assay may have more extreme deviations than a normal distribution allows.

Count-Like Noise and Overdispersion

For counts, a common model is the negative binomial parameterized by its mean \(\lambda_{ijm}\) and dispersion \(\kappa_{jm}>0\). Under this parameterization, the conditional mean is \(\lambda_{ijm}\) and the conditional variance is \(\lambda_{ijm}+\lambda_{ijm}^2/\kappa_{jm}\). The extra quadratic term permits more variation than a Poisson model, whose variance equals its mean.

One way to motivate this model is a Poisson variable with a random rate. Let \(\Lambda\) have a gamma distribution with shape \(\kappa\) and rate \(\kappa/\lambda\), so that \(E(\Lambda)=\lambda\) and \(\operatorname{Var}(\Lambda)=\lambda^2/\kappa\). Conditional on \(\Lambda\), let \(Y\mid\Lambda\sim\operatorname{Poisson}(\Lambda)\). The law of total expectation gives \(E(Y)=E\{E(Y\mid\Lambda)\}=E(\Lambda)=\lambda\). The law of total variance gives

For simulations with differing sequencing depths, let \(D_i\) be the library-size offset and define \(\log(\lambda_{ijm})=\log(D_i)+\mu_{jm}+\eta_{ijm}\). The offset changes the expected count proportionally to depth; it does not remove all depth-related variability. The log-scale signal is then connected to counts through the mean, rather than added as an unconstrained Gaussian error to a count.

Heavy-Tailed Continuous Noise

A Student \(t\) distribution can represent continuous errors with heavier tails than a Gaussian distribution. If \(T\sim t_\nu\), then for \(\nu>2\), \(E(T)=0\) and \(\operatorname{Var}(T)=\nu/(\nu-2)\). Thus \(T/\sqrt{\nu/(\nu-2)}\) has mean zero and variance one. Multiplying this standardized variable by \(\sigma\) creates heavy-tailed noise with variance \(\sigma^2\), allowing a direct comparison with Gaussian noise at the same variance.

A further choice is whether errors are independent across features. Independent heavy-tailed errors model isolated extreme measurements. A shared sample-level multiplier or a correlated error vector instead produces groups of features that move together. Those are different simulation scenarios and should not be conflated with changing only the marginal tail shape.

Designing a Fair Noise Comparison

To attribute performance differences to noise shape, define the same signal values before generating each alternative noise distribution. For continuous errors, use the same target variance where meaningful. For counts, state the expected mean, depth offset, and dispersion; count variance changes with the mean, so “same variance” cannot generally be imposed independently of the count mean. Record whether features are simulated independently, whether their noise is correlated, and whether any block-specific scaling is applied.

It can help to use paired simulation replicates: within a replicate, hold the signal configuration and other design choices fixed while changing the noise generator. Repeated independent replicates are still needed to assess variation in benchmark results. Avoid silently changing signal-to-noise ratio by altering the signal, noise variance, feature scales, or number of extreme values at the same time as the distribution family.

Transformations are part of the scenario, not a neutral repair. Applying \(\log(1+Y)\) to counts compresses large values and changes both the mean and variance relationships. It may be appropriate for an analysis pipeline, but it does not make the transformed data identical to Gaussian measurements. Likewise, scaling each feature to unit sample variance can conceal designed differences in variance and should be estimated using training data only when evaluating predictive strategies.

Diagnostics and Failure Modes

In a simulation, the noise-free values are known, so residuals can be calculated as observed value minus noise-free value for continuous measurements. For counts, compare observed counts with their specified conditional means and variances, preferably across many observations or replicates. Useful checks include residual means, variance by fitted mean, tail quantiles, extreme-value frequencies, zero proportions, and plots of residuals against depth or feature intensity.

• Gaussian errors: check for mean shifts, changing variance, skewness, tail excess, and residual correlation. A histogram alone is weak evidence, especially with few observations.
• Counts: check that generated values are nonnegative integers and that average variability increases as implied by the mean-dispersion specification. Inspect zeros and depth patterns; an inappropriate offset can create systematic differences between samples.
• Heavy tails: compare tail quantiles or exceedance rates with the chosen \(t\) model. A handful of extreme values can dominate means, variances, and early concatenated feature representations.

A frequent failure is to standardize each simulated feature until all distributions appear comparable, then conclude that their noise is equivalent. Equal variance does not imply equal tail probability, skewness, discreteness, or mean-variance dependence. Another failure is to generate counts with a Gaussian additive error and round negative results to zero. That procedure changes the distribution in a way not described by either the original Gaussian model or a standard count model.

Implications for Integration Choices

Early integration places measurements from multiple blocks into a shared representation, so scale and distribution can affect which features dominate. A count block and a continuous block may require different preprocessing before concatenation; otherwise, numerical range can be mistaken for relative information. Yet aggressive transformations or standardization can also alter meaningful block differences.

Intermediate integration can preserve block-specific modelling or transformations before combining summaries. This can accommodate different measurement scales, but the summaries still depend on suitable within-block assumptions. Late integration can combine block-level predictions or evidence without directly concatenating all raw features; it does not eliminate misspecification if each block model handles counts or outliers poorly. No integration timing is universally robust to all noise distributions.

These comparisons have limits. A marginal noise distribution alone does not represent batch effects, missingness, feature correlation, or all biological variability. Synthetic scenarios are controlled abstractions, and conclusions should be reported as conditional on the chosen data-generating assumptions. The most informative benchmark varies plausible noise settings and checks whether conclusions remain stable, rather than declaring one noise distribution universally realistic.