Generating Partially Observed Modalities

Why Simulate Missing Modalities?

A multi-omics benchmark that gives every method a complete matrix tests performance under a special data condition. Real analyses can instead encounter a modality unavailable for an entire participant, or individual features absent within an otherwise measured modality. These patterns change both the information available to an integration method and the composition of the samples it can use. A useful simulation therefore specifies not only how complete data are generated, but also how they become partially observed.

The central implementation rule is simple but consequential: define an indicator for missingness or for observation, state which value means what, and keep that convention consistent. Here, an observation indicator equals one when the value is available. Missingness is generated separately, then converted to that indicator.

A modality-level indicator masks all features in that modality for a subject. A feature-level indicator can mask just one value. If both mechanisms are used, the final availability indicator is \(R_{imj}=R_{im}R^{(F)}_{imj}\), where \(R^{(F)}_{imj}\) is the feature-level indicator. The product is one only if the modality and the particular feature are both available.

Generate the complete data first, using the cohort, batch, marginal-distribution, and cross-omics dependence choices already set for the simulation. Then generate missingness from those values and any specified covariates. This sequence makes the target complete data and the masking process explicit. The simulation can retain the complete values and generating variables for evaluation, but those hidden values must not be passed to an analysis as though they were observed.

Three Mechanisms and Their Assumptions

A missingness mechanism describes how observation indicators are generated. Under missing completely at random (MCAR), the probability of missingness does not depend on measured covariates or data values. Under covariate-dependent missingness, it depends on specified covariates, such as cohort or batch. When those covariates are observed, this is often described as missing at random (MAR): conditional on observed information, missingness does not further depend on the unobserved value itself. Under signal-dependent missingness, the probability depends on the underlying measurement value, including a value that may be hidden. Relative to the observed data, this is generally missing not at random (MNAR).

These labels concern the variables used to generate the indicators, not whether the observed missingness rates happen to look similar across groups. Covariate-dependent missingness is only appropriately called MAR if the relevant covariates are available to the analysis. A signal-dependent mechanism may be MAR if the signal used to determine missingness is itself fully observed as auxiliary information; classification is relative to what the analyst observes.

For MCAR, \(q\) is a fixed probability for the relevant modality or feature. For covariate-dependent missingness, a common specification is \(q_i=\operatorname{logit}^{-1}(\beta_0+\beta_1 Z_i)\), where \(Z_i\) is a specified covariate and \(\operatorname{logit}^{-1}(t)=1/(1+\exp(-t))\). For signal-dependent missingness, a feature-level probability might instead be \(q_{imj}=\operatorname{logit}^{-1}(\gamma_0+\gamma_1 X_{imj})\). The coefficients determine probabilities, not deterministic thresholds: an individual value with higher \(q\) is more likely to be missing, but is not guaranteed to be missing.

These basic specifications assume indicators are independent draws conditional on the inputs in their probability model. That is a modeling choice, not a universal property of missing data. If missingness is intended to occur in blocks, across related features, or in coordinated patterns across modalities, the simulation must encode that dependence rather than independently drawing every indicator.

Expected Rates and Combining Levels

Expected missingness is a useful check on a simulation, but it is not a promise about the realized rate in a finite replicate. Let \(D_k\) be the missingness indicator for each of \(N\) potential records, with missingness probability \(q_k\), which may vary by record. By linearity of expectation, the expected number missing is the sum of their probabilities. This statement does not require the indicators to be independent.

The second equality follows because \(E(D_k)=1q_k+0(1-q_k)=q_k\), and expectation distributes over sums. Under a fixed MCAR probability \(q\), the expected missing proportion is therefore \(q\). With varying covariate-dependent probabilities, it is the average of the participant-specific probabilities, not necessarily the probability at the average covariate value.

If modality-level and feature-level masking are drawn independently conditional on their specified inputs, the probability a particular feature is observed is the product of the probabilities of passing both masks. If either mask is dependent on the other, this product formula need not hold; use the joint probability implied by the actual generation rule.

Signal-Dependent Feature Missingness

Signal-dependent masking can represent a measurement process in which low or high abundance is less likely to be detected, for example. It is important to generate this mechanism using the complete value \(X_{imj}\), because that value is known to the simulator even if it will later be hidden. Specify the direction and scale deliberately: a positive slope in a logistic model increases missingness probability as the signal increases, while a negative slope decreases it.

Implementation Checks, Alternatives, and Limits

Keep a record of the mechanism alongside each simulation scenario. At minimum, record the indicator level (modality or feature), the meaning of indicator values, the probability model and its parameters, the variables used to calculate each probability, whether draws are shared or independent, and the random draws or reproducible seed information used by the simulation. The earlier material on reporting analysis decisions and reproducible materials is relevant to making these choices auditable.

After generation, check both the input process and its result. Compare realized missing counts with the probabilities used to generate them; verify that modality-level missingness masks all and only the intended features; and confirm that observed values were not altered. For covariate-dependent and signal-dependent mechanisms, tabulate missingness by the generating covariate or signal range. A coding error can produce plausible overall rates while reversing the intended group pattern.

A frequent pitfall is to use \(q\), the missingness probability, as the probability that the observation indicator equals one. That reverses the intended meaning: if \(q=0.20\), it would make only 20% of records observed. The correction is to generate a missingness indicator \(D=1\) with probability \(q\), then set \(R=1-D\), so that 80% are observed on average. Always test a small hand-worked case before scaling the code to many modalities and features.

Another limitation is that observed-data diagnostics cannot generally identify whether missingness depends on an unobserved signal. A simulated benchmark can know the true mechanism because it generated the complete values, but an analyst receiving only the masked data does not have that same information. Simulation assumptions also do not establish that a corresponding real data set follows the same mechanism. The purpose is to vary and report plausible mechanisms, not to claim that one masking rule is universally realistic.