Posterior Predictive Checks For Multi-Omics Factors

Why Check Replicated Data?

A fitted multi-omics factor model can summarize shared structure while still failing to reproduce a feature of the data that matters scientifically. For example, it may capture average cross-view association but miss unusually strong correlations, skewed feature distributions, or extreme observations. A posterior predictive check asks a direct question: if this fitted model generated another data set, would that data set resemble the observed one in the feature we care about?

The check is only as useful as its target. A comparison of cross-view correlations probes dependence across matched samples; a comparison of feature-wise distributions probes marginal fit; and a comparison of blockwise extremes probes whether the model can generate unusually large values. These are different questions, so they require different summaries. A satisfactory result for one summary does not establish overall model adequacy.

Definition and Replication Scheme

Write \(Y^{(v)}\) for the observed data in omics view \(v\), with \(n\) matched samples and \(p_v\) features. Let \(\theta\) collect model parameters, such as loadings, noise parameters, and covariate effects, and let \(F\) denote latent factor scores. A fitted Bayesian model gives a posterior distribution for \((\theta,F)\) conditional on the observed data. A posterior predictive replicate is generated by drawing from that posterior and then simulating data from the model's observation distribution.

The model's likelihood and dependence assumptions must be stated before interpreting a check. For example, a continuous-data factor model might use

Here \(\mathbf{x}_i\) contains covariates, \(F_i\) contains factors for sample \(i\), and \(\boldsymbol{\lambda}_j^{(v)}\) is the loading vector for feature \(j\) in view \(v\). The distribution of \(\varepsilon_{ij}^{(v)}\), and any dependence among residuals, is part of the model rather than an automatic property of factor analysis. For instance, conditional independence of feature residuals given \((\theta,F)\) is an assumption in some models, not a general rule. Count or binary data require observation distributions appropriate to those data, rather than a Gaussian likelihood by default.

For a check of the observed matched cohort, a common scheme draws \((\theta^{(b)},F^{(b)})\) from \(p(\theta,F\mid Y)\), then generates all views jointly using those draws and the model's observation distribution. Shared factors must be shared across views in each replicate if the fitted model represents shared structure. Holding posterior factor scores fixed while regenerating observations gives a conditional check of the observation model around those scores. To check a new cohort's overall distribution, latent factors should instead be generated from their population model. These schemes address different targets; silently mixing them can make a cross-view check misleading.

Choosing and Interpreting Discrepancies

A discrepancy \(T(Y)\) is a numerical feature of the data selected before examining the check's conclusion. It can be a correlation, a quantile, a dispersion measure, or a maximum. For a one-sided upper-tail check, the posterior predictive tail area is

In computation, generate \(B\) replicates and estimate this quantity by

For a two-sided discrepancy, define a distance from a prespecified reference \(c\), such as the predictive median, and compare \(|T(Y^{\mathrm{rep}})-c|\) with \(|T(Y)-c|\). The reference should be chosen for the scientific question; mechanically doubling a one-sided tail area is not always appropriate. A posterior predictive tail area is a model-checking summary, not a frequentist \(p\)-value and not the probability that the model is true. Since the observed data inform the posterior and also supply the observed discrepancy, these checks can be conservative: some lack of fit may be hard to detect.

Worked Checks

Why the Simulation Estimate Has Error

The posterior predictive distribution averages over uncertainty in parameters and latent factors. This follows directly from conditioning and integration: for any event \(A\) involving a replicate,

Drawing \((\theta,F)\) from the posterior and then drawing \(Y^{\mathrm{rep}}\) from the conditional observation model samples from this mixture. For the tail-area check, each indicator \(I\{T(Y^{\mathrm{rep}})\geq T(Y)\}\) is a Bernoulli variable with success probability \(q_T\). With independent predictive replicates, the Monte Carlo standard error of \(\widehat{q}_T\) is approximately

For the extreme-value example, this is \(\sqrt{0.09(0.91)/100}=0.0286\). Thus the estimate \(0.090\) has noticeable simulation uncertainty; more replicates can reduce that error. If posterior draws are dependent, as in Markov chain Monte Carlo, the effective number of independent draws can be smaller than \(B\), so simulation error should not be assessed as if every draw were independent.

Diagnostics, Alternatives, and Limitations

Use checks that target distinct ways the model could miss important structure. Cross-view correlations can be examined for selected feature pairs or summarized by a prespecified distribution of correlations. Marginal checks can compare distributions or quantiles feature by feature and by view. Extreme-value checks can use maxima within each view or a scientifically defined feature set. For every comparison, compute the observed and replicated statistic using the same rules, including sample inclusion, transformations, and handling of missing values.

Warning signs include observed summaries lying well outside the bulk of their predictive distributions, systematic discrepancies across many features, or observed tails consistently more extreme than simulated tails. Also inspect posterior predictive plots: a single tail area can conceal asymmetric or subgroup-specific mismatch. A check can fail because the likelihood, factor structure, residual dependence, covariate adjustment, or data-processing assumptions are inadequate. It does not identify which component is responsible without further targeted checks.

Posterior predictive checks complement the earlier material on residual diagnostics for factor models. Residual diagnostics focus on discrepancies after fitting; posterior predictive checks ask whether data generated under the fitted model reproduce selected data features, while propagating posterior uncertainty. Neither replaces the other. A held-out predictive assessment instead evaluates prediction on observations not used to fit the model, which addresses generalization more directly but requires a suitable holdout design and a clear predictive target.

Common pitfalls include treating a tail area as a calibrated test of the null hypothesis, checking only one convenient summary, and generating each omics view separately when the model specifies shared latent structure. Another is allowing an observed-data-dependent choice of statistic to go unreported; choosing the most discrepant feature after inspecting all features changes the interpretation of the check. Predefine summaries where possible, or reproduce the selection rule in every replicate. Finally, posterior predictive agreement is not proof of correctness: different models can reproduce the same summaries, and an unexamined feature may still be poorly fit.