Posterior Predictive Checks for Multi-Omics Clusters

Why Check Replicated Multi-Omics Data?

A fitted clustering model summarizes a complicated joint data set using cluster assignments and cluster-specific profiles. A useful question is whether data generated by that fitted model would resemble the data actually observed. Posterior predictive checks answer this by repeatedly generating replicated data from the model, with uncertainty in its parameters and latent cluster assignments carried through the simulation.

For multi-omics data, resemblance must be assessed across blocks, not only feature by feature. A model may reproduce the distribution of RNA measurements within each cluster while failing to reproduce a cross-block pattern, such as the relationship between an RNA feature and a protein feature. A check is only informative when the summaries are chosen to reflect features that matter for the scientific and modeling goals.

In a shared-partition model, the same \(z_i\) informs the distribution of sample \(i\)'s measurements in every block. Conditional on \(z\) and the model parameters, the blocks may be modeled as independent, or the likelihood may specify additional dependence. The replication step must use the likelihood actually fitted; it should not impose a dependence structure simply because the observed blocks are correlated.

Generating Replicates and Choosing Discrepancies

Write \(T(Y,z)\) for a discrepancy statistic: a numerical summary of a feature to be checked. It may describe a within-cluster distribution, a cross-block pattern, or another feature implied by the model. For posterior draw \(m\), draw \((z^{(m)},\theta^{(m)})\) from the posterior and then draw \(Y^{\mathrm{rep},(m)}\) from the model conditional on that draw. Compare \(T(Y,z^{(m)})\) with \(T(Y^{\mathrm{rep},(m)},z^{(m)})\), using the same sampled assignments for both summaries when the check is conditional on the existing partition.

Here \(M\) is the number of posterior predictive replicates and \(I\{\cdot\}\) equals 1 when the stated comparison holds and 0 otherwise. This tail proportion describes how often the replicated discrepancy is at least as large as the observed discrepancy, under the chosen statistic and fitted model. It is a diagnostic summary, not a classical p-value with a uniform null distribution. Posterior predictive checks reuse the data to estimate the posterior and to calculate the observed discrepancy, so their tail proportions are often conservative and should not be read as calibrated tests of a null hypothesis.

The comparison can be two-sided in spirit when unusually small as well as unusually large values would reveal misfit. For example, inspect a replicated distribution or compare both tails rather than relying on the upper-tail proportion alone. The discrepancy should be selected before interpreting the result and should have a direct interpretation in the units or structure of interest.

The posterior predictive distribution is obtained by averaging the sampling distribution over the posterior:

This identity follows from the law of total probability: conditional on \(z,\theta\), the replicate has distribution \(p(Y^{\mathrm{rep}}\mid z,\theta)\); averaging that conditional distribution over the posterior gives the unconditional predictive distribution given the observed data. In computation, posterior draws approximate this average. Each replicate must use a coherent joint draw: independently combining a cluster assignment from one draw with parameters from another generally does not represent the fitted posterior.

Worked Examples

Assumptions, Diagnostics, and Alternatives

A posterior predictive check is meaningful only if replication follows the fitted data-generating specification. In particular, the likelihood must suit the measurement type and support in each block; the replicated data must use the same relevant design, offsets, and exposure structure as the observed data; and missingness or preprocessing must be handled consistently with the model. If a block was normalized or transformed before fitting, the check should be performed on that modeled scale, or the simulation should be transformed back appropriately before comparing with raw-scale summaries.

For a within-cluster check, the assignment used to identify a cluster must be drawn from the posterior and applied consistently to the observed and replicated summaries. Mixture labels are arbitrary: cluster 1 in one draw need not correspond to cluster 1 in another. Use label-invariant summaries, such as the collection of cluster-specific rates, or compare within each draw using its own assignments. Do not average parameters by raw component label across draws unless labels have been aligned appropriately.

Useful diagnostics include replicated histograms or empirical distributions for continuous features, replicated category proportions for categorical features, and summaries of cross-block associations. Inspect spread and shape, not only a single tail proportion. A systematic shift, excess tail mass, too little dispersion, or failure to reproduce cross-block patterns can point to model misspecification. Conversely, a reasonable check does not prove that the model is correct: the chosen summaries may simply fail to detect its shortcomings.

A common mistake is to generate each omics block using an independently sampled cluster assignment. That breaks the shared-partition structure and answers a different question: the fitted model says a sample’s blocks share its assignment. The correction is to draw one \(z_i\) per sample for each replicate and use it across all blocks, while drawing block-specific measurements according to their own likelihoods. Another mistake is to treat a tail proportion near 0.5 as proof of fit, or a value near 0 or 1 as a formal rejection. These values are descriptive evidence conditional on the chosen statistic and model, not calibrated decision thresholds.

Posterior predictive checks differ from prior predictive checks, which simulate from the prior before conditioning on the observed data. They also differ from a held-out predictive assessment, which evaluates prediction on data not used to fit the model. A residual-style diagnostic can focus on a particular discrepancy, while a posterior predictive check provides a model-based reference distribution for that discrepancy. These approaches are complementary; none removes the need to choose relevant features and interpret them in context.

A further limitation is that the data influence both the posterior and the observed discrepancy, which can make a model look less discrepant than it would under an independent test. Checks are also sensitive to sample size, feature selection, and the number of replicates: with \(M\) simulations, the estimated tail proportion changes in increments of \(1/M\). Use enough replicates for the resolution needed, and report the statistic, replication scheme, and number of simulations so the diagnostic can be interpreted.