Bayesian Generative Models for Joint Multi-Omics Clustering

Why a Generative Model for Joint Clustering?

A multi-omics clustering analysis asks whether samples can be grouped according to patterns across several measured blocks, such as RNA and protein profiles. A Bayesian generative model makes the assumptions behind that grouping explicit: it describes how a cluster is selected for each sample and how observations from each block arise conditional on that cluster. This matters because the resulting clusters depend not only on the observed measurements, but also on the model’s representation of their joint behavior.

Here the basic model has a fixed number \(K\) of clusters and \(B\) measured blocks for each of \(n\) matched samples. The focus is the model’s structure and probability calculations, not a particular choice of likelihood or prior family.

Model Specification and Assumptions

The basic generative construction has three stages. First, draw the shared cluster weights and cluster-specific parameters from a prior \(p(\boldsymbol{\pi},\boldsymbol{\theta})\), where \(\boldsymbol{\theta}\) collects all \(\theta_{kb}\). Second, draw a cluster label for each sample, \(z_i\mid\boldsymbol{\pi}\sim\operatorname{Categorical}(\boldsymbol{\pi})\). Third, generate each block observation for that sample from the distribution associated with its label. This formulation assumes that all blocks for sample \(i\) use the same \(z_i\).

For the simple likelihood factorization used here, assume that samples are independent conditional on the model parameters and their labels, and that the blocks are independent conditional on a sample’s label and the parameters. Thus, for a fixed label \(z_i=k\), the contribution of sample \(i\) is the product of its block-specific contributions. These are modeling assumptions, not consequences of having collected several data types. Real blocks can retain dependence after conditioning on cluster, so the adequacy of this factorization must be assessed rather than presumed. The earlier material on factor models offers a different way to represent cross-block structure; it does not make this mixture assumption automatic.

A prior is needed for the unknown weights and profile parameters. A proper joint prior is a straightforward choice for this formulation; it need not factor into independent priors for \(\boldsymbol{\pi}\) and \(\boldsymbol{\theta}\). The posterior is defined when the observed-data normalizing probability or density is finite and positive. The model below treats \(K\) as fixed; it does not specify a prior on the number of clusters.

The distinction among these expressions is essential. Given \(\mathbf{z}\), the conditional likelihood for \(\mathbf{Y}\) contains block likelihoods, but it does not contain the probabilities of the labels. The label probabilities belong to \(p(\mathbf{z}\mid\boldsymbol{\pi})\). Their product with the conditional likelihood is the complete-data joint likelihood.

Marginalizing the Unknown Labels

The labels are unobserved. To find the likelihood of the data for fixed parameters, sum the complete-data joint likelihood over every possible label vector. Because the assignments are independent across samples conditional on \(\boldsymbol{\pi}\), this sum can be rearranged into a product of sample-specific sums.

For the last equality, each factor in the product depends only on the corresponding \(z_i\). Distributing the sums over \(z_1,\ldots,z_n\) therefore gives one sum for each sample. This is a marginal likelihood conditional on fixed parameters: the uncertainty about each label has been summed out, but uncertainty about \(\boldsymbol{\pi}\) and \(\boldsymbol{\theta}\) has not.

The denominator sums the unnormalized contributions over all possible clusters, so the probabilities are nonnegative and sum to one. These probabilities condition on fixed parameters. In a full Bayesian analysis, the posterior for a label also reflects uncertainty about the parameters by averaging over their posterior distribution.

Bayesian Inference Also Averages Over Parameters

Writing a Bayesian model requires specifying a prior as well as a likelihood. With the joint prior \(p(\boldsymbol{\pi},\boldsymbol{\theta})\), the posterior distribution is proportional to the prior times the complete-data joint likelihood if labels are included, or to the prior times the label-marginalized likelihood if they are not. The marginal evidence sums over labels and integrates over parameters:

The integral notation includes a sum for discrete parameters where needed. This marginalization is why a Bayesian posterior is not generally obtained by inserting a single estimated parameter value into the fixed-parameter label formula. The prior and likelihood together determine how uncertainty in weights and profiles affects the result.

Diagnostics, Alternatives, and Limitations

The factorization is useful because it clarifies how evidence from several blocks contributes to a cluster assignment: each block supplies a likelihood term, and the cluster weight supplies a prior contribution to the label. But a product can give a misleadingly strong impression of evidence if blocks remain dependent after conditioning on cluster, or if one block’s likelihood is poorly calibrated. The product formula is justified only under the stated model, not merely because the blocks are recorded in separate files.

Useful checks include inspecting whether the model reproduces block-specific patterns and cross-block relationships, checking for samples with extremely concentrated or nearly uniform assignment probabilities, and assessing sensitivity to plausible changes in prior and likelihood assumptions. An assignment probability near one is not proof that the sample has a uniquely correct biological class; it may instead reflect a restrictive model, misspecified likelihood, or overly influential block.

The basic formulation also has limits. It assumes a fixed \(K\), represents within-cluster block distributions through the chosen \(g_b\), and does not automatically handle missing blocks or residual dependence. A cluster-specific profile can summarize a group without establishing that its members share a causal mechanism. The model is a probabilistic description of observed data, not by itself a causal analysis.

Common Mistakes

• Putting label probabilities in the conditional likelihood. The expression \(p(\mathbf{Y}\mid\mathbf{z},\boldsymbol{\theta})\) does not include \(\pi_{z_i}\). Those factors belong to \(p(\mathbf{z}\mid\boldsymbol{\pi})\), or to the complete-data joint likelihood.
• Using a sum where a product is required across blocks. Under the stated conditional independence model, a sample’s block likelihoods multiply within each cluster; the sum is over the alternative clusters.
• Calling a fixed-parameter probability a full Bayesian posterior. \(P(z_i=k\mid\mathbf{Y},\boldsymbol{\pi},\boldsymbol{\theta})\) conditions on parameter values. A posterior that accounts for parameter uncertainty averages over their posterior distribution.
• Reading model assumptions as empirical facts. Conditional independence and a shared label across blocks must be defended and checked; they do not follow from the data format.