Dirichlet Process Mixtures for Integrative Clustering

Why Use an Unbounded Mixture?

In integrative clustering, each sample contributes measurements from several omics blocks, but the number of biologically meaningful groups may not be known in advance. A finite mixture requires a chosen finite set of components. A Dirichlet process mixture instead provides a prior over allocations that can create new latent tables as samples are considered, without fixing a maximum table count in advance. This flexibility does not remove the need to choose a model for the measurements: each table still has parameters governing its multi-omics profiles, and the likelihood assumptions determine how evidence from the blocks is combined.

The phrase unbounded number of possible clusters needs care. A finite sample can occupy only finitely many tables. Every finite number of tables up to the sample size has positive prior probability under the Chinese restaurant process. Whether different tables have distinct parameter values depends on the base distribution: if it is nonatomic, they are distinct with probability one; if it has finite atomic support, different tables can draw the same value, and the number of distinct parameter-value clusters is bounded by that support.

Dirichlet Processes and the Restaurant Allocation Rule

For a sample \(i=1,\ldots,n\), let \(\theta_i\) denote its cluster parameter, with \(\theta_i\mid G\sim G\). Integrating out \(G\) gives the predictive distribution

Here \(\delta_{\theta_j}\) is a point mass at \(\theta_j\). The predictive distribution has a base-distribution part and a reinforcement part: it can produce a draw from \(G_0\), or repeat a parameter value already observed. A useful representation introduces tables. At each step, a sample joins an existing table or starts a new one; a new table then draws its parameter from \(G_0\). With a nonatomic \(G_0\), a new draw differs from all previous draws with probability one, so table labels and distinct parameter values coincide. With an atomic \(G_0\), a new table can draw a parameter value already used at another table.

Using Omics Evidence Alongside the Allocation Prior

In a shared-allocation model, let \(x_i^{(m)}\) denote the measurements for sample \(i\) in omics block \(m=1,\ldots,M\), and let \(\theta_k\) collect the cluster-specific parameters for table \(k\). Under conditional independence of blocks given the cluster parameters, a joint likelihood has the form

This factorization is a modeling assumption, not a consequence of using a Dirichlet process. It says that, conditional on the relevant cluster parameters, the blocks are independent for a sample. The likelihood for each block must suit its data type and any required preprocessing or exposure terms. For example, a count likelihood and a continuous-data likelihood contribute different predictive densities; neither can be substituted merely because the allocation prior is shared.

When cluster parameters are uncertain, compare allocations using integrated predictive densities. Let \(D_k\) denote the data currently assigned to table \(k\), and let \(p(\theta_k\mid D_k)\) be its parameter posterior. Define \(q_{ik}\) as the predictive density of all blocks for sample \(i\) under table \(k\), integrating over that posterior. Let \(q_{i0}\) be the corresponding prior predictive density for a newly drawn table parameter from \(G_0\). Conditional on the existing data, the allocation weights are proportional to the CRP weights multiplied by these predictive densities:

Normalizing these weights gives the conditional allocation probabilities. This is the mathematical reason that the CRP rule is not, by itself, a complete data-based clustering decision: a table with many members has a larger prior allocation weight, but a candidate sample's measurements can favor another table or a new one.

What the Concentration Parameter Controls

The CRP probability of a new table at step \(i\) is \(\alpha/(\alpha+i-1)\), so increasing \(\alpha\) raises the new-table probability at a fixed step and allocation history. It does not set the number of clusters directly, nor does it guarantee a particular number of occupied tables. For the table count \(K_n\), define an indicator \(I_i\) that equals 1 if sample \(i\) starts a new table. Then \(K_n=\sum_{i=1}^n I_i\), with \(I_1=1\) and, for \(i\ge 2\), \(P(I_i=1)=\alpha/(\alpha+i-1)\). Taking expectations and using linearity gives

This is an expectation under the prior over table allocations, not a prediction that a particular data set must have that many clusters. For every \(1\le K\le n\), the CRP assigns positive probability to a partition into \(K\) tables. For example, one can produce such a partition by starting a new table for the first \(K\) samples, then seating each remaining sample at an existing table; every choice in that sequence has positive probability.

For a partition into \(K\) tables of sizes \(n_1,\ldots,n_K\), the CRP probability of that particular partition is

This exchangeable partition probability depends on the table sizes, not on arbitrary table names. It describes the prior probability of an allocation partition after integrating out \(G\); it does not include the multi-omics likelihood. In a mixture analysis, posterior support for an allocation also depends on how well the cluster-specific profiles explain the observed measurements.

Assumptions, Diagnostics, and Alternatives

A Dirichlet process mixture is appropriate only when its assumptions match the scientific and measurement structure. The shared-allocation version assumes that samples use a common partition across blocks; it also requires suitable block likelihoods, a defensible base distribution for cluster parameters, and a meaningful conditional-independence structure if the joint likelihood is factored across modalities. A Dirichlet process does not correct dependence left unexplained within or between blocks. Nor does it automatically prevent a high-dimensional or poorly scaled block from dominating the joint predictive density.

Useful checks begin with prior predictive simulation: generate table allocations and cluster parameters from the proposed \(\alpha\) and \(G_0\), then examine the implied table counts and profile variability before fitting the observed data. Repeat this across plausible values of \(\alpha\) and reasonable base distributions. After fitting, inspect whether allocation conclusions change substantially across those choices and whether joint predictive contributions appear implausibly concentrated in one modality. These checks diagnose sensitivity and model mismatch; they do not establish that the inferred groups are uniquely true biological classes.

A finite mixture with a prior on the number of clusters instead makes the possible component count explicit and finite in each model. It can be preferable when scientific constraints or prior knowledge support a restricted range. The Dirichlet process is useful when a flexible prior over partitions is wanted without choosing a fixed upper bound on the number of tables. Neither approach eliminates uncertainty about the partition, the profile likelihoods, or the prior. In particular, the DP table count should not be read as a guaranteed estimate of the number of biological subtypes.

• Pitfall: treating \(\alpha\) as the number of clusters. The incorrect reasoning is that \(\alpha=3\) implies three clusters. In fact, \(\alpha\) enters each new-table probability and influences a distribution over table counts; it is not a fixed count.
• Pitfall: equating every new table with a new parameter value. This holds almost surely for nonatomic \(G_0\), but not for an atomic base distribution. Report whether a count refers to latent tables or distinct parameter values.
• Pitfall: using allocation weights without the likelihood. The CRP weights alone express prior predictive allocation probabilities. A data-informed allocation also uses the joint predictive density of the candidate sample.