Split-Merge Moves for Multimodal Mixture Posteriors

Why Split–Merge Moves Matter

A cluster assignment can be difficult to change when the posterior has several well-separated modes. A single-sample update may move one sample to another cluster, but reaching a substantially different partition can require many coordinated updates. A split move proposes dividing one cluster into two; a merge move proposes joining two clusters. These larger moves can help a sampler explore alternative cluster configurations, but they must account for how likely the proposal is in each direction.

In integrative clustering, each sample has one shared cluster assignment while its measurements may be observed across several omics blocks. Under the conditional-independence model, the blocks contribute jointly to the evidence for a partition. A split that fits one block well but fits another poorly must be judged using the joint posterior target, not a single modality in isolation.

Partitions and the Posterior Target

Write \(X=(X^{(1)},\ldots,X^{(M)})\) for the observed data in \(M\) omics blocks. Let \(\theta\) collect the cluster-specific profile parameters. A collapsed target integrates these parameters out:

Here \(p(\mathcal{C})\) includes the chosen prior on cluster prevalence and, when applicable, the number of clusters. The integrated likelihood includes all omics blocks and their modelled contributions. Conjugate integration can make this target computable without retaining \(\theta\), as in collapsed sampling. We use \(\pi(\mathcal{C}\mid X)\) for the posterior probability of a partition, up to a common normalizing constant.

The basic Metropolis–Hastings rule requires a proposal kernel \(q(\mathcal{C}'\mid\mathcal{C})\): the probability of proposing \(\mathcal{C}'\) from the current partition \(\mathcal{C}\). For a proposed transition, its acceptance probability is:

Constructing Split and Merge Proposals

A simple split proposal selects a cluster and proposes dividing its samples into two nonempty groups. A merge proposal selects two clusters and proposes their union. More elaborate split proposals may choose two seed samples and assign the remaining samples in sequence using probabilities informed by the joint multi-omics data. Whatever the construction, its probability must be included in \(q\). In particular, a split proposal that is more likely to produce some partitions than others cannot be treated as if every partition were equally likely.

One common approach alternates between split and merge proposal types. A move may first select split or merge, then select the cluster or pair of clusters, and then generate the proposed partition. The transition probability \(q(\mathcal{C}'\mid\mathcal{C})\) is the probability of the complete proposal: all these selection and construction steps that lead from \(\mathcal{C}\) to \(\mathcal{C}'\). If multiple proposal paths can generate the same partition, their probabilities must be summed, or the algorithm must use a valid augmented-state construction that accounts for its auxiliary choices.

Why the Acceptance Rule Is Correct

The rule preserves the desired posterior distribution by balancing probability flow between any two partitions. From \(\mathcal{C}\) to \(\mathcal{C}'\), the flow under stationarity is proportional to \(\pi(\mathcal{C}\mid X)q(\mathcal{C}'\mid\mathcal{C})a(\mathcal{C},\mathcal{C}')\). The reverse flow has the corresponding expression with the partitions exchanged. Substituting the acceptance rule makes these flows equal:

This expression is unchanged when \(\mathcal{C}\) and \(\mathcal{C}'\) are exchanged, so detailed balance holds for this pair. The argument relies on the ratio of the actual posterior targets and the actual forward and reverse proposal probabilities. A sampler also needs to be able to reach the relevant partitions through its proposals; detailed balance alone does not guarantee good exploration.

Using All Modalities in the Target

Under conditional independence of omics blocks given the shared partition and their block-specific profiles, the likelihood conditional on parameters has a product form:

The product expresses the model’s conditional-independence assumption; it does not mean that one may assess a proposed split using only the most favorable block. The collapsed likelihood in the posterior target must reflect the model’s integration over the parameters and all observed blocks. A proposed split may improve the fit in one modality while worsening it in another; the posterior ratio weighs these consequences together with the partition prior.

Diagnostics, Alternatives, and Limitations

A split–merge sampler should be checked for both computational correctness and useful exploration. Track how often each move type is proposed and accepted, and inspect whether the chain visits different cluster counts and distinct partitions. A very low acceptance rate for splits can indicate proposals that routinely produce poor partitions, an overly concentrated target, or a calculation error. Acceptance near 1 is not automatically a sign of success: proposals may be accepted because they make only small, predictable changes or because a proposal probability has been omitted from the ratio.

• Verify reversibility. For each proposal, check that the reverse move exists and that both proposal probabilities include the move-type selection, cluster or pair selection, and partition-construction probabilities.
• Check the target calculation. Confirm that every omics block and the partition prior are included, with no block inadvertently counted twice.
• Inspect reachable states. If the proposal mechanism cannot generate certain splits or merges, the chain cannot explore those transitions, even if its acceptance calculation is correct.
• Compare acceptance by move type. Separate split and merge summaries can reveal a proposal that moves readily in one direction but rarely back.

Single-sample Gibbs updates are an alternative that can make small changes using conditional probabilities. They can be useful for local refinement but may move slowly between distant partitions. Split–merge moves can change several assignments at once, at the cost of more complex proposal accounting and potentially low acceptance. Neither approach removes dependence on the chosen likelihood, prior, or proposal design. A split–merge chain can still mix poorly if its proposals rarely match the structure of plausible partitions.

A common mistake is to accept a proposed split whenever its posterior probability is higher, without considering the proposal mechanism. The correction is to use the posterior ratio and the reverse-to-forward proposal ratio. Another mistake is to interpret an acceptance probability of 0.30 as a 30% posterior probability that the split is true. It is instead the probability of accepting that proposal on this iteration, conditional on the current state and the proposed move.