Bayesian Priors on the Number of Clusters

Why Put a Prior on the Number of Clusters?

An integrative clustering model assigns samples to groups using measurements from multiple omics blocks. Even when the model for each block is specified, the number of groups remains a modeling choice. Fixing that number can be appropriate when it is part of the scientific design, but it does not represent uncertainty about the number of groups. A Bayesian model can instead assign prior probabilities to several possible counts and update them using the observed data.

The distinction is consequential. A prior over cluster counts does not simply add a setting to a fixed-count analysis: posterior probabilities depend on each count's marginal likelihood, which averages over the model's possible partitions and cluster-specific parameters. The resulting inference is conditional on the candidate models and their assumptions.

For a particular \(K\), write \(z\) for a partition of the samples into exactly \(K\) nonempty groups, and write \(\theta\) for the cluster-specific parameters. These parameters may describe, for example, Gaussian profiles for continuous features or count distributions for sequencing features. The earlier material on conjugate priors for cluster-specific omics profiles gives examples of priors that make some of these parameter integrations tractable.

A typical model assumes that, conditional on a sample's cluster and the relevant cluster-specific parameters, its measurements follow the chosen likelihood. If the likelihood factorizes across omics blocks, that is an additional conditional-independence assumption; it should be justified for the application rather than presumed. The model also specifies a prior \(p(z\mid K)\) over the allowed partitions and a proper parameter prior \(p(\theta\mid z,K)\). The partition prior must be normalized over the partitions represented by the model.

Marginal Likelihood and Posterior Cluster Count

For each candidate count, the marginal likelihood is the probability density of the observed data under that model, after averaging over partitions and integrating out parameters. For continuous data it is a density, not the probability of observing one exact data value. In a common notation,

The sum is over partitions with exactly \(K\) nonempty groups. The integral averages the likelihood over parameter values according to the prior. Thus, \(p(Y\mid K)\) is not generally the likelihood evaluated at a best-fitting partition or at maximum-likelihood parameter estimates. It accounts for the full model, including its prior distributions.

Bayes' rule combines this evidence with the prior probability of \(K\):

The denominator is the marginal likelihood under the full collection of candidate models. For two counts \(K_a\) and \(K_b\), the posterior odds are the prior odds multiplied by the Bayes factor:

This identity separates the data's relative support, expressed by the Bayes factor, from the prior odds. It also makes clear why the posterior count is not determined by fit alone. A count with a larger marginal likelihood can still have a smaller posterior probability if its prior probability is sufficiently low.

Choosing and Interpreting the Prior

The prior \(\pi_K\) encodes beliefs before seeing \(Y\) about how many occupied groups are plausible within the specified candidate set. A uniform prior gives each candidate count equal probability; it does not mean that every possible partition is equally likely. A prior decreasing with \(K\) places more mass on smaller counts. Neither choice is automatically neutral: the support and probability assigned to each count matter.

One possible construction is a truncated geometric prior. For \(K=1,\ldots,K_{\max}\), assign weights proportional to \(q^{K-1}\), with \(0<q<1\), then normalize them to sum to one. The truncation must be stated because it changes the normalized probabilities. This prior can be useful when smaller counts are preferred a priori, but its consequences should be checked against other plausible choices.

A prior over counts is distinct from a prior over cluster prevalences. The latter describes relative group sizes conditional on a clustering model; the former assigns probability to how many occupied groups there are. A model can have both kinds of prior, and the marginal likelihood for each \(K\) must account for the prevalence and allocation assumptions used within that model. In particular, the definition of \(K\) must match the model's actual occupied groups.

Why the Prior Can Matter: A Derivation and Example

The posterior odds identity follows directly from Bayes' rule. For two candidate counts, both posterior probabilities share the same normalization constant \(p(Y)\), so it cancels in their ratio:

Thus, even fixed marginal likelihoods do not determine posterior odds without prior odds. The following hypothetical example shows how the same evidence can yield different posterior conclusions under different priors.

Model Checks, Alternatives, and Limitations

Before interpreting posterior probabilities for \(K\), check the assumptions that make the comparison meaningful. Each candidate model should describe the same observed samples and data blocks, use comparable measurement and missing-data assumptions, and have proper normalized priors. The partition prior and parameter prior must be specified for every candidate count. If models differ in more than cluster count, posterior differences cannot be attributed to cluster count alone.

Marginal likelihoods can be difficult to estimate accurately, especially when a model has many possible partitions or a high-dimensional parameter space. Because posterior model probabilities depend directly on these values, numerical error can change the apparent ranking of counts. Compare results across appropriate computational settings or estimators, and verify that computed probabilities are stable. A posterior concentrated at \(K_{\max}\) is also a warning: the chosen support may exclude plausible larger counts, so the analysis should not treat the upper-bound result as proof that the true count equals \(K_{\max}\).

Prior predictive checks can expose implausible choices before fitting: draw \(K\) from \(\pi_K\), then generate partitions and data under the specified model, and ask whether the resulting counts, cluster sizes, and omics profiles are scientifically credible. Posterior sensitivity checks repeat the analysis with defensible alternatives for \(\pi_K\) and, where relevant, the within-count partition and parameter priors. If conclusions change sharply, report that dependence rather than presenting one count as unambiguous.

Fixing \(K\) remains useful when the count is genuinely prespecified or when the goal is explicitly conditional inference: results then describe assignments and profiles given that count, not uncertainty about it. Information criteria such as BIC offer a different approach by approximating fit-versus-complexity trade-offs; they are not posterior probabilities for \(K\), and do not provide the same direct model-averaging interpretation. Explicit priors are attractive when uncertainty over a finite set of counts is itself the target, but their results depend on the candidate set and prior specification.

A further distinction matters in finite mixtures: a model may specify a number of available component slots while a particular allocation uses fewer groups. The count studied here is the number of occupied groups. If a computational formulation uses component slots, its prior and posterior calculations must target that occupied count rather than silently treating the slot count as the occupied count. The representation should make this target explicit.