Dirichlet Priors for Cluster Prevalence

Why Put a Prior on Cluster Prevalence?

In a shared-cluster model, each sample has one allocation label, and the cluster proportions determine how likely each label is before considering that sample’s omics measurements. These proportions are unknown. A prior on them therefore expresses assumptions about prevalence while allowing uncertainty in the prevalence itself. This matters especially when a cluster has few assigned samples: its next allocation probability can differ substantially depending on the prior.

Let $N$ be the number of samples and $K$ the fixed number of clusters. Write $\boldsymbol{\pi}=(\pi_1,\ldots,\pi_K)$ for the cluster proportions, where $\pi_k\geq 0$ and $\sum_{k=1}^K\pi_k=1$. Let $z_i\in\{1,\ldots,K\}$ be the cluster label of sample $i$, and let $n_k=\sum_{i=1}^N I(z_i=k)$ be the number of samples assigned to cluster $k$. The vector $\mathbf n=(n_1,\ldots,n_K)$ is the cluster-count vector.

Definition: A Dirichlet prior with positive parameters $\eta_1,\ldots,\eta_K$ assigns density $$ p(\boldsymbol{\pi})= \frac{\Gamma(\eta_0)}{\prod_{k=1}^K\Gamma(\eta_k)} \prod_{k=1}^K\pi_k^{\eta_k-1}, \qquad \eta_0=\sum_{k=1}^K\eta_k, $$ on the probability simplex. We write $\boldsymbol{\pi}\sim\operatorname{Dirichlet}(\eta_1,\ldots,\eta_K)$. The parameter $\eta_k$ controls prior weight for cluster $k$, while $\eta_0$ is the total concentration.

The model assumption for the allocations is $z_i\mid\boldsymbol{\pi}\sim\operatorname{Categorical}(\boldsymbol{\pi})$, independently across samples conditional on $\boldsymbol{\pi}$. Thus the prior is on a common set of mixture proportions, not a separate proportion vector for each sample. Conditional on $\boldsymbol{\pi}$, the allocation probability of a particular label sequence is $\prod_k\pi_k^{n_k}$.

From Proportions to Cluster Counts

The Dirichlet prior mean for a proportion is $p_k=\eta_k/\eta_0$. Consequently, the expected number of samples in cluster $k$, before seeing allocations, is $N p_k$. However, these means do not describe the whole prior: integrating over uncertain proportions makes the cluster counts more variable than they would be with fixed proportions.

Integrating the allocation likelihood over the Dirichlet prior gives the probability of a particular ordered allocation sequence. Multiplying by the number of sequences with counts $\mathbf n$ gives the probability of the count vector.

\begin{aligned} P(\mathbf n) &=\frac{N!}{\prod_{k=1}^K n_k!} \int \left(\prod_{k=1}^K\pi_k^{n_k}\right) p(\boldsymbol{\pi})\,d\boldsymbol{\pi}\\ &=\frac{N!}{\prod_{k=1}^K n_k!} \frac{\Gamma(\eta_0)}{\Gamma(\eta_0+N)} \prod_{k=1}^K\frac{\Gamma(\eta_k+n_k)}{\Gamma(\eta_k)}. \end{aligned}

The integral is the normalizing integral of a Dirichlet density with parameters $\eta_k+n_k$, multiplied by the ratio of its normalizing constants to those of the prior. This is the Dirichlet-multinomial distribution. It is a distribution over counts, not a claim that every count vector is equally likely.

Result: Under the Dirichlet-categorical allocation model, $E(n_k)=Np_k$ and $$ \operatorname{Var}(n_k)=N p_k(1-p_k)\frac{N+\eta_0}{1+\eta_0}. $$ For $j\ne k$, $\operatorname{Cov}(n_j,n_k)=-N p_jp_k\frac{N+\eta_0}{1+\eta_0}$. The factor $(N+\eta_0)/(1+\eta_0)$ relative to a multinomial with fixed proportions reflects the additional count variation from uncertainty in $\boldsymbol{\pi}$.

Worked Example: Comparing Count Distributions

Suppose there are $K=3$ clusters and $\boldsymbol{\pi}\sim\operatorname{Dirichlet}(1,1,1)$. For $N=6$, calculate the prior probability of the count vector $(4,1,1)$, and compare the chance that all six samples fall in one cluster with the chance under fixed equal proportions.

Here $\eta_0=3$. The Dirichlet-multinomial formula gives $$ P(4,1,1)=\frac{6!}{4!1!1!}\frac{\Gamma(3)}{\Gamma(9)} \frac{\Gamma(5)}{\Gamma(1)} \frac{\Gamma(2)}{\Gamma(1)} \frac{\Gamma(2)}{\Gamma(1)} =30\cdot\frac{2}{40320}\cdot24 =\frac{1}{28}\approx0.03571. $$

There are three count vectors in which all six samples occupy one cluster, namely $(6,0,0)$ and its two permutations. With this prior, each particular count vector has probability $1/28$, so the probability of all six in one cluster is $3/28\approx0.1071$. By contrast, if the proportions were fixed at $(1/3,1/3,1/3)$, that probability would be $3(1/3)^6=1/243\approx0.004115$. The random proportions allow more extreme count vectors. This comparison concerns the allocation prior, not the posterior probability of a biological cluster structure.

How Concentration Shapes Prevalence

For a symmetric prior, $\eta_k=\eta$ for every cluster and $p_k=1/K$. The prior mean is balanced, but the amount of variability depends on $K\eta$, the total concentration. With small $\eta$, the prior places more mass near the simplex boundaries, where one or a few proportions dominate. With larger $\eta$, it concentrates more tightly around equal proportions. Neither setting guarantees balanced realized counts: a mean of $1/K$ is not the same as requiring equal cluster sizes.

An asymmetric prior can encode different expected prevalences. For example, if one cluster is expected to be less common, choosing a smaller $\eta_k$ for that cluster lowers its prior mean proportion. The parameters should be selected to reflect a defensible prevalence assumption or subjected to sensitivity analysis; they are not automatically estimated from the omics profiles by merely specifying the prior.

Worked Example: Prior Probability for the Next Allocation

Suppose three clusters currently contain $8$, $2$, and $0$ samples, respectively, and use the prior $\operatorname{Dirichlet}(1,1,1)$. Find the probability of each label for one additional sample after integrating out $\boldsymbol{\pi}$.

The posterior distribution of the proportions given these allocations is $\operatorname{Dirichlet}(9,3,1)$: the allocation likelihood adds $n_k$ to each prior parameter. The next-sample probabilities are therefore $$ P(z_{\mathrm{new}}=k\mid\mathbf n) =\frac{n_k+\eta_k}{N+\eta_0}. $$

Here the denominator is $10+3=13$. The probabilities for clusters 1, 2, and 3 are $9/13\approx0.6923$, $3/13\approx0.2308$, and $1/13\approx0.07692$, respectively. The empty cluster retains positive probability because its prior parameter is positive, but it is less favored than clusters with existing memberships. These are probabilities before using the new sample’s omics data.

Allocating Samples Using Both Prevalence and Omics Evidence

For a sample with observed data $y_i$, let $f_k(y_i)$ denote its likelihood under cluster $k$, conditional on the current cluster-specific model parameters. In a multi-omics model, $y_i$ collects the sample’s measurements across blocks; the likelihood can be the product of block likelihoods under the model’s conditional-independence assumptions. The prevalence contribution and the data likelihood combine as $$ P(z_i=k\mid\mathbf z_{-i},\mathbf y,\boldsymbol{\theta}) \propto (n_{-i,k}+\eta_k)f_k(y_i), $$ where $n_{-i,k}$ counts the other samples currently assigned to $k$, and $\boldsymbol{\theta}$ denotes the current profile parameters. Normalize these $K$ weights by their sum to obtain probabilities.

The count term favors clusters with more existing members, while the likelihood term can favor a smaller cluster when the sample’s measurements fit it better. This is a conditional allocation calculation with profile parameters held fixed. It does not derive or update priors for those profiles.

Worked Example: Strong Evidence for a Smaller Cluster

Use counts $(8,2,0)$ and prior parameters $(1,1,1)$. Suppose a new sample has likelihoods $f_1(y)=0.02$, $f_2(y)=0.30$, and $f_3(y)=0.10$ under the three current cluster profiles. Compute its allocation probabilities.

The unnormalized weights are $(8+1)(0.02)=0.18$, $(2+1)(0.30)=0.90$, and $(0+1)(0.10)=0.10$. Their sum is $1.18$, so the normalized probabilities are $$ (0.18/1.18,\;0.90/1.18,\;0.10/1.18) \approx(0.1525,\;0.7627,\;0.08475). $$

The second cluster is smaller than the first, but its likelihood for this sample is sufficiently larger that its allocation probability is highest. The calculation illustrates why cluster prevalence is one contribution to assignment, not a substitute for the data likelihood. The likelihood numbers are hypothetical and are assumed to be evaluated under the same model and on a comparable scale.

Small Groups and Prior Reinforcement

Sometimes the question concerns a set of $r$ samples treated as one group whose members must share a cluster label. Conditional on all other assignments, the prior predictive probability that the group goes to cluster $k$ is $$ P(\text{group in }k\mid\mathbf n_{-}) =\frac{(\eta_k+n_{-,k})^{\overline r}}{(\eta_0+N_-)^{\overline r}}, \qquad a^{\overline r}=a(a+1)\cdots(a+r-1), $$ where $N_-$ is the number of samples outside the group. The numerator reflects reinforcement: after the first group member is assigned to $k$, the integrated predictive weight for another member assigned there increases by one. If the group’s observations have likelihoods $f_k(y_j)$, this prior factor is multiplied by $\prod_{j=1}^r f_k(y_j)$ for a shared cluster assignment, then normalized across clusters.

Worked Example: Prior Odds for a Two-Sample Group

Suppose the other samples give counts $(8,2,0)$, the prior is $\operatorname{Dirichlet}(1,1,1)$, and two new samples are required to share a cluster. Compare the prior probabilities for assigning both to cluster 2 versus both to cluster 3.

There are $N_-=10$ other samples, so the common denominator for either assignment is $13\cdot14$. The numerator for cluster 2 is $(2+1)(2+2)=3\cdot4=12$; for cluster 3 it is $(0+1)(0+2)=1\cdot2=2$. Thus the probabilities are $12/(13\cdot14)=6/91\approx0.06593$ and $2/(13\cdot14)=1/91\approx0.01099$. Their prior odds are $6:1$, favoring cluster 2. If the group’s measurements fit cluster 3 much better, the likelihood ratio can counteract those odds; prevalence alone does not settle the assignment.

Assumptions, Diagnostics, and Alternatives

These calculations rely on a finite $K$, positive Dirichlet parameters, and conditionally independent categorical labels given one shared $\boldsymbol{\pi}$. The group calculation additionally assumes its members are constrained to share one label. If the samples have different allocation mechanisms or cluster proportions that vary across subgroups, this model’s predictive probabilities may not represent the intended structure.

Check the implied prior.
For the chosen $\eta_k$, inspect $p_k=\eta_k/\eta_0$ and simulate or calculate plausible count vectors for the intended sample size.

Check sensitivity.
Repeat the analysis under reasonable alternative concentration values or prevalence means. Large changes in rare-cluster assignments indicate meaningful prior influence.

Separate prevalence from fit.
Review the count weights and the cluster-specific likelihood contributions separately. A large cluster can dominate because of prevalence even when its fit is weaker.

A common mistake is to say that $\eta_k=1$ “forces equal cluster sizes.” It gives equal prior means when used symmetrically, but the resulting count distribution still permits unequal and even highly uneven sizes. Another mistake is to treat $(n_k+\eta_k)/(N+\eta_0)$ as the complete assignment probability for a data-bearing sample; it is only the prevalence-based predictive probability, and the sample likelihood must also be included.

A fixed-proportion model is a useful contrast: it uses specified $\boldsymbol{\pi}$ directly and has multinomial count variation, without integrating over uncertainty in the proportions. A Dirichlet prior instead gives a transparent finite-dimensional way to express prevalence uncertainty and its induced count variation. Its limitation is that the prior family may not capture every desired pattern of prevalence, and conclusions for small clusters can be sensitive to its parameters. In particular, an apparently data-driven allocation may still be strongly shaped by the prior when likelihood evidence is weak.

Key takeaway: A Dirichlet prior induces a Dirichlet-multinomial distribution over cluster sizes. After integrating out the proportions, an existing cluster’s predictive weight is $n_k+\eta_k$; for an observed sample, this weight must be combined with its cluster-specific likelihood.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For $K=4$ and a symmetric Dirichlet prior with $\eta_k=2$, what are the prior mean proportions and total concentration?
With prior parameters $(2,1)$ and counts $(3,1)$, calculate the prior predictive probabilities for the next allocation.
For a group of two samples assigned together, explain why its integrated prior weight for cluster $k$ is proportional to $(n_k+\eta_k)(n_k+\eta_k+1)$, rather than just $n_k+\eta_k$.
What is wrong with concluding that a symmetric Dirichlet prior guarantees equal realized cluster sizes?

Solutions. (1) Each prior mean is $2/(4\cdot2)=1/4$, and $\eta_0=8$. (2) The updated parameters are $(5,2)$, with total $7$, so the probabilities are $5/7\approx0.7143$ and $2/7\approx0.2857$. (3) Integrating out the shared proportions gives sequential predictive factors: the first allocation contributes $n_k+\eta_k$, and the second contributes $n_k+\eta_k+1$, with a common denominator across candidate clusters. (4) Symmetry gives equal expected proportions, not equal counts in every realization; uncertainty in the proportions allows uneven count vectors.