Why the Factorization Matters
In an integrative mixture, each subject has measurements from several omics blocks and one shared cluster label. A common modeling choice is to assume that the blocks are independent after conditioning on that label. This makes the joint likelihood manageable: each block contributes its own likelihood, and those contributions multiply. The assumption is consequential, however. If two blocks remain strongly associated within clusters, multiplying their separate likelihoods can count similar evidence more than once and make cluster assignments appear more certain than they are.
The central task is to identify exactly what the product assumes, derive the likelihood it gives, and separate two kinds of dependence: dependence due to a common cluster label, which the mixture allows, and residual dependence within a cluster, which this assumption rules out. No particular distribution for a block is required here; a block-specific probability mass function or density can be used.
Model and Conditional Independence
The model also assumes subjects are independent conditional on the cluster probabilities and the component parameters, and that each subject has one shared label governing all of their blocks. The block-specific parameters may differ across blocks and clusters. The factorization does not require the blocks to have the same measurement scale or the same distributional family.
The conditional independence statement is made for fixed component parameters. It does not assert that the blocks are marginally independent after the cluster label is omitted: subjects in the same cluster tend to have profiles generated from the same cluster-specific distributions. Nor does it assert that features within a block are independent. A block likelihood \(f_b\) may describe a multivariate profile with within-block dependence.
For the likelihood below, assume the block measurements are matched to the same subjects, the label is shared across blocks, and the chosen block likelihoods are appropriate for the observed data. If a block is missing for a subject, its contribution can be omitted only when the missingness mechanism is ignorable for the likelihood being used; informative missingness requires modeling that mechanism or otherwise accounting for it.
From the Joint Model to the Likelihood
For a particular subject, the joint probability of the label and all observed blocks is the label probability times the conditional block likelihood. Multiplying across independent subjects gives the complete-data likelihood, meaning the likelihood when the labels are included as if observed.
In practice, the labels are usually unknown. The observed-data likelihood sums over every possible label for each subject. Since labels vary independently across subjects conditional on the model parameters, the sum can be written as a product of subject-level sums:
The order of the operations matters: within each candidate cluster, multiply the block likelihoods; then sum across candidate clusters with their mixing probabilities. In general, it is not correct to sum over clusters separately for each block and then multiply. That alternative would allow different blocks from the same subject to select different latent labels.
For subject \(i\), suppose \(Z_i=k\). The conditional-independence assumption gives the joint block likelihood \(\prod_b f_b(Y_{ib}\mid\theta_{bk})\).
The joint contribution for label \(k\) is \(\pi_k\prod_b f_b(Y_{ib}\mid\theta_{bk})\), because \(P(Z_i=k)=\pi_k\).
Sum the joint contribution over \(k=1,\ldots,K\). This gives the observed-data contribution for subject \(i\).
Multiply the subject-level contributions under conditional independence across subjects to obtain the observed-data likelihood.
This derivation is simply the product rule for the stated generative model followed by summation over an unobserved discrete variable. It also gives the conditional cluster probability for a subject, with parameters treated as fixed:
Thus each block contributes evidence for or against a candidate cluster. Taking logarithms turns the product into a sum of block log-likelihoods, which is useful for computation and interpretation: \(\log\{\pi_k\}+\sum_b\log f_b(Y_{ib}\mid\theta_{bk})\). If the model assigns prior distributions to parameters and averages over them, the relevant posterior calculation must account for that parameter uncertainty. A factorization conditional on fixed parameters should not automatically be treated as an identical factorization after arbitrary shared parameters have been integrated out.
Worked Example: A Subject’s Posterior Cluster Probability
Consider one hypothetical subject with two observed blocks and two clusters. Suppose the prior cluster probabilities are \(\pi_1=0.6\) and \(\pi_2=0.4\). The fitted block likelihoods for this subject are \(0.8\) and \(0.5\) under cluster 1, and \(0.3\) and \(0.6\) under cluster 2. Assume these values are probabilities for the observed patterns, and that the conditional-independence model is appropriate for this calculation.
The unnormalized contribution from cluster 1 is \(0.6(0.8)(0.5)=0.24\). The contribution from cluster 2 is \(0.4(0.3)(0.6)=0.072\). Therefore the observed-data contribution is \(0.24+0.072=0.312\), and
Under this model and these fixed parameter values, the subject’s conditional probability of belonging to cluster 1 is about 0.769. This is not evidence that the blocks are independent in the data; it is the result of applying the stated assumption.
How Block Evidence Accumulates
For two clusters, conditional independence makes posterior odds especially transparent. The posterior odds for cluster 1 versus cluster 2 equal the prior odds multiplied by the likelihood ratio from each block. This follows by dividing the cluster-1 numerator of the posterior formula by its cluster-2 numerator.
Worked Example: Combining Block-Specific Likelihood Ratios
Suppose a hypothetical subject has prior odds \(\pi_1/\pi_2=2/3\) for cluster 1 versus cluster 2. Three block-specific likelihood ratios, \(f_b(Y_{ib}\mid\theta_{b1})/f_b(Y_{ib}\mid\theta_{b2})\), are \(4\), \(0.5\), and \(3\). Assume the blocks are conditionally independent given the cluster and the parameters.
The posterior odds are \((2/3)(4)(0.5)(3)=4\). Odds of \(4\) correspond to probability \(4/(1+4)=0.8\) for cluster 1. The second block counts against cluster 1 because its likelihood ratio is below 1, but the combined evidence still favors cluster 1. On the log-odds scale, the same calculation adds \(\log(2/3)+\log(4)+\log(0.5)+\log(3)=\log(4)\).
The multiplication is justified only by the conditional-independence assumption. If two likelihood ratios arise from nearly redundant measurements, treating them as separate contributions can exaggerate the evidence.
When Conditional Independence Misleads
The most important failure mode is residual cross-block association within a cluster. For example, two assays may measure closely related biological processes, share a technical artifact, or capture the same underlying signal. If their dependence remains after conditioning on the cluster, the product of separate block likelihoods is not the correct joint likelihood. It can count the same pattern repeatedly, yielding overconfident posterior assignments or distorted estimates of cluster prevalence and profiles.
Worked Example: Double-Counting a Shared Signal
Suppose two hypothetical binary blocks both record whether a signal is present. Within cluster 1, the blocks are perfectly concordant, with \(P(Y_{i1}=1,Y_{i2}=1\mid Z_i=1)=0.8\). Within cluster 2, they are also perfectly concordant, with \(P(Y_{i1}=1,Y_{i2}=1\mid Z_i=2)=0.2\). In each cluster the individual block probabilities of a present signal are therefore 0.8 and 0.2, respectively. Let the prior probabilities of the two clusters both be 0.5, and suppose the observed pattern is \((1,1)\).
Using the correct joint probabilities, the posterior probability of cluster 1 is \((0.5)(0.8)/\{(0.5)(0.8)+(0.5)(0.2)\}=0.8\). If the blocks are incorrectly treated as independent, the likelihoods are instead multiplied as \(0.8^2\) and \(0.2^2\), giving \((0.5)(0.64)/\{(0.5)(0.64)+(0.5)(0.04)\}=0.9412\). The independence calculation raises the cluster-1 probability from 0.8 to about 0.941, even though the second block supplies no new information once the first is known in this perfectly concordant example.
The example is deliberately simple: perfect concordance makes the dependence visible. In applications, dependence is often weaker and harder to spot, but the principle is the same. A large posterior probability under the factorized model is conditional on its assumed likelihood; it does not verify that assumption.
Diagnostics and Alternatives
No single diagnostic proves conditional independence. Useful checks ask whether the fitted model can reproduce the cross-block structure that remains within inferred clusters, rather than only matching each block’s marginal distribution. Posterior predictive checks can compare observed and model-replicated cross-block correlations, concordance rates, or other summaries chosen for the data. Such comparisons should account for uncertainty in cluster labels: conditioning on one hard assignment can make apparent within-cluster patterns depend on classification error.
Other warning signs include unusually decisive assignments driven by several similar blocks, strong residual associations within clusters, and substantial changes in assignments when a redundant block is omitted or down-weighted. Sensitivity analyses can compare the factorized model with a model that represents relevant residual dependence, or with a model that gives blocks different cluster allocations. These alternatives answer different scientific questions and add modeling complexity; they should not be treated as automatic fixes.
| Approach | What it assumes or allows | Main trade-off |
|---|---|---|
| Conditional-independence integrative mixture | One shared label; blocks are independent given that label and component parameters. | Simple likelihood and clear block contributions, but residual dependence can be misrepresented. |
| Joint dependent block likelihood | One shared label; the within-cluster joint distribution can include cross-block dependence. | Can represent residual association, but requires specifying and estimating a more complex joint distribution. |
| Modality-specific allocations | Different blocks need not share the same cluster label or partition. | Allows block-specific structure, but changes the shared-subject-cluster interpretation. |
A further limitation is that a good fit to each block separately does not guarantee a good fit to their joint distribution. Conversely, evidence of marginal association between blocks does not by itself refute conditional independence: the shared cluster label can induce marginal association. The diagnostic question is whether important dependence remains after accounting for cluster membership and the model’s other conditioning variables.
Common Mistakes
- Wrong: “The blocks are associated, so conditional independence is impossible.” Correction: They may be associated marginally because they share \(Z_i\). Assess dependence conditional on cluster membership.
- Wrong: “Each block has a suitable marginal distribution, so multiplying their likelihoods is valid.” Correction: Suitable marginals do not establish the required joint factorization.
- Wrong: “A posterior probability near 1 confirms the assumed model.” Correction: It quantifies assignment under the model and parameter values; misspecified dependence can make it too large.
- Wrong: “Conditional independence means every feature in every block is independent.” Correction: The stated factorization is across blocks. Each \(f_b\) may still model dependence among features within its block.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- For one subject with three blocks and two clusters, write the observed-data likelihood contribution under conditional independence.
- Two clusters have equal prior probabilities. The likelihood ratios across two blocks are 2 and 0.5. What are the posterior odds and posterior probability for cluster 1?
- Why can blocks be marginally associated even when they are conditionally independent given cluster membership?
- In the concordant-signal example, why does the independence calculation produce more extreme evidence than the correct joint likelihood?
Solutions. (1) The contribution is \(\sum_{k=1}^{2}\pi_k f_1(Y_{i1}\mid\theta_{1k})f_2(Y_{i2}\mid\theta_{2k})f_3(Y_{i3}\mid\theta_{3k})\). (2) The prior odds are 1, so posterior odds are \(1(2)(0.5)=1\); the posterior probability for cluster 1 is \(1/(1+1)=0.5\). (3) The shared label changes the distribution of all blocks together. Averaging over different possible labels can therefore associate the blocks, even if their distributions factor after conditioning on the label. (4) Given perfect concordance, observing the first block already determines the second. Multiplying the two marginal likelihood ratios as if they were independent counts the same evidence twice; the correct joint likelihood uses the concordant pair’s joint probability once.