Gaussian Likelihoods for Continuous Omics Profiles

Why the Gaussian Likelihood Needs a Covariance Choice

A continuous omics profile might contain transformed expression measurements, protein abundances, or other real-valued features. In a clustering model, each cluster needs a probability distribution for profiles assigned to it. A multivariate Gaussian is a common choice: it describes both the cluster’s typical profile and the variability and dependence of measurements around that profile. The key modeling decision is how much structure to allow in the covariance matrix.

The earlier material on conditional independence in integrative mixtures motivates a distinction: blocks may be modeled as independent of one another conditional on a sample’s cluster, while features within a continuous block may still be correlated. The covariance choice determines whether and how that within-block dependence is represented.

For a profile \(x_i\), the density under cluster \(k\) is

Here \(|\Sigma_k|\) is the determinant. The quadratic form is the squared Mahalanobis distance from \(x_i\) to the cluster mean: it measures deviation while accounting for the covariance structure. The determinant term adjusts the density for the volume and orientation of the distribution. A broader distribution spreads probability over more space and, all else equal, has a smaller density near its center.

For \(n_k\) profiles assigned to cluster \(k\), conditional independence of samples given the cluster parameters gives a likelihood that is the product of their densities. Taking logs is convenient because it turns that product into a sum:

This likelihood treats the observed feature values as continuous and assumes that, conditional on membership and the cluster parameters, profiles assigned to the cluster follow the specified Gaussian distribution. The covariance matrix must be positive definite for the ordinary density to exist. The model is a distributional assumption, not a claim that cluster membership causes the measured profile.

Three Covariance Structures

The structures differ in which variances and covariances they permit. In all three cases the mean can vary across features. Their restrictions concern the shape of variability around that mean.

A spherical covariance assumes that all features have the same within-cluster variance and are uncorrelated. A diagonal covariance relaxes the equal-variance restriction but still assumes conditional independence of features within that block. A full covariance can represent linear dependence and different orientations of cluster variation, but it requires many more parameters. These assumptions concern features within a profile; they are distinct from assumptions about dependence between omics blocks.

Reading and Comparing Gaussian Likelihoods

A likelihood value measures how well a specified parameter setting assigns density to observed profiles. It is not itself the probability that a cluster assignment or model is correct. To see how covariance changes the calculation, consider a hypothetical two-feature profile \(x=(2,1)\) evaluated at mean \(\mu=(0,0)\) under three covariance choices. The numerical comparison below is for this one profile only.

For a fixed cluster assignment and an unrestricted Gaussian covariance, the maximum likelihood estimates are the feature-wise sample mean and the scatter matrix divided by \(n_k\). This follows directly from the log-likelihood. First, differentiating with respect to the mean gives the condition \(\sum_{i:z_i=k}(x_i-\mu_k)=0\), so \(\widehat{\mu}_k=\bar{x}_k\). Substituting this mean, let \(A=\sum_{i:z_i=k}(x_i-\bar{x}_k)(x_i-\bar{x}_k)^\top\). The covariance-dependent part is \(-\frac{n_k}{2}\log|\Sigma_k|-\frac{1}{2}\operatorname{tr}(\Sigma_k^{-1}A)\). Its maximum, when \(A\) is positive definite, occurs at \(\widehat{\Sigma}_k=A/n_k\). The divisor is \(n_k\) because this is a likelihood estimate; the familiar divisor \(n_k-1\) gives an unbiased sample covariance under the usual independent, identically distributed sampling conditions.

Diagnostics, Failure Modes, and Model Choice

The central practical tension is flexibility versus estimability. A full covariance has \(d(d+1)/2\) covariance parameters per cluster, so even a moderate feature count can require many observations in each cluster. If the centered profiles do not span all \(d\) dimensions, the scatter matrix is singular. In particular, its rank is at most \(n_k-1\), so \(n_k\leq d\) guarantees singularity. A singular estimate has determinant zero and no ordinary Gaussian density with that covariance; the inverse and log-determinant in the likelihood are undefined.

Useful checks include inspecting cluster sizes relative to the number of features, checking covariance eigenvalues and condition numbers, and looking for extreme correlations or variances that arise from very few profiles. A numerical routine that returns a covariance is not by itself proof that the estimate is stable: near-zero eigenvalues can make the inverse highly sensitive to small changes in the data. Also inspect whether the Gaussian shape is a plausible description, using residual patterns or simulated profiles from the fitted distribution. Strong skewness, heavy tails, or outliers can make a Gaussian likelihood poorly matched to the data.

A diagonal or spherical model can be a defensible alternative when data are limited, but each restriction should be scientifically and empirically plausible. The diagonal model may miss coordinated feature variation; the spherical model may additionally misrepresent features with different scales. Standardizing features can change the practical meaning of the spherical assumption, so any transformation should be considered part of the model specification and reported. A full covariance may capture dependence, but its extra flexibility can produce unstable estimates or overly tailored clusters. Regularization or dimension reduction may help with estimation, but those choices require their own justification and should not be confused with evidence that the unrestricted Gaussian assumptions hold.

When several omics blocks are modeled in one shared-cluster mixture, a Gaussian density can be used for a continuous block, with its own mean and covariance for each cluster. Under conditional independence of blocks given cluster membership, their block likelihoods multiply, or their log-likelihood contributions add. That factorization does not imply independence among features within a block: a full covariance can still represent those dependencies. Likelihood comparisons should use the same observations and account for model complexity; a higher maximized likelihood alone tends to favor a more flexible covariance structure.

Common Mistakes

• Confusing sample covariance with the Gaussian maximum likelihood estimate. Dividing the scatter by \(n_k-1\) gives the unbiased sample covariance; the Gaussian maximum likelihood estimate with an estimated mean divides by \(n_k\).
• Assuming diagonal means equal variance. A diagonal covariance allows each feature its own variance. Equal variances are imposed by the spherical structure.
• Calling a full covariance automatically preferable. It can model within-block dependence, but it needs enough information for stable estimation and may fit noise.
• Treating a singular covariance as an ordinary Gaussian covariance. The standard density requires a positive-definite covariance; a zero determinant makes the usual log-likelihood undefined.
• Interpreting the largest single-profile density as proof of the best clustering. A density comparison at fixed parameter values is not a model-selection result and does not account for complexity or uncertainty.