The Gaussian Parsimonious Clustering Model Families

Why Covariance Models Need a Shared Vocabulary

In model-based clustering, each Gaussian component has a covariance matrix that determines the component’s spread and geometric form. A completely unrestricted covariance matrix for every component can describe many patterns, but it may use more flexibility than an analysis needs. Gaussian parsimonious clustering models, commonly abbreviated GPCM, organize covariance assumptions by asking which aspects of the component covariances are shared and which may vary.

The key distinction is among volume, shape, and orientation. The standard model label records whether each aspect is equal across components or allowed to vary. It also indicates whether the covariance is spherical or restricted to be diagonal. The label is a compact summary of structural constraints, not a ranking of models or a statement about the number of clusters.

Volume, Shape, and Orientation in the GPCM Decomposition

Suppose there are \(G\) Gaussian components in \(p\) dimensions, with positive-definite covariance matrices \(\Sigma_1,\ldots,\Sigma_G\). For component \(k\), write its spectral decomposition in the GPCM form

\Sigma_k=\lambda_k D_k A_k D_k^{\mathsf T}.

Here \(\lambda_k>0\) is a scalar, \(D_k\) is an orthogonal matrix whose columns give the principal directions, and \(A_k\) is diagonal with positive entries and determinant one. The eigenvalues of \(\Sigma_k\) are \(\lambda_k\) times the diagonal entries of \(A_k\). This separates an overall scale from relative axis lengths and directions.

Definition: In the GPCM decomposition, volume is represented by \(\lambda_k\), shape by the determinant-one diagonal matrix \(A_k\), and orientation by the orthogonal matrix \(D_k\). Equal volume means a common \(\lambda_k\); equal shape means a common \(A_k\); equal orientation means a common \(D_k\). A diagonal covariance model fixes orientation to the coordinate axes, while a spherical covariance has no nontrivial shape or orientation.

The determinant-one normalization is what makes the split between volume and shape precise: multiplying all the entries of \(A_k\) by a common factor would otherwise let the same overall scale be represented in two different ways. The following result shows how the volume is recovered directly from \(\Sigma_k\).

Derived result: For a positive-definite \(p\times p\) covariance matrix with the decomposition above, \(\det(\Sigma_k)=\lambda_k^p\), and therefore \(\lambda_k=(\det\Sigma_k)^{1/p}\).

Derivation. The determinant of a product is the product of the determinants. Since \(D_k\) is orthogonal, \(D_k^{\mathsf T}D_k=I\), so \((\det D_k)^2=1\). Also, \(\det A_k=1\). Consequently,

\det(\Sigma_k) =\det(\lambda_k I)\det(D_k)\det(A_k)\det(D_k^{\mathsf T}) =\lambda_k^p(\det D_k)^2 =\lambda_k^p.

The positive \(p\)th root gives the stated formula. In particular, two components have equal volume exactly when their covariance determinants are equal. Once \(\lambda_k\) is obtained, the eigenvalues divided by \(\lambda_k\) give the entries of \(A_k\), while the corresponding eigenvectors give \(D_k\). Thus equality of determinants alone checks only the volume constraint; it does not establish equal shape or orientation.

How to Read the Model Labels

A standard GPCM label has three positions. The first describes volume, the second shape, and the third orientation. An E means equal across components; a V means variable across components. In the third position, I denotes the identity orientation: the principal axes are fixed to the coordinate axes. With this convention, \(EVE\) means equal volume, variable shape, and equal orientation.

There are two important qualifications. First, an \(I\) in the orientation position does not by itself mean spherical: a diagonal covariance can have unequal diagonal entries and thus unequal axis lengths. Second, spherical models are a special case in which \(A_k=I\); there is no varying orientation to encode. Their labels use \(E\) or \(V\) for volume and \(I\) for the spherical structure.

Model label	Covariance structure	Constraints across components
EII	Spherical	Equal volume; spherical shape
VII	Spherical	Variable volume; spherical shape
EEI	Diagonal	Equal volume and shape; coordinate axes fixed
VEI	Diagonal	Variable volume; equal shape; coordinate axes fixed
EVI	Diagonal	Equal volume; variable shape; coordinate axes fixed
VVI	Diagonal	Variable volume and shape; coordinate axes fixed
EEE	General ellipsoidal	Equal volume, shape, and orientation
VEE	General ellipsoidal	Variable volume; equal shape and orientation
EVE	General ellipsoidal	Equal volume and orientation; variable shape
VVE	General ellipsoidal	Variable volume and shape; equal orientation
EEV	General ellipsoidal	Equal volume and shape; variable orientation
VEV	General ellipsoidal	Variable volume and orientation; equal shape
EVV	General ellipsoidal	Equal volume; variable shape and orientation
VVV	General ellipsoidal	Variable volume, shape, and orientation

The table lists 14 standard families. Two are spherical, four are diagonal, and eight allow general ellipsoidal covariance matrices. The count follows directly from the structures: spherical models have only the two volume choices; diagonal models have two choices each for volume and shape; general ellipsoidal models have two choices for each of the three features.

Derived result: The standard GPCM families comprise \(2+4+8=14\) labels: two spherical, four diagonal, and eight general ellipsoidal models. The count reflects the distinct covariance structures, not a claim that all parameter settings within different families are disjoint.

For the diagonal family, the \(I\) restriction fixes \(D_k=I\). Volume and shape can each be equal or variable, producing \(EEI\), \(VEI\), \(EVI\), and \(VVI\). For the general family, each of volume, shape, and orientation can be equal or variable, giving \(2^3=8\) labels. For the spherical family, shape is fixed to \(I\), so only the volume choice remains. This is why the spherical models are not counted by treating all three label positions as independent E/V choices.

Worked Examples: Decoding and Checking Covariance Constraints

Worked Example: A Diagonal Model with Common Shape

Consider two components in two dimensions with covariance matrices

\Sigma_1= \begin{pmatrix}4&0\\0&1\end{pmatrix}, \qquad \Sigma_2= \begin{pmatrix}8&0\\0&2\end{pmatrix}.

Both matrices are diagonal, so their principal axes are the coordinate axes. Their determinants are \(\det(\Sigma_1)=4\) and \(\det(\Sigma_2)=16\). Since \(p=2\), their volume parameters are

\lambda_1=\sqrt{4}=2,\qquad \lambda_2=\sqrt{16}=4.

Dividing each covariance matrix by its volume gives

A_1=\frac{\Sigma_1}{2} =\begin{pmatrix}2&0\\0&1/2\end{pmatrix}, \qquad A_2=\frac{\Sigma_2}{4} =\begin{pmatrix}2&0\\0&1/2\end{pmatrix}.

The volumes vary, the shapes are equal, and the orientation is fixed to the coordinate axes. The correct label is therefore VEI. The determinant check verifies the varying volumes, while the normalized matrices verify common shape.

Worked Example: Equal Volume, Different Shape, Shared Orientation

Let \(R\) be the two-dimensional rotation matrix through \(45^\circ\),

R=\frac{1}{\sqrt{2}} \begin{pmatrix}1&-1\\1&1\end{pmatrix}.

Take component covariances

\Sigma_1=R \begin{pmatrix}4&0\\0&1\end{pmatrix} R^{\mathsf T} = \begin{pmatrix}5/2&3/2\\3/2&5/2\end{pmatrix}, \qquad \Sigma_2=R \begin{pmatrix}8&0\\0&1/2\end{pmatrix} R^{\mathsf T} = \begin{pmatrix}17/4&15/4\\15/4&17/4\end{pmatrix}.

The determinant of each covariance is the product of its displayed eigenvalues: \(4\cdot1=4\) for \(\Sigma_1\) and \(8\cdot(1/2)=4\) for \(\Sigma_2\). Thus both volumes are \(\sqrt{4}=2\). Their normalized shapes are

A_1=\begin{pmatrix}2&0\\0&1/2\end{pmatrix}, \qquad A_2=\begin{pmatrix}4&0\\0&1/4\end{pmatrix}.

The shapes differ, but both covariance matrices use the same rotation \(R\), so their orientation is shared. The label is EVE: equal volume, variable shape, equal orientation. The nonzero off-diagonal entries are consistent with rotated axes; they do not indicate that the matrices have different orientations.

Worked Example: Variable-Volume Spherical Components

Suppose two components have covariance matrices \(\Sigma_1=3I_2\) and \(\Sigma_2=7I_2\). Each matrix is a scalar multiple of the identity, so both components are spherical: their two eigenvalues are equal, and neither has a distinguished orientation. Their volumes are

\lambda_1=(\det\Sigma_1)^{1/2} =(3^2)^{1/2}=3, \qquad \lambda_2=(\det\Sigma_2)^{1/2} =(7^2)^{1/2}=7.

The volume varies across components, while the spherical structure is common. The label is VII. It would be incorrect to use \(VVI\): that label describes diagonal covariances with potentially different shapes, not spherical covariances.

A Reliable Decoding Procedure

To assign a label from covariance constraints, first decide whether the family is spherical, diagonal, or general ellipsoidal. Then record the permitted equality or variation for the features that remain. Checking the normalized eigenvalues and eigenvectors prevents a visual impression—such as “these ellipses look alike”—from substituting for the actual constraints.

Identify the structural family.
If every covariance is a scalar multiple of \(I\), use a spherical label. If the covariances are diagonal in the fixed coordinates, use a diagonal label. Otherwise, use a general ellipsoidal label.

Compare volumes.
Compute \(\lambda_k=(\det\Sigma_k)^{1/p}\). Equal values across components give E in the first position; allowed component-specific values give V.

Compare shapes and orientations.
Normalize eigenvalues by \(\lambda_k\) to compare shape matrices. Compare principal directions for orientation. In a diagonal model, the orientation position is I; in a spherical model, shape and orientation are already fixed by the spherical restriction.

Common Mistakes and Why the Distinction Matters

The GPCM labels make it possible to state a covariance assumption compactly, but a short label can hide different geometric meanings if its positions are read carelessly. In particular, the first letter is not a general indicator of how much the covariance changes: it refers only to volume.

Reading E as “the whole covariance is equal.” E in the first position means equal volume only. For example, EVE allows shape to vary despite equal volume.
Treating I as “isotropic.” I fixes the axes to the coordinate directions in a diagonal model. Unequal diagonal entries still give an elongated, non-spherical covariance. Spherical models are a separate structural case.
Using determinants to infer all three features. Equal determinants establish equal volume, not equal normalized eigenvalues or equal principal directions.
Calling a model EEE merely because one fitted set of covariances happens to be equal. A model label describes the constraints permitted by the family. A particular fit may satisfy extra equalities without changing the family’s definition.
Assuming the letters are interchangeable. The positions have fixed meanings: volume, shape, orientation. For instance, EEV and EVE impose different constraints.

These distinctions matter because covariance constraints determine which cluster geometries a model family can represent. A spherical family cannot describe an elongated cluster; a diagonal family cannot rotate its principal axes away from the coordinate system; and a general family can permit that rotation. The label therefore communicates a geometric assumption that should be interpreted before comparing or fitting covariance families.

Check Your Understanding

For each question, identify the covariance features involved rather than relying on the label alone.

What does the label VVE say about volume, shape, and orientation?
Why does an \(I\) in the third position not imply that the covariance is spherical?
Two covariance matrices have equal determinants but different normalized eigenvalues. Which feature is shared, and which feature differs?
How many standard labels are in the general ellipsoidal group, and which choices generate them?
For \(\Sigma_1=2I_2\) and \(\Sigma_2=5I_2\), identify the structural family and the appropriate label.