The Minimum Covariance Determinant Estimator

Why Minimize a Covariance Determinant?

The ordinary sample covariance uses every observation. A few extreme observations can therefore strongly affect its estimate of multivariate scatter. The minimum covariance determinant estimator, usually abbreviated MCD, takes a different approach: it considers subsets of a chosen size and selects one with small covariance determinant. The selected subset supplies both a location estimate and a scatter estimate.

The determinant is a measure of the overall volume of a covariance ellipsoid. Minimizing it favors a subset whose observations are concentrated together in several directions, rather than merely having small spread along one coordinate. The estimator is defined by an explicit finite optimization problem; the subset size must be specified as part of that problem.

Definition: Let $x_1,\ldots,x_n\in\mathbb{R}^p$ be observed vectors, and fix an integer $h$ with $p+1\leq h\leq n$. For each index subset $H\subseteq\{1,\ldots,n\}$ of size $h$, define its mean and covariance matrix by $$ \bar{x}_H=\frac{1}{h}\sum_{i\in H}x_i, \qquad S_H=\frac{1}{h}\sum_{i\in H}(x_i-\bar{x}_H)(x_i-\bar{x}_H)^{\mathsf T}. $$ A minimizing subset $H^*$ satisfies $\det(S_{H^*})=\min_{|H|=h}\det(S_H)$. The raw MCD location and scatter estimates are $\hat{\mu}_{\mathrm{MCD}}=\bar{x}_{H^*}$ and $\hat{\Sigma}_{\mathrm{MCD}}=S_{H^*}$. If several subsets attain the same minimum, the raw estimator can have more than one minimizing subset.

This convention uses divisor $h$, not $h-1$. For any fixed $h$, changing to divisor $h-1$ multiplies every candidate covariance by the same scalar $h/(h-1)$, and hence multiplies every candidate determinant by the same positive scalar $\{h/(h-1)\}^p$. It does not change which subsets minimize the determinant, though it does change the reported scatter estimate.

The constraint $h\geq p+1$ is a necessary size condition for a subset to span $p$-dimensional space. It is not sufficient: the selected observations must also be positioned so that their centered vectors span that space. When $S_H$ is singular, its determinant is zero. Thus, if singular candidate subsets are allowed, they can win the minimization; for a nonsingular scatter estimate one needs conditions ensuring the relevant candidate subsets are full rank.

Determinant and Ellipsoid Volume

The reason for using a determinant is geometric. If $S$ is positive definite, an ellipsoid centered at $\mu$ with shape matrix $S$ has the form $\{x:(x-\mu)^{\mathsf T}S^{-1}(x-\mu)\leq c\}$, where $c>0$. Its volume increases with $\sqrt{\det(S)}$.

Result: For positive definite $S\in\mathbb{R}^{p\times p}$, the volume of the ellipsoid $\{x:(x-\mu)^{\mathsf T}S^{-1}(x-\mu)\leq c\}$ is $v_p c^{p/2}\sqrt{\det(S)}$, where $v_p$ is the volume of the unit ball in $\mathbb{R}^p$.

To verify this, let $y=S^{-1/2}(x-\mu)/\sqrt{c}$. The ellipsoid maps to the unit ball. Reversing the map gives $x=\mu+\sqrt{c}\,S^{1/2}y$, whose Jacobian determinant in absolute value is $c^{p/2}\det(S^{1/2})=c^{p/2}\sqrt{\det(S)}$. The change-of-variables formula therefore gives the stated volume. Consequently, among positive definite candidate covariance matrices, minimizing the determinant is equivalent to minimizing the volume of their covariance ellipsoids at a fixed $c$. $\square$

This interpretation does not mean that the MCD minimizes every directional spread separately. A subset may be relatively wide in one direction and narrow in another; the determinant summarizes their combined volume. Nor is the raw MCD automatically calibrated for a particular underlying distribution. Here the target is the subset-based definition and its determinant comparison.

Worked Example: Exhaustive Selection in a Bivariate Sample

Consider five observations $A=(0,0)$, $B=(2,0)$, $C=(0,2)$, $D=(4,1)$, and $E=(1,4)$. Set $p=2$ and $h=3$. There are $\binom{5}{3}=10$ candidate subsets. We will compare all ten, rather than assume that a visually central-looking subset must minimize the determinant.

For three points $x_1,x_2,x_3$ in the plane, let $u=x_2-x_1$ and $v=x_3-x_1$. Their triangle has area $|\det[u\ v]|/2$, and the covariance determinant with divisor 3 is $$ \det(S_{\{1,2,3\}})=\frac{\det[u\ v]^2}{27}. $$ A derivation is given below. Thus, for this fixed $h$, we can rank subsets by the absolute value of the two-dimensional determinant.

Taking the first listed point as $x_1$, the six pair calculations involving $A,B,C,D$ or $E$ are as follows:

$ABC$: $\det[(2,0),(0,2)]=4$.
$ABD$: $\det[(2,0),(4,1)]=2$.
$ABE$: $\det[(2,0),(1,4)]=8$.
$ACD$: $\det[(0,2),(4,1)]=-8$.
$ACE$: $\det[(0,2),(1,4)]=-2$.
$ADE$: $\det[(4,1),(1,4)]=16-1=15$.

For the four remaining subsets, use their first listed point as the base point:

$BCD$: $\det[(-2,2),(2,1)]=-2-4=-6$.
$BCE$: $\det[(-2,2),(-1,4)]=-8+2=-6$.
$BDE$: $\det[(2,1),(-1,4)]=8+1=9$.
$CDE$: $\det[(4,-1),(1,2)]=8+1=9$.

The smallest absolute determinant is 2, attained by $ABD$ and $ACE$. Each therefore has covariance determinant $2^2/27=4/27$, and both are MCD-minimizing subsets. The raw MCD is not unique in this sample unless an additional tie-breaking convention is imposed.

For $ABD$, the mean is $(2,1/3)$. The centered coordinate sums of squares are $8$ and $2/3$, and the centered cross-product sum is $2$. Dividing each by 3 gives $$ S_{ABD}= \begin{pmatrix} 8/3 & 2/3\\ 2/3 & 2/9 \end{pmatrix}, \qquad \det(S_{ABD})=(8/3)(2/9)-(2/3)^2=16/27-12/27=4/27. $$ This independently checks the determinant formula and the subset ranking.

A Formula for Three Points in Two Dimensions

The previous example used a convenient identity. Here is its full derivation. Let the three points be $x_1$, $x_1+u$, and $x_1+v$, and form the matrix $U=[u\ v]$. Their mean is $x_1+(u+v)/3$. The three centered vectors are therefore $-(u+v)/3$, $(2u-v)/3$, and $(-u+2v)/3$.

Writing these three vectors as columns of a matrix gives $UC$, where $$ C=\frac{1}{3} \begin{pmatrix} -1 & 2 & -1\\ -1 & -1 & 2 \end{pmatrix}. $$ The covariance matrix is $S=(1/3)UCC^{\mathsf T}U^{\mathsf T}$. Direct multiplication gives $CC^{\mathsf T}=(1/3)\begin{pmatrix}2&-1\\-1&2\end{pmatrix}$, whose determinant is $1/3$. Since $U$ is square, taking determinants yields $$ \det(S)=\frac{1}{9}\det(U)^2\det(CC^{\mathsf T}) =\frac{\det(U)^2}{27}. $$ This proves the identity. It also shows that three points give a singular covariance precisely when the three points are collinear, because then $\det(U)=0$. $\square$

Worked Example: Comparing the Full-Sample Covariance

Use the same five points, but now take $h=5$. Their coordinate means are both $7/5$. The raw sums of squared $x$- and $y$-coordinates are each $21$, and the sum of coordinate products is $8$. Centering gives $$ \sum (x_i-\bar{x})^2=21-5(7/5)^2=21-49/5=56/5, \qquad \sum (y_i-\bar{y})^2=56/5, $$ and $$ \sum (x_i-\bar{x})(y_i-\bar{y})=8-5(7/5)(7/5)=8-49/5=-9/5. $$

With divisor $h=5$, the full-sample covariance is $$ S_{\{A,B,C,D,E\}}= \begin{pmatrix} 56/25 & -9/25\\ -9/25 & 56/25 \end{pmatrix}, \qquad \det(S)=\frac{56^2-9^2}{625} =\frac{3055}{625}=\frac{611}{125}. $$ The three-point minimizing determinant was $4/27$, approximately $0.148$, whereas the full-sample determinant is $611/125$, approximately $4.888$. This comparison illustrates the optimization’s effect in this particular sample: including all five points produces much larger covariance volume than the best three-point subsets. It does not establish that every smaller subset will have a smaller determinant; the minimizing subset must still be checked.

Worked Example: A Singular Subset Can Win

Take $P=(0,0)$, $Q=(1,0)$, $R=(2,0)$, and $S=(0,1)$, with $p=2$ and $h=3$. The subset $PQR$ is collinear, so its covariance determinant is zero. For comparison, the absolute two-dimensional determinants for $PQS$, $PRS$, and $QRS$ are respectively $$ |\det[(1,0),(0,1)]|=1,\quad |\det[(2,0),(0,1)]|=2,\quad |\det[(1,0),(-1,1)]|=1. $$ Their covariance determinants are $1/27$, $4/27$, and $1/27$, respectively. The minimum over all four subsets is therefore zero, attained by $PQR$. The resulting scatter estimate is singular, so it cannot be used as an ordinary positive-definite covariance shape matrix. This is why a full-rank condition is important when a nonsingular MCD scatter estimate is required.

Choosing and Interpreting the Subset Size

The MCD definition has two separate ingredients: the fixed subset size $h$, and the minimization over all subsets of that size. Changing $h$ changes the candidates and can change both the selected observations and the estimates. A smaller $h$ allows the estimator to focus on a tighter portion of the sample, while a larger $h$ forces more observations into every candidate. There is no single subset size implicit in the determinant criterion; it must be stated or selected according to the intended analysis.

For a fixed $h$, the optimization is finite: there are $\binom{n}{h}$ candidate subsets. This guarantees that at least one subset attains the minimum, since a finite collection of real determinant values has a smallest member. It does not guarantee uniqueness, and it does not guarantee that the minimizing covariance is positive definite. Those are distinct issues to check.

Common Mistakes and Practical Meaning

Comparing only marginal variances. The MCD minimizes the determinant of the full covariance matrix, not each diagonal entry separately. Off-diagonal covariance contributes to the determinant.
Forgetting the divisor convention. Here the subset covariance uses divisor $h$. A different common divisor preserves the minimizing subset for fixed $h$, but changes the numerical scatter estimate.
Assuming the answer is unique. Distinct subsets can tie, as $ABD$ and $ACE$ did. Report the tie or specify the tie-breaking rule used.
Assuming $h\geq p+1$ guarantees nonsingularity. It does not: $p+1$ or more observations can lie in a lower-dimensional affine subspace. A singular subset can have determinant zero and become the minimizer.
Calling every selected subset “the central observations” without checking. The definition is an optimization over determinants. Intuition about which points look central is not a substitute for computing or otherwise establishing the minimum.

The MCD is useful because its location and scatter estimates are determined by a subset selected for joint multivariate concentration. The determinant criterion makes that target precise, while the subset size, possible ties, and rank of candidate subsets remain essential parts of its interpretation.

Check Your Understanding

Answer each question by referring to the fixed-size subset definition and its determinant criterion.

For a sample in $\mathbb{R}^3$, what is the smallest subset size that can possibly yield a full-rank covariance matrix?
Why does changing the covariance divisor from $h$ to $h-1$ leave the minimizing subset unchanged when $h$ is fixed?
In the five-point example, which subsets minimize the determinant, and what is their common determinant?
Why is the determinant of a covariance matrix connected to ellipsoid volume?
Can a subset of size $p+1$ have a singular covariance matrix? Explain using the geometry of its points.

The Minimum Covariance Determinant Estimator

What You'll Learn

Why Minimize a Covariance Determinant?

Determinant and Ellipsoid Volume

Worked Example: Exhaustive Selection in a Bivariate Sample

Worked Example: Exhaustive Selection in a Bivariate Sample

A Formula for Three Points in Two Dimensions

Worked Example: Comparing the Full-Sample Covariance

Worked Example: Comparing the Full-Sample Covariance

Worked Example: A Singular Subset Can Win

Worked Example: A Singular Subset Can Win

Choosing and Interpreting the Subset Size

Common Mistakes and Practical Meaning

Check Your Understanding

Previous

Next