Comparing Robust Estimators Under Point Contamination

Why Compare Estimators on the Same Contaminated Sample?

Robustness is not a single numerical property. Under point contamination, a location estimate may move, a scatter estimate may expand or contract, and the estimated scatter ellipse may change orientation. Comparing several procedures on the same data makes these effects visible. It also guards against a tempting but incomplete conclusion: one estimator is not automatically preferable because it has the smallest scatter determinant.

We will use a small bivariate sample with eight central observations and one extreme observation. The central observations occur twice each at \((1,0),(-1,0),(0,1),(0,-1)\); the contaminating point is \((10,10)\). The origin is the uncontaminated center. We compare estimates by their Euclidean location displacement from that center and by the determinant of their scatter matrix.

The comparison is deliberately finite-sample and descriptive. We specify each estimator, its tuning or normalization, and its reference point. The determinant depends on the scale calibration, so the numerical comparison is meaningful only with the stated definitions; it is not a universal ranking of estimator families.

The Sample and the Classical Estimates

There are \(n=9\) observations. The sum of the eight central observations is \((0,0)\), and their coordinatewise sums of squares are both \(4\). The outlier contributes \((10,10)\), \(100\) to each squared-coordinate sum, and \(100\) to the cross-product sum. Thus the sample mean is \(\bar{x}=(10/9,10/9)\).

For the classical scatter estimate, use the sample covariance with divisor \(n\), rather than \(n-1\), so it has the same maximum-likelihood normalization convention used for the subset covariance below. The raw sums of products form the matrix \(\sum_i x_i x_i^{\mathsf T}=\begin{pmatrix}104&100\\100&104\end{pmatrix}\). Centering at the mean subtracts \(n\bar{x}\bar{x}^{\mathsf T}\), whose entries are \(100/9\). Therefore:

The location displacement is \(\sqrt{(10/9)^2+(10/9)^2}=10\sqrt{2}/9\), approximately \(1.571\). The scatter determinant is:

A Specified M Location-Scatter Pair

An M-estimator is defined through an estimating equation or objective, with its score function and tuning constants stated explicitly. To make this calculation reproducible, use a componentwise Huber location estimate with score \(\psi_2(r)=\max(-2,\min(r,2))\). Each coordinate solves \(\sum_i\psi_2(x_{ij}-\hat{\mu}_{Mj})=0\). For this comparison, define the associated scatter estimate separately by clipping each coordinate residual at \(1\) and using the stated normalization \(1/36\). The two constants serve different parts of this specified location-scatter pair.

For either coordinate, the sample values are \(1\) twice, \(-1\) twice, \(0\) four times, and \(10\) once. At \(\hat{\mu}_{Mj}=1/4\), the location scores sum to \(2(3/4)+2(-5/4)+4(-1/4)+2=0\). All eight central residuals and the outlier residual are in the appropriate linear or clipped regions of the score. Since the sum of Huber scores is nonincreasing in the candidate location and strictly decreasing in a neighborhood of \(1/4\), this root is the unique solution. Hence \(\hat{\mu}_M=(1/4,1/4)\), with displacement \(\sqrt{(1/4)^2+(1/4)^2}=\sqrt{1/8}\approx0.354\).

For scatter, let \(r_i=x_i-\hat{\mu}_M\), and clip each residual coordinate to \([-1,1]\), writing the resulting vector as \(w_i\). For the first coordinate, the clipped residual contributions to the sum of squares are \(2(9/16)\) from the two \((1,0)\) observations, \(2\) from the two \((-1,0)\) observations, \(4(1/16)\) from the four observations with first coordinate zero, and \(1\) from the outlier. The total is \(17/4\). The second-coordinate sum is the same by symmetry. The cross-product sum is \(5/4\): the two positive-axis groups contribute \(-3/8+1/2\), the two corresponding groups on the other axis contribute \(-3/8+1/2\), and the clipped outlier contributes \(1\). Thus:

MCD and a Projection-Based Estimate

For the minimum covariance determinant (MCD) estimate here, take \(h=8\): among all eight-point subsets, select one with minimum covariance determinant, where the subset covariance uses divisor \(h\). The central eight-point subset has mean \((0,0)\), covariance \(\operatorname{diag}(1/2,1/2)\), and determinant \(1/4\). To verify that the outlier does not give a better subset, an eight-point subset containing it must omit one central point. Direct calculation of the centered subset covariance gives determinant \(1243/128\) when the omitted point is \((1,0)\) or \((0,1)\), and \(1569/128\) when it is \((-1,0)\) or \((0,-1)\). Each is greater than \(1/4\), so the central subset is the minimizing choice. Therefore \(\hat{\mu}_{\mathrm{MCD}}=(0,0)\), its displacement is \(0\), and \(\det(\hat{\Sigma}_{\mathrm{MCD}})=1/4\).

For a transparent projection-based comparison, use the two coordinate directions \(e_1=(1,0)\) and \(e_2=(0,1)\). Set the location to the coordinatewise sample median, which is \((0,0)\). In each projected coordinate, winsorize values to \([-1,1]\), then form the average outer product of the paired winsorized coordinates over all nine observations. This is a finite-direction projection-winsorized estimate, not a claim about every projection-pursuit procedure. The eight central observations contribute \(2\) to each diagonal sum and zero to the cross-product sum; the winsorized outlier \((1,1)\) contributes \(1\) to all three. Thus:

Its location displacement is zero. The projection-based scatter is smaller in determinant than the MCD scatter in this example, but it is not the same estimate: the MCD procedure selects the central subset, whereas this specified projection procedure retains all observations after winsorization.

Putting the Numbers in Context

The table makes several distinct effects clear. The classical estimate is strongly affected by the extreme point. Both robust location estimates centered at the origin have zero displacement here, but their scatter constructions differ. The specified Huber M pair has a displaced location and a determinant smaller than the MCD determinant; the projection-winsorized estimate has the smallest determinant in this comparison. These are properties of the chosen definitions on this sample, not general ordering results.

Common Pitfall: Treating the Smallest Determinant as the Winner

A common mistake is to reason, “The projection estimate has the smallest determinant, so it is the best estimate.” That conclusion does not follow. A determinant summarizes overall ellipse volume, not whether the ellipse has the right orientation or whether a location-scatter model matches the scientific target. A small determinant can result from aggressive clipping or a particular calibration. Likewise, zero location displacement does not imply that two methods have the same scatter estimate.

The correction is to interpret the numbers alongside the estimator definitions and the intended use. Here the projection estimate’s determinant is \(8/81\), while the MCD determinant is \(1/4\), but the two procedures handle the outlier differently. Neither number alone assesses uncertainty, sampling variability, or performance under a different contamination pattern. Those are inferential questions rather than conclusions available from this single descriptive comparison.