Confidence Ellipsoids for Robust Location Estimates

Why a Robust Location Estimate Needs an Ellipsoid

A multivariate location estimate is a vector, so uncertainty about it is also multivariate. Reporting a separate interval for each coordinate can hide dependence between coordinates and does not describe the joint uncertainty region. A confidence ellipsoid uses the estimator’s covariance structure to show both the scale of uncertainty and its orientation.

For robust inference, the covariance in that calculation must describe the sampling variation of the robust location estimator. It is generally a sandwich covariance, not simply the robust scatter estimate used to summarize the observations. The inferential target also matters: if the estimator targets a location functional under a specified distribution or contamination model, the region is for that functional.

The formula requires \(\hat V\) to be invertible. In the usual fixed-dimension asymptotic setting, assume that \(p\) is fixed, the estimator has the stated asymptotic normal distribution, and \(\hat V\xrightarrow{p}V\) for a positive-definite \(V\). Then the ellipsoid is a large-sample region; it is not automatically an exact finite-sample confidence region.

From Sandwich Covariance to the Region

A common route to a robust estimator’s covariance is an estimating equation. Suppose \(\hat{\theta}\) solves \(\sum_{i=1}^n\psi(Z_i,\theta)=0\), where the observations \(Z_i\) are independent and identically distributed, and \(\psi\) is a \(p\)-vector. Under regularity conditions that justify linearization and a central limit theorem, define \(A=E[\partial\psi(Z,\theta)/\partial\theta^\mathsf{T}]\) and \(B=E[\psi(Z,\theta)\psi(Z,\theta)^\mathsf{T}]\), evaluated at the target. If \(A\) is nonsingular and \(B\) is finite, the asymptotic covariance is \(V=A^{-1}BA^{-\mathsf{T}}\). A consistent sandwich estimate substitutes suitable sample estimates of these components. The exact form of the estimating equation and its regularity conditions depend on the estimator.

This distinction is important for robust location methods, including M-estimators and the MCD. Their robust scatter estimates describe a robust notion of data spread, while the sandwich covariance describes repeated-sample variation of the location estimate. The two matrices need not be equal. Use a covariance estimate calibrated for the estimator and inferential target at hand.

Coverage: Why the Chi-Square Cutoff Works

The ellipsoid’s coverage follows from the asymptotic distribution of the estimator and consistency of its covariance estimate. The chi-square cutoff is for the joint \(p\)-dimensional distance, not for each coordinate separately.

Proof. Let \(Z_n=\sqrt n(\hat{\theta}-\theta)\). By the assumed convergence, \(Z_n\xrightarrow{d}Z\), where \(Z\sim N_p(0,V)\). Since \(V\) is positive definite and \(\hat V\xrightarrow{p}V\), matrix inversion is continuous at \(V\), so \(\hat V^{-1}\xrightarrow{p}V^{-1}\). Slutsky’s theorem and the continuous mapping theorem give \[ Z_n^\mathsf{T}\hat V^{-1}Z_n \xrightarrow{d} Z^\mathsf{T}V^{-1}Z. \] Write \(Z=V^{1/2}U\), where \(U\sim N_p(0,I_p)\). Then \(Z^\mathsf{T}V^{-1}Z=U^\mathsf{T}U\), which has a chi-square distribution with \(p\) degrees of freedom. This distribution has a continuous cumulative distribution function at \(\chi^2_{p,1-\alpha}\). Consequently, the probability that the quadratic form is at most this cutoff converges to \(1-\alpha\), as claimed. \(\square\)

Axes, Scale, and Orientation

The geometry is easiest to read from an eigendecomposition. If \(\hat V=Q\Lambda Q^\mathsf{T}\), where \(Q\) is orthogonal and \(\Lambda=\operatorname{diag}(\hat\lambda_1,\ldots,\hat\lambda_p)\) has positive eigenvalues, then the ellipsoid’s principal directions are the columns of \(Q\). Its semi-axis length in direction \(j\) is \(\sqrt{\chi^2_{p,1-\alpha}\hat\lambda_j/n}\). Thus larger eigenvalues correspond to greater estimated uncertainty in their respective directions. Increasing \(n\) shortens every axis at rate \(n^{-1/2}\); changing the confidence level changes all axes through the chi-square quantile.

Linear Contrasts and Ellipsoid Projections

A linear contrast has the form \(c^\mathsf{T}\theta\) for a specified vector \(c\). Projecting the entire ellipsoid onto this contrast gives the interval \[ c^\mathsf{T}\hat{\theta} \ \pm\ \sqrt{\frac{\chi^2_{p,1-\alpha}}{n}\,c^\mathsf{T}\hat Vc}. \] This is the range of contrast values attained by points in the full ellipsoid. It is not generally the same as the contrast’s own Wald confidence interval, which uses the chi-square quantile with one degree of freedom. The projected interval inherits the full ellipsoid’s joint cutoff and is generally wider.

Derivation. Put \(d=\theta_0-\hat{\theta}\). The ellipsoid constraint is \(d^\mathsf{T}\hat V^{-1}d\leq q/n\). Write \(d=\hat V^{1/2}u\), so \(u^\mathsf{T}u\leq q/n\). Then \(c^\mathsf{T}d=(\hat V^{1/2}c)^\mathsf{T}u\). By the Cauchy–Schwarz inequality, \[ |c^\mathsf{T}d| \leq \sqrt{c^\mathsf{T}\hat Vc}\sqrt{u^\mathsf{T}u} \leq \sqrt{\frac{q}{n}c^\mathsf{T}\hat Vc}. \] If \(c\ne0\), equality is attained by choosing \(u\) parallel or antiparallel to \(\hat V^{1/2}c\) with norm \(\sqrt{q/n}\). Adding \(c^\mathsf{T}\hat{\theta}\) gives the stated endpoints. For \(c=0\), the contrast is identically zero and the same formula gives a zero-width interval. \(\square\)

More generally, for a full-row-rank matrix \(R\) with \(r\) rows, the contrast vector \(R\theta\) has asymptotic covariance factor \(RVR^\mathsf{T}\). The image of the \(p\)-dimensional ellipsoid under \(R\) uses the original \(p\)-degree-of-freedom cutoff. A contrast-specific Wald region uses an \(r\)-degree-of-freedom cutoff. This distinction matters when relating a joint confidence region to tests of linear restrictions, as in robust Wald inference.

Common Pitfalls and Interpretation

• Using robust scatter as if it were the estimator’s covariance. A robust scatter matrix and the sandwich covariance of a robust location estimate have different inferential roles. Use a consistent covariance estimate for the estimator and target.
• Using the wrong chi-square degrees of freedom. A joint ellipsoid for a \(p\)-vector uses \(p\) degrees of freedom. A test or confidence region for \(r\) independent restrictions uses \(r\), not automatically \(p\).
• Calling the region exact. The chi-square calibration above gives asymptotic coverage under its assumptions. Small samples, poor covariance estimation, or a poor normal approximation can affect actual coverage.
• Ignoring the target under contamination. Robustness does not by itself make an estimator target the uncontaminated population location. Interpret the ellipsoid as uncertainty about the functional that the estimation procedure targets under the stated model.

A confidence ellipsoid is useful when uncertainty in several location coordinates must be assessed jointly, when correlation among coordinates matters, or when the orientation of uncertainty is scientifically meaningful. Its shape summarizes estimated precision; its coverage claim depends on the estimator’s asymptotic distribution and a suitable sandwich covariance estimate.