Contamination Neighborhoods And Robustness Functionals

Why Contamination Must Be Described at Two Levels

A small fraction of unusual observations can have two different effects. It can change the population quantity being studied, and it can change how an estimator behaves on a particular sample. Those are related but distinct questions. A method may be stable under unusual observations while estimating a different location target from the one specified by the scientific question.

Gross-error contamination gives a precise distribution-level description of a small corrupted or atypical fraction. Statistical functionals then map each distribution to a target such as a location vector or scatter matrix. Studying how that map changes clarifies what “robust target” means; it does not, by itself, establish that a particular finite-sample procedure is robust.

Contamination Neighborhoods and Functionals

Definition: Let $F$ be a probability distribution on $\mathbb{R}^p$. Its gross-error contamination neighborhood of size $\varepsilon$, for $0\leq\varepsilon<1$, is $$ \mathcal{U}_{\varepsilon}(F) = \left\{(1-\varepsilon)F+\varepsilon G: G\text{ is a probability distribution on }\mathbb{R}^p \right\}. $$ A distribution $F_{\varepsilon,G}=(1-\varepsilon)F+\varepsilon G$ represents a fraction $1-\varepsilon$ from $F$ and a fraction $\varepsilon$ from an otherwise unrestricted contaminating distribution $G$.

The distribution $G$ need not be far from $F$: it may describe a subtle shift, an unusually dispersed group, or probability concentrated at an extreme point. “Unrestricted” refers to its form, not to whether the functional is defined. For example, the mean requires a finite first moment, while covariance requires a finite second moment. When discussing those functionals, the contamination distributions must satisfy the corresponding moment condition.

Definition: A statistical functional $T$ is a specified map from a class of probability distributions to a parameter space. A location functional $T_{\mathrm{loc}}(F)$ takes values in $\mathbb{R}^p$; a scatter functional $T_{\mathrm{sc}}(F)$ takes values in the symmetric matrices, usually with the additional requirement that the result be positive definite. The pair $(T_{\mathrm{loc}}(F),T_{\mathrm{sc}}(F))$ is a location-scatter functional.

A functional describes a population target, not an algorithm. The mean functional, for instance, assigns a vector to a distribution; a sample mean is an estimator obtained by applying that functional to an empirical distribution. The choice of functional determines what “location” or “scatter” means. It is therefore important to identify the target before judging an estimator’s performance.

What Contamination Does to the Mean and Covariance

For a distribution $F$ with finite second moments, write its mean as $\mu_F$ and covariance matrix as $\Sigma_F$. Let $G$ also have finite second moments, with mean $\mu_G$ and covariance matrix $\Sigma_G$. The contaminated distribution has mean and covariance

\mu_{\varepsilon}=(1-\varepsilon)\mu_F+\varepsilon\mu_G, \qquad \Sigma_{\varepsilon} =(1-\varepsilon)\Sigma_F+\varepsilon\Sigma_G +\varepsilon(1-\varepsilon)(\mu_F-\mu_G)(\mu_F-\mu_G)^{\mathsf T}.

The final covariance term records the separation between the component means. Thus, even if the contaminating group has no internal spread, separating it from the main distribution can inflate the overall scatter.

Result: The mean of a mixture is the mixture of its component means, and its covariance equals the weighted average of the component covariances plus the between-component covariance term displayed above.

Proof. Let $X$ have distribution $F_{\varepsilon,G}$. Its expectation, by the mixture definition, is $$ E[X]=(1-\varepsilon)\mu_F+\varepsilon\mu_G=\mu_{\varepsilon}. $$ For the covariance, condition on which component generated $X$. In the $F$ component, write $X-\mu_{\varepsilon}=(X-\mu_F)+(\mu_F-\mu_{\varepsilon})$. The cross terms have expectation zero because $E_F[X-\mu_F]=0$. The corresponding contribution is therefore $\Sigma_F+(\mu_F-\mu_{\varepsilon})(\mu_F-\mu_{\varepsilon})^{\mathsf T}$. The $G$ component contributes $\Sigma_G+(\mu_G-\mu_{\varepsilon})(\mu_G-\mu_{\varepsilon})^{\mathsf T}$. Weighting these terms by their component probabilities gives $$ \begin{aligned} \operatorname{Cov}(X) ={}&(1-\varepsilon)\Sigma_F+\varepsilon\Sigma_G\\ &+(1-\varepsilon)(\mu_F-\mu_{\varepsilon})(\mu_F-\mu_{\varepsilon})^{\mathsf T} +\varepsilon(\mu_G-\mu_{\varepsilon})(\mu_G-\mu_{\varepsilon})^{\mathsf T}. \end{aligned} $$ Since $\mu_F-\mu_{\varepsilon}=\varepsilon(\mu_F-\mu_G)$ and $\mu_G-\mu_{\varepsilon}=-(1-\varepsilon)(\mu_F-\mu_G)$, the last two terms combine to $\varepsilon(1-\varepsilon)(\mu_F-\mu_G)(\mu_F-\mu_G)^{\mathsf T}$. This proves the formula. $\square$

Worked Example: A Small Contaminating Group Shifts the Mean and Scatter

Let $F=N_2((0,0),I_2)$, and let the contaminating distribution be a point mass $G=\delta_{(10,0)}$, with $\varepsilon=0.1$. Find the mean and covariance of the contaminated distribution.

The component means are $\mu_F=(0,0)^{\mathsf T}$ and $\mu_G=(10,0)^{\mathsf T}$. The component covariance matrices are $\Sigma_F=I_2$ and $\Sigma_G=0$. Hence $$ \mu_{\varepsilon} =0.9(0,0)^{\mathsf T}+0.1(10,0)^{\mathsf T} =(1,0)^{\mathsf T}. $$ The covariance formula gives $$ \begin{aligned} \Sigma_{\varepsilon} &=0.9I_2+0.1(0) +0.1(0.9) \begin{pmatrix}-10\\0\end{pmatrix} \begin{pmatrix}-10&0\end{pmatrix}\\ &= \begin{pmatrix}0.9&0\\0&0.9\end{pmatrix} + \begin{pmatrix}9&0\\0&0\end{pmatrix} = \begin{pmatrix}9.9&0\\0&0.9\end{pmatrix}. \end{aligned} $$ The contaminating group comprises only one tenth of the mixture, yet it moves the mean and makes the variance in the first coordinate much larger. The increase in scatter comes from the separation of component means, not from spread within $G$.

Worked Example: A Contaminating Group Can Inflate Scatter Without Shifting Location

Let $F=N_2((0,0),I_2)$, let $G=N_2((0,0),\operatorname{diag}(25,1))$, and take $\varepsilon=0.1$. Calculate the contaminated mean and covariance.

Both component means are zero, so the mixture mean is $$ \mu_{\varepsilon}=0.9(0,0)^{\mathsf T}+0.1(0,0)^{\mathsf T}=(0,0)^{\mathsf T}. $$ Because the component means agree, the between-component term is zero. Thus $$ \begin{aligned} \Sigma_{\varepsilon} &=0.9I_2+0.1\begin{pmatrix}25&0\\0&1\end{pmatrix}\\ &=\begin{pmatrix}0.9+2.5&0\\0&0.9+0.1\end{pmatrix} =\begin{pmatrix}3.4&0\\0&1\end{pmatrix}. \end{aligned} $$ Here the mean target is unchanged, while the scatter target changes substantially in the first coordinate. Robustness of location and robustness of scatter are separate questions.

Target Robustness Is Not the Same as Estimator Robustness

A useful way to describe stability of a target is to examine its change throughout a contamination neighborhood. For a location functional, for example, consider $$ B_F(\varepsilon) = \sup_{G\in\mathcal{G}} \left\|T_{\mathrm{loc}}(F_{\varepsilon,G})-T_{\mathrm{loc}}(F)\right\|, $$ where $\mathcal{G}$ is the stated class of contaminating distributions for which the functional is defined. If $B_F(\varepsilon)$ is small for small $\varepsilon$, the target is uniformly stable against that class at $F$. The class and the size of the perturbation must be stated: a claim of stability is incomplete if it leaves possible contaminants unspecified.

For the mean, take $F$ with mean zero and let $G=\delta_{a v}$, where $v$ is a unit vector and $a$ is any positive number. Then $$ \mu_{\varepsilon}=\varepsilon a v, \qquad \|\mu_{\varepsilon}-\mu_F\|=\varepsilon a. $$ For any fixed $\varepsilon>0$, this displacement has no finite upper bound as $a$ increases. This does not say that every fixed contaminating distribution causes a large change when $\varepsilon$ is small. It says that the mean is not uniformly stable over unrestricted contaminating locations. The analogous covariance problem is visible in the between-component term: its magnitude can grow with the squared separation of the component means.

Worked Example: Adding One Extreme Observation to an Empirical Distribution

Consider four observations in $\mathbb{R}^2$: $(1,0),(-1,0),(0,1),(0,-1)$. Their empirical distribution $F_4$ has mean $(0,0)^{\mathsf T}$. Add the observation $(20,0)$, and find the mean of the resulting empirical distribution.

The new empirical distribution is exactly the contamination mixture $$ F_5=\frac{4}{5}F_4+\frac{1}{5}\delta_{(20,0)}. $$ Applying the mean functional, $$ \mu_{F_5} =\frac{4}{5}(0,0)^{\mathsf T}+\frac{1}{5}(20,0)^{\mathsf T} =(4,0)^{\mathsf T}. $$ One added observation changes the sample mean from $(0,0)$ to $(4,0)$. The calculation is a finite-sample illustration of the same functional response to a contamination fraction of $1/5$; it does not, on its own, characterize every property of an estimator.

An estimator is a data-based rule $\hat{\theta}_n$; a functional is a distribution-based target $T(F)$. For an empirical distribution $F_n$, the plug-in estimator is $T(F_n)$, when that expression is defined. If $T$ is the mean, this plug-in rule is the sample mean. Its behavior under unusual observations reflects the behavior of the mean functional, while finite-sample behavior also depends on the data configuration and the estimator’s construction.

Robustness Depends on Which Target Is Intended

A stable location rule is not automatically a robust way to estimate every location parameter. For example, a coordinatewise median targets the marginal medians, whereas the mean targets the vector of marginal expectations when those expectations exist. These targets coincide under some distributions, including many centrally symmetric models, but need not agree in general. Substituting one for the other changes the estimand unless the scientific model or question justifies that change.

Worked Example: A Stable Location Target Need Not Equal the Mean

Let a bivariate distribution put probability $0.6$ at $(0,0)$, $0.2$ at $(10,0)$, and $0.2$ at $(0,10)$. Find its mean and its coordinatewise marginal medians.

The mean is $$ \mu_F =0.6(0,0)^{\mathsf T} +0.2(10,0)^{\mathsf T} +0.2(0,10)^{\mathsf T} =(2,2)^{\mathsf T}. $$ For the first coordinate, probability $0.8$ is at zero and probability $0.2$ is at 10, so zero is its unique median: at least half the probability is at or below zero and at least half is at or above zero. The same reasoning applies to the second coordinate. Thus the coordinatewise median vector is $(0,0)^{\mathsf T}$, not the mean. An estimator aimed at that median cannot be described as estimating the mean merely because both are called “location.”

When a procedure appears robust, ask two questions separately: Which functional is its population target? And how does the estimator behave when the observed sample is contaminated? A method can have a stable target but a poorly behaved implementation, or it can be stable in a sample-level sense while converging to a target that differs from the intended parameter. Establishing estimator-specific properties requires analyzing the estimator itself, beyond defining the contamination neighborhood.

Common Mistakes

Assuming a small fraction guarantees a small effect. The mean shift is $\varepsilon(\mu_G-\mu_F)$; an unrestricted contaminating mean can make it large even when $\varepsilon$ is small.
Ignoring between-group separation in scatter. A contaminating distribution with zero internal covariance can still greatly increase mixture covariance if its mean is far from the main component.
Calling a target and an estimator the same object. The functional is defined on distributions; an estimator is a rule applied to data. Their robustness questions are connected but not identical.
Changing the target without saying so. Replacing a mean by a median may improve stability, but it changes the parameter being estimated unless the targets coincide under the model.
Using a robustness claim without specifying the contaminants. Stability over a bounded class of perturbations is different from stability over all distributions in a gross-error neighborhood.

Key takeaway: A contamination neighborhood describes how the data-generating distribution may be perturbed; a functional describes the target assigned to each distribution. Robustness requires attention to both the target’s response and the estimator’s behavior, with the intended location or scatter parameter made explicit.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

What does the parameter $\varepsilon$ represent in a gross-error contamination neighborhood?
Why can a point-mass contaminant increase covariance even though its own covariance matrix is zero?
For a mean-zero distribution contaminated by a point mass at $a v$, where $v$ is a unit vector, what is the contaminated mean?
In what sense is robustness of a statistical functional different from robustness of an estimator?
Why might using a coordinatewise median instead of a mean change the scientific target?