Gross-Error Sensitivity and Local Robustness

Contamination and First-Order Effect

An estimator can behave well under a specified probability model yet react sharply to a small amount of data from elsewhere. A useful way to quantify that reaction is to replace a fraction of a model distribution by a point mass at a possible contaminating value. The derivative of the resulting target with respect to the contamination fraction describes its first-order response.

This derivative is the influence function. Its largest magnitude over possible contamination points is the gross-error sensitivity. A bounded influence function limits the first-order effect of a single extreme value; an unbounded one means that this first-order effect can grow without bound as the contaminating value moves farther away.

Definition: Let \(T(P)\) be a real-valued statistical functional, and let \(P\) be the distribution at which it is evaluated. For a point \(z\), write \(\Delta_z\) for the distribution placing probability one at \(z\). The point-contamination path is \(P_{\varepsilon,z}=(1-\varepsilon)P+\varepsilon\Delta_z\), for \(0\leq\varepsilon<1\). If the right derivative exists, the influence function is \( \operatorname{IF}(z;T,P)=\left.\frac{d}{d\varepsilon}T(P_{\varepsilon,z})\right|_{\varepsilon=0+}\).

The derivative is taken from the right because a contamination fraction cannot be negative. The functional \(T\) represents the target of the estimator at a distribution; it need not be the estimator computed from one finite sample. Gross-error sensitivity, for a scalar functional, is \(\gamma^*(T,P)=\sup_z|\operatorname{IF}(z;T,P)|\). For a vector-valued functional, specify a norm \(\|\cdot\|\) and use \(\sup_z\|\operatorname{IF}(z;T,P)\|\). Boundedness depends on the chosen norm in the vector case, though all norms on a fixed finite-dimensional space give the same bounded-versus-unbounded distinction.

Influence From an Estimating Equation

Suppose \(T(P)\) is defined locally by an estimating equation. Write \(\Psi(P,\theta)=E_P[\psi(X,\theta)]\), where \(X\) has distribution \(P\), and suppose \(\Psi(P,\theta_0)=0\) with \(\theta_0=T(P)\). The equation for the contaminated distribution is \((1-\varepsilon)\Psi(P,\theta)+\varepsilon\psi(z,\theta)=0\). To differentiate its solution, the contaminating term must have the regularity needed for that differentiation; regularity of \(\Psi\) alone is not enough.

Theorem: Suppose \(\theta\in\mathbb R^p\), \(\Psi(P,\theta_0)=0\), and \(\Psi(P,\cdot)\) is continuously differentiable near \(\theta_0\), with nonsingular Jacobian \(A=\partial\Psi(P,\theta)/\partial\theta^\top|_{\theta=\theta_0}\). For a fixed \(z\), suppose \(\psi(z,\cdot)\) is continuous near \(\theta_0\), and the contaminated equation has a solution branch \(\theta_\varepsilon\) with \(\theta_\varepsilon\to\theta_0\). If that branch is differentiable from the right at zero, then \(\operatorname{IF}(z;T,P)=-A^{-1}\psi(z,\theta_0)\). A sufficient alternative regularity condition is that \(G(\varepsilon,\theta)=(1-\varepsilon)\Psi(P,\theta)+\varepsilon\psi(z,\theta)\) is continuously differentiable near \((0,\theta_0)\); then the implicit function theorem gives the local differentiable branch.

Here \(A\) is the derivative of the population estimating equation with respect to the parameter. In particular, the formula does not follow just from differentiability of \(\Psi(P,\cdot)\): the value of the contaminating term must also behave appropriately as \(\theta\) changes, or the solution branch and its derivative must be assumed directly.

Proof. Set \(G(\varepsilon,\theta)=(1-\varepsilon)\Psi(P,\theta)+\varepsilon\psi(z,\theta)\). Along the stated solution branch, \(G(\varepsilon,\theta_\varepsilon)=0\). Differentiate from the right at \(\varepsilon=0\), using \(\theta_0=T(P)\) and \(\Psi(P,\theta_0)=0\):

0=-\Psi(P,\theta_0)+A\left.\frac{d\theta_\varepsilon}{d\varepsilon}\right|_{0+} +\psi(z,\theta_0) =A\left.\frac{d\theta_\varepsilon}{d\varepsilon}\right|_{0+} +\psi(z,\theta_0).

Since \(A\) is nonsingular, solving gives \(\left.d\theta_\varepsilon/d\varepsilon\right|_{0+}=-A^{-1}\psi(z,\theta_0)\), as claimed. Under the alternative continuously differentiable condition, the implicit function theorem applies because \(G(0,\theta_0)=0\) and its Jacobian in \(\theta\) there is \(A\). \(\square\)

For each fixed \(z\), differentiability also gives the local expansion \(T(P_{\varepsilon,z})=\theta_0+\varepsilon\operatorname{IF}(z;T,P)+o(\varepsilon)\) as \(\varepsilon\downarrow0\). Thus the influence function quantifies first-order bias relative to \(\theta_0\). The remainder in this expansion is pointwise in \(z\); bounded influence alone does not guarantee a remainder uniform over all contaminating points.

Worked Example: The Mean Has Unbounded Influence

Let \(T(P)=E_P[X]\), assuming the mean \(\mu\) exists. The mean under point contamination is

T(P_{\varepsilon,z})=(1-\varepsilon)\mu+\varepsilon z =\mu+\varepsilon(z-\mu).

Therefore \(\operatorname{IF}(z;T,P)=z-\mu\), and \(\gamma^*(T,P)=\sup_z|z-\mu|=\infty\). For \(P=N(0,1)\), the baseline mean is \(0\). Contamination at \(z=10\) with \(\varepsilon=0.01\) changes the mean exactly to \(0.01(10)=0.1\); the first-order prediction is also \(0.01(10)=0.1\). As \(z\) grows, the effect per unit of contamination grows without bound.

A Bounded-Influence Example: The Median

The median provides a contrasting location functional. Let \(q_p(P)\) be the \(p\)-quantile, defined as \(\inf\{x:F(x)\geq p\}\), where \(0<p<1\). Suppose \(F\) is continuous near \(q=q_p(P)\), \(F(q)=p\), and has a continuous, strictly positive density \(f\) near \(q\). For a fixed contaminating point \(z<q\), the contaminated quantile lies near \(q\) and above \(z\) for sufficiently small \(\varepsilon\). Its defining equation is \((1-\varepsilon)F(q_p(P_{\varepsilon,z}))+\varepsilon=p\). Differentiating this equation at zero gives derivative \(-(1-p)/f(q)\).

For \(z>q\), the contaminating point lies above the quantile for sufficiently small \(\varepsilon\), and the equation is \((1-\varepsilon)F(q_p(P_{\varepsilon,z}))=p\). Its derivative is \(p/f(q)\). If \(z=q\), the contaminated distribution still has \(q\) as its \(p\)-quantile: its left limit at \(q\) is \((1-\varepsilon)p<p\), and its value at \(q\) is \((1-\varepsilon)p+\varepsilon\geq p\). Hence the right derivative at that particular point is zero.

Result: Under the stated density conditions, the point-contamination influence function of the \(p\)-quantile is \(\operatorname{IF}(z;q_p,P)=-(1-p)/f(q)\) for \(z<q\), \(p/f(q)\) for \(z>q\), and \(0\) for \(z=q\). Consequently, its gross-error sensitivity is \(\gamma^*(q_p,P)=\max(p,1-p)/f(q)\).

The value at \(z=q\) follows from the quantile definition along the actual point-contamination path; it is not obtained by substituting \(z=q\) into either neighboring branch. This distinction matters for the exact pointwise influence function, although it does not affect the supremum above.

Worked Example: Median Under a Standard Normal Model

Let \(P=N(0,1)\) and take \(p=1/2\). Symmetry gives \(q=0\), and the standard normal density there is \(f(0)=1/\sqrt{2\pi}\). For \(z<0\), the influence is \(-0.5/f(0)\); for \(z>0\), it is \(0.5/f(0)\); and at \(z=0\), it is zero. Therefore

\gamma^*(q_{1/2},P)=\frac{1/2}{1/\sqrt{2\pi}} =\frac{\sqrt{2\pi}}{2}\approx 1.2533.

For a contaminant at \(z=2\), the first-order change at \(\varepsilon=0.01\) is \(0.01(0.5/f(0))\approx0.01253\). The calculation illustrates bounded first-order effect: moving \(z\) farther above zero does not increase this derivative. It does not say that every finite-sample median changes by exactly this amount.

Unbounded Influence for a Scale Functional

Sensitivity is not limited to location. Consider the variance functional \(T(P)=\operatorname{Var}_P(X)\), with finite second moment, mean \(\mu\), and variance \(\sigma^2\). The mixture has mean \(\mu_\varepsilon=(1-\varepsilon)\mu+\varepsilon z\) and second moment \(m_{2,\varepsilon}=(1-\varepsilon)E_P[X^2]+\varepsilon z^2\). Since variance is second moment minus squared mean,

\begin{aligned} T(P_{\varepsilon,z}) &=(1-\varepsilon)(\sigma^2+\mu^2)+\varepsilon z^2 -\bigl(\mu+\varepsilon(z-\mu)\bigr)^2\\ &=\sigma^2+\varepsilon\bigl((z-\mu)^2-\sigma^2\bigr) -\varepsilon^2(z-\mu)^2. \end{aligned}

The coefficient of \(\varepsilon\) is the influence function. It grows quadratically in the distance from \(\mu\), so the variance has unbounded gross-error sensitivity.

Worked Example: Variance Under a Standard Normal Model

For \(P=N(0,1)\), \(\mu=0\) and \(\sigma^2=1\), so \(\operatorname{IF}(z;T,P)=z^2-1\). At \(z=3\), the influence is \(9-1=8\); at \(z=10\), it is \(100-1=99\). With \(\varepsilon=0.01\) and \(z=3\), the exact contaminated variance is \(1+0.01(8)-0.01^2(9)=1.0791\), while the first-order approximation is \(1+0.01(8)=1.08\). The difference, \(0.0009\), is precisely the second-order term in the expansion.

Interpreting Gross-Error Sensitivity

Gross-error sensitivity is a local, worst-case summary: it asks how large the derivative can be over contamination points. If it is finite, then for each fixed \(z\) the first-order change is bounded by \(\varepsilon\gamma^*(T,P)\). If it is infinite, arbitrarily extreme points can have arbitrarily large first-order effects. This is a statement about the derivative at the model distribution, not a complete guarantee about behavior at a non-negligible contamination fraction.

The examples show why the choice of target and estimating equation matters. The mean responds in proportion to distance, while the median’s influence under a positive-density model is bounded. Variance responds quadratically to distance. Bounded influence is therefore one useful feature when extreme observations are plausible, but it should be considered alongside the estimator’s target, assumptions, and sampling variability.

Common Pitfall

A tempting but incorrect argument is: “The population equation \(\Psi(P,\theta)\) is differentiable in \(\theta\), so the implicit function theorem immediately gives the influence formula.” This overlooks the other term in the contaminated equation, \(\psi(z,\theta)\). If that term is discontinuous or otherwise irregular in \(\theta\), the contaminated equation need not satisfy the hypotheses needed to differentiate its solution, even when \(\Psi(P,\theta)\) is smooth.

The correction is to check the regularity of the full contaminated equation \(G(\varepsilon,\theta)\), or to state directly that the solution branch is differentiable and that the contaminating term is continuous at the baseline parameter. Under those conditions the derivative calculation is justified; without them, the displayed estimating-equation formula cannot simply be assumed.

Key takeaway: The influence function is the derivative of a functional along a point-contamination path. Its supremum in magnitude is gross-error sensitivity: finite sensitivity means bounded first-order response, while unbounded sensitivity permits arbitrarily large first-order effects from extreme contaminating points.

Check Your Understanding

Use the definitions and calculations above to answer the following questions.

For a mean functional with \(E_P[X]=\mu\), derive its influence function directly from the contaminated mean.
Why does differentiability of \(\Psi(P,\theta)\) alone not guarantee the estimating-equation influence formula?
For the median of \(N(0,1)\), what is the influence at \(z=-4\), and what is the gross-error sensitivity?
For the variance under \(N(0,1)\), calculate the influence at \(z=4\). Is the gross-error sensitivity finite?
What does bounded gross-error sensitivity say about first-order effects, and what does it not establish about finite contamination?