Projection Outlyingness For Multivariate Data

Why Look in Every Direction?

A multivariate point can look ordinary on each coordinate separately and still be unusual when its coordinates are considered together. Conversely, a large value on one coordinate may be unsurprising if the data are highly variable in that direction. Projection outlyingness addresses both issues by comparing a point with the data after projecting onto every one-dimensional direction, then retaining its largest standardized deviation.

The standardization matters: directions with different amounts of spread should not be compared using raw distances. Using robust univariate location and scale in each projection also limits the effect of extreme observations on the reference against which a point is assessed. The result is a diagnostic score for a point, not by itself a procedure for estimating multivariate location or scatter.

Definition and Robust Projection Scores

Definition: Let $X$ be a random vector in $\mathbb{R}^p$, with distribution $F$. For a unit direction $u$, let $F_u$ be the distribution of the scalar projection $u'X$. Let $T$ be a univariate location functional and $S$ a univariate scale functional. When $S(F_u)>0$ and the relevant quantities are finite, the projection outlyingness of $x\in\mathbb{R}^p$ is $$ O(x;F)=\sup_{\|u\|=1} \frac{|u'x-T(F_u)|}{S(F_u)}. $$ The ratio inside the supremum is the standardized deviation in direction $u$. If the ratio is continuous in $u$ on the unit sphere, the supremum is attained and can be written as a maximum.

For data $x_1,\ldots,x_n$, replace $F_u$ by the empirical distribution of the projected values $z_i(u)=u'x_i$. A common robust choice is the sample median for $T$, and the normal-calibrated median absolute deviation for $S$:

\widehat T_u=\operatorname{median}_{i}(z_i(u)),\qquad \widehat S_u=c_{\mathrm{MAD}}\operatorname{median}_{i}|z_i(u)-\widehat T_u|, \qquad c_{\mathrm{MAD}}=\frac{1}{\Phi^{-1}(0.75)}\approx 1.4826.

Here $\Phi$ is the standard normal distribution function. The calibration makes the MAD consistent for the standard deviation under a normal distribution; it does not make the scale estimate nonrobust. For an even number of values, this tutorial uses the conventional sample median obtained by averaging the two middle order statistics. Other robust univariate location and scale choices can be used, but the choice and any scale calibration must be stated.

If a projected sample has zero MAD, the corresponding standardized ratio may be undefined. When its numerator is nonzero, it is natural to regard that direction as indicating unbounded outlyingness; when both numerator and denominator are zero, the ratio needs an explicit convention or a different scale estimate. Thus a finite, useful score requires positive projected scales in the directions being considered.

Worked Calculations

Worked Example: A Spherical Normal Reference

Suppose $X\sim N(\mu,4I_2)$, where $\mu=(1,1)'$. Find the population projection outlyingness of $x=(4,5)'$, using the normal-calibrated MAD for projected scale.

For every unit vector $u$, the projection $u'X$ is normal with median $u'\mu$ and standard deviation $2$. Its raw median absolute deviation is $2\Phi^{-1}(0.75)$, so the calibrated scale is $c_{\mathrm{MAD}}\,2\Phi^{-1}(0.75)=2$. Therefore

O(x;F)=\sup_{\|u\|=1}\frac{|u'(x-\mu)|}{2} =\frac{\|x-\mu\|}{2} =\frac{\sqrt{3^2+4^2}}{2} =2.5.

The largest deviation is in the direction of $x-\mu=(3,4)'$. This calculation shows why examining only the two coordinate directions would not generally suffice: the direction of greatest standardized deviation can be an oblique combination of coordinates.

Worked Example: An Exact Score for a Symmetric Four-Point Sample

Consider the four observed points $(1,0)',(-1,0)',(0,1)',(0,-1)'$. Calculate the sample projection outlyingness of the observation $x=(1,0)'$, using the median and normal-calibrated MAD.

Write a unit direction as $u=(u_1,u_2)'$, so $u_1^2+u_2^2=1$. The projected values are $u_1,-u_1,u_2,-u_2$. Their median is zero: after sorting, the two middle values are opposites or both zero, and their average is zero. The four absolute deviations are $|u_1|,|u_1|,|u_2|,|u_2|$. Their median, using the average of the two middle values, is $(|u_1|+|u_2|)/2$. Hence

\widehat S_u =c_{\mathrm{MAD}}\frac{|u_1|+|u_2|}{2}, \qquad \frac{|u'x-\widehat T_u|}{\widehat S_u} =\frac{2|u_1|}{c_{\mathrm{MAD}}(|u_1|+|u_2|)}.

The fraction $|u_1|/(|u_1|+|u_2|)$ is at most $1$, with equality when $u_2=0$. Thus the maximum over directions is attained along the first coordinate axis, and

O(x;\{x_i\})=\frac{2}{c_{\mathrm{MAD}}} \approx \frac{2}{1.4826} \approx 1.349.

The score is not required to be large merely because $x$ is an observation in the sample. It describes how far the point lies from projected robust centers relative to projected robust scales.

Worked Example: Anisotropic Normal Data

Let $X\sim N(\mu,\Sigma)$, with $\mu=(1,0)'$ and $\Sigma=\operatorname{diag}(4,1)$. Using normal-calibrated projected MAD scales, calculate the outlyingness of $x=(4,2)'$.

For any unit vector $u=(u_1,u_2)'$, the projected median is $u'\mu$ and the calibrated projected scale is $\sqrt{u'\Sigma u}=\sqrt{4u_1^2+u_2^2}$. Since $x-\mu=(3,2)'$, the directional ratio is

\frac{|3u_1+2u_2|}{\sqrt{4u_1^2+u_2^2}}.

The maximum is attained in a direction proportional to $\Sigma^{-1}(x-\mu)=(3/4,2)'$. Substituting a unit vector is unnecessary because multiplying both components by the same positive constant leaves the ratio unchanged. Evaluating with $(3/4,2)'$ gives

\frac{(3/4)3+2(2)} {\sqrt{4(3/4)^2+2^2}} = \frac{25/4}{\sqrt{25/4}} =\frac{25/4}{5/2} =2.5.

The first coordinate has twice the variance of the second, so raw Euclidean distance would not account for the directional spread. Projection standardization does.

Two Useful Properties

Theorem: Suppose $A$ is a nonsingular $p\times p$ matrix, $b\in\mathbb{R}^p$, and the univariate location and scale functionals satisfy $T(\mathcal{L}(aZ+c))=aT(\mathcal{L}(Z))+c$ and $S(\mathcal{L}(aZ+c))=|a|S(\mathcal{L}(Z))$ for $a\ne0$. If $Y=AX+b$ and the projection scales are positive, then $$ O(Ax+b;F_Y)=O(x;F_X). $$

This is affine invariance: changing coordinates by an invertible linear transformation and translating the origin does not change the score, provided the univariate location and scale definitions have the stated equivariance properties.

Proof. Fix a unit direction $v$ in the transformed coordinates. Its projection is $v'Y=(A'v)'X+v'b$. Because $A$ is nonsingular, $A'v\ne0$. Set $w=A'v/\|A'v\|$, another unit direction. Then $v'Y=\|A'v\|w'X+v'b$. By the assumed location and scale properties, the transformed directional numerator and denominator are respectively $\|A'v\|$ times the original numerator in direction $w$, and $\|A'v\|$ times the original scale in that direction. Their ratio is therefore the ratio for $X$ in direction $w$. As $v$ ranges over the unit sphere, $w=A'v/\|A'v\|$ ranges over the whole unit sphere: for any unit $w$, choose $v$ proportional to $(A')^{-1}w$ and normalize it. Thus the two suprema are equal. $\square$

Result: If $X\sim N(\mu,\Sigma)$, with positive-definite $\Sigma$, and the projected scales are the true normal standard deviations, then $$ O(x;F)=\sqrt{(x-\mu)'\Sigma^{-1}(x-\mu)}. $$

Derivation. For every nonzero $u$, the projected standard deviation is $\sqrt{u'\Sigma u}$. Put $\delta=x-\mu$. The squared directional ratio is $(u'\delta)^2/(u'\Sigma u)$. By the Cauchy–Schwarz inequality applied to $\Sigma^{1/2}u$ and $\Sigma^{-1/2}\delta$,

(u'\delta)^2 =\big((\Sigma^{1/2}u)'(\Sigma^{-1/2}\delta)\big)^2 \le (u'\Sigma u)(\delta'\Sigma^{-1}\delta).

So every directional ratio is at most $\sqrt{\delta'\Sigma^{-1}\delta}$. If $\delta\ne0$, equality holds for $u$ proportional to $\Sigma^{-1}\delta$, since then the two vectors in the Cauchy–Schwarz inequality are proportional. If $\delta=0$, every numerator is zero and both sides are zero. Taking the supremum proves the result. $\square$

For normal data, the robust population median and normal-calibrated MAD produce these same projected location and scale values. The resulting score is therefore the Mahalanobis distance in this model. Outside that setting, projection outlyingness remains defined through the chosen robust univariate functionals and need not equal a Mahalanobis distance.

Computation, Interpretation, and a Common Pitfall

In dimensions greater than one, evaluating the definition means searching over directions. A finite grid of directions is often useful, but its largest score is only a lower bound on the true supremum: an unexamined direction might produce a larger standardized deviation. The score also depends on the selected robust univariate location and scale procedures, the finite-sample median convention, and the handling of zero projected scales. These choices should be reported when scores are used in an analysis.

A common mistake is to calculate the standardized deviations only along the coordinate axes and call their maximum projection outlyingness. That calculation can miss a point whose greatest standardized deviation lies in an oblique direction. The correction is to optimize over all unit directions, or to say clearly that a finite directional search gives an approximation rather than the exact score. Another mistake is to use raw projected deviations: a deviation of the same size is more unusual in a direction with small robust scale than in one with large robust scale.

Key takeaway: Projection outlyingness is the supremum of robustly standardized one-dimensional deviations. It compares directions on a common scale, is affine invariant under the stated equivariance conditions, and reduces to Mahalanobis distance for a normal model with its true projected standard deviations.

Check Your Understanding

Answer these questions using the definitions and calculations above.

What feature of projection outlyingness makes it different from checking deviations along coordinate axes alone?
Why is the factor $c_{\mathrm{MAD}}$ included when the target model is normal?
In the four-point example, why does the direction $u=(1,0)'$ attain the largest standardized deviation for $x=(1,0)'$?
State the conditions on location and scale that support affine invariance of the score.
Why can the largest score on a finite grid of directions underestimate projection outlyingness?