Affine Equivariance Of Location And Scatter

Why Affine Equivariance Matters

A multivariate data set can be expressed in different coordinate systems without changing the underlying observations. One analyst might measure length in centimeters and another in inches; a third might rotate the axes or use linear combinations of the original measurements. A location estimate should respond to such a change by moving and rescaling with the data. A scatter estimate should change its orientation and scale in the corresponding way.

This requirement is called affine equivariance. It is a structural property of a procedure, not a claim that its output stays numerically unchanged. The distinction matters in multivariate analysis: a method that depends on the chosen axes can give different conclusions after a harmless change of units or coordinates.

Definition: Let $P$ be a probability distribution on $\mathbb{R}^p$, and let $A$ be a nonsingular $p\times p$ matrix and $b\in\mathbb{R}^p$. Write $P^{A,b}$ for the distribution of $AX+b$ when $X\sim P$. A location functional $T$ is affine equivariant if $$ T(P^{A,b})=A T(P)+b. $$ A scatter functional $S$ is affine equivariant if $$ S(P^{A,b})=A S(P)A^T. $$ Here location takes values in $\mathbb{R}^p$, while scatter takes values in the symmetric positive-definite $p\times p$ matrices whenever the functional is defined.

The matrix $A$ allows rotations, changes of scale, and nonsingular linear mixing of coordinates. The vector $b$ translates every observation. The two transformation laws reflect the different mathematical roles of location and scatter: a vector transforms by $A$ and translation, while a covariance-like matrix transforms by multiplication on both sides.

Deriving the Transformation Laws

The ordinary mean and covariance provide the basic model for these laws. Suppose $X$ has finite second moments, mean $\mu=E[X]$, and covariance matrix $\Sigma=E[(X-\mu)(X-\mu)^T]$. For $Y=AX+b$, linearity of expectation gives

E[Y]=E[AX+b]=A\mu+b.

Also, $Y-E[Y]=A(X-\mu)$. Substituting this centered vector into the definition of covariance gives the scatter transformation directly.

\begin{aligned} \operatorname{Cov}(Y) &=E\!\left[(Y-E[Y])(Y-E[Y])^T\right]\\ &=E\!\left[A(X-\mu)(X-\mu)^T A^T\right]\\ &=A\,E\!\left[(X-\mu)(X-\mu)^T\right]A^T\\ &=A\Sigma A^T. \end{aligned}

The passage of the fixed matrices $A$ and $A^T$ outside the expectation is valid entry by entry; finite second moments ensure the expectations exist. Nonsingularity of $A$ preserves positive definiteness: for any nonzero $v$, $A^Tv\ne 0$, so $v^T A\Sigma A^T v=(A^Tv)^T\Sigma(A^Tv)>0$ when $\Sigma$ is positive definite.

Theorem: If $X$ has finite second moments, then its mean and covariance satisfy $$ E[AX+b]=AE[X]+b,\qquad \operatorname{Cov}(AX+b)=A\operatorname{Cov}(X)A^T. $$ If $\operatorname{Cov}(X)$ is positive definite and $A$ is nonsingular, the transformed covariance is positive definite as well.

The identities are not limited to the mean and covariance. They define the desired behavior for robust location and scatter functionals too, including functionals that are not expectations and may be defined for distributions without finite second moments. In that setting, the transformation laws are requirements on the functionals themselves; they do not follow merely from covariance algebra.

Worked Example: Transforming a Location and Scatter Pair

Suppose a distribution has location $t=(1,-1)^T$ and scatter $$ S=\begin{pmatrix}2&1\\1&2\end{pmatrix}. $$ Apply the affine transformation with $$ A=\begin{pmatrix}1&1\\0&1\end{pmatrix}, \qquad b=\begin{pmatrix}2\\0\end{pmatrix}. $$ The matrix $A$ is nonsingular because its determinant is $1$.

The transformed location is $$ At+b =\begin{pmatrix}1&1\\0&1\end{pmatrix} \begin{pmatrix}1\\-1\end{pmatrix} +\begin{pmatrix}2\\0\end{pmatrix} =\begin{pmatrix}0\\-1\end{pmatrix} +\begin{pmatrix}2\\0\end{pmatrix} =\begin{pmatrix}2\\-1\end{pmatrix}. $$

For scatter, first calculate $$ AS= \begin{pmatrix}1&1\\0&1\end{pmatrix} \begin{pmatrix}2&1\\1&2\end{pmatrix} =\begin{pmatrix}3&3\\1&2\end{pmatrix}, \qquad A^T=\begin{pmatrix}1&0\\1&1\end{pmatrix}. $$ Then $$ ASA^T= \begin{pmatrix}3&3\\1&2\end{pmatrix} \begin{pmatrix}1&0\\1&1\end{pmatrix} =\begin{pmatrix}6&3\\3&2\end{pmatrix}. $$ The result is symmetric, and its determinant is $6\cdot2-3\cdot3=3>0$, with positive upper-left entry, so it is positive definite. The shear has changed the covariance between the coordinates as well as their marginal scales; simply retaining the old diagonal entries would not give the transformed scatter.

Equivariance Versus Invariance

An equivariant output changes in a prescribed way when the input is transformed. An invariant output stays the same. For example, a location vector is equivariant rather than invariant: translation by $b$ must translate the estimate by $b$. Some scalar summaries derived from a location-scatter pair can be invariant. The squared Mahalanobis distance between a point $x$ and location $t$ is $$ d^2(x;t,S)=(x-t)^T S^{-1}(x-t), $$ where $S$ is positive definite. Under an affine transformation, both the point and the estimates transform, but this scalar distance does not.

To verify the claim, set $x'=Ax+b$, $t'=At+b$, and $S'=ASA^T$. Since $A$ is nonsingular, $$(ASA^T)^{-1}=A^{-T}S^{-1}A^{-1},$$ as is checked by multiplying this expression by $ASA^T$. Therefore $$ \begin{aligned} (x'-t')^T(S')^{-1}(x'-t') &=(A(x-t))^T A^{-T}S^{-1}A^{-1}A(x-t)\\ &=(x-t)^T S^{-1}(x-t). \end{aligned} $$ The affine transformation changes the vector and matrix estimates, but preserves this scalar comparison.

Worked Example: Checking an Invariant Distance

Take $S=\operatorname{diag}(4,1)$, $t=(0,0)^T$, and $x=(2,1)^T$. The squared Mahalanobis distance is $$ (x-t)^T S^{-1}(x-t) =\begin{pmatrix}2&1\end{pmatrix} \begin{pmatrix}1/4&0\\0&1\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} =1+1=2. $$

Now use $A=\operatorname{diag}(2,1/2)$ and $b=(3,-1)^T$. Then $x'=(7,-1/2)^T$, $t'=(3,-1)^T$, so $x'-t'=(4,1/2)^T$. The transformed scatter is $$ S'=ASA^T =\operatorname{diag}(2,1/2)\operatorname{diag}(4,1)\operatorname{diag}(2,1/2) =\operatorname{diag}(16,1/4). $$ Consequently, $$ (x'-t')^T(S')^{-1}(x'-t') =\frac{4^2}{16}+\frac{(1/2)^2}{1/4} =1+1=2. $$ Both terms have been checked using the corresponding transformed coordinate and variance. The distance agrees with its original value.

What Equivariance Requires of an Estimator

For data $x_1,\ldots,x_n$, let $\hat t(x_1,\ldots,x_n)$ and $\hat S(x_1,\ldots,x_n)$ be location and scatter estimates. Their sample-level affine equivariance means that for every nonsingular $A$ and every $b$, $$ \hat t(Ax_1+b,\ldots,Ax_n+b)=A\hat t(x_1,\ldots,x_n)+b, $$ and $$ \hat S(Ax_1+b,\ldots,Ax_n+b)=A\hat S(x_1,\ldots,x_n)A^T. $$ These equations apply to the estimator as a function of the full data set. For instance, the sample mean and the sample covariance with divisor $n$, or with divisor $n-1$ when $n>1$, obey these equations whenever the covariance estimate is defined. The same algebra as for the distributional mean and covariance verifies both identities.

Worked Example: A Coordinatewise Median Fails Under a Shear

For three bivariate observations, define the coordinatewise median by taking the ordinary median separately in each coordinate. Consider $$ x_1=(0,0)^T,\qquad x_2=(1,10)^T,\qquad x_3=(2,0)^T. $$ The first-coordinate values are $0,1,2$, and the second-coordinate values are $0,10,0$. Their medians are $1$ and $0$, so the estimated location is $\hat t=(1,0)^T$.

Apply the shear $A=\begin{pmatrix}1&0\\1&1\end{pmatrix}$ with $b=0$, which maps $(x,y)^T$ to $(x,x+y)^T$. The transformed observations are $(0,0)^T,(1,11)^T,(2,2)^T$. Their coordinatewise median is $(1,2)^T$, since the second-coordinate values in order are $0,2,11$.

But transforming the original estimate gives $$ A\hat t =\begin{pmatrix}1&0\\1&1\end{pmatrix} \begin{pmatrix}1\\0\end{pmatrix} =\begin{pmatrix}1\\1\end{pmatrix}, $$ which is not $(1,2)^T$. Thus this coordinatewise median is not affine equivariant. Its failure is not about a numerical approximation: the coordinatewise operation depends on the selected axes, and the shear changes which observations determine the second-coordinate median.

Equivariance therefore constrains how an estimator may combine coordinates. Applying a scalar robust location rule separately to each coordinate does not, in general, respect arbitrary linear mixing. Such a rule may still respect some restricted transformations, such as separate changes of units, but that weaker property should not be confused with full affine equivariance.

Standardization as a Construction Principle

A common way to build affine-equivariant procedures is to separate the problem into a location, a scatter, and standardized coordinates. The following calculation explains the key constraint. Suppose $T_0(P)$ and $S_0(P)$ are an affine-equivariant location and positive-definite scatter functional. Use the symmetric positive-definite square root $S_0(P)^{1/2}$, and let $R_P$ denote the distribution of $$ S_0(P)^{-1/2}(X-T_0(P)),\qquad X\sim P. $$ After an affine transformation, write $T_0'=AT_0+b$ and $S_0'=AS_0A^T$. Define $$ Q=(S_0')^{-1/2}A S_0^{1/2}. $$ Then $$ QQ^T=(S_0')^{-1/2}A S_0 A^T(S_0')^{-1/2}=I, $$ so $Q$ is orthogonal. The standardized transformed observation is $$ (S_0')^{-1/2}(AX+b-T_0') =Q S_0^{-1/2}(X-T_0). $$ Thus standardization removes the translation and scale changes, leaving only an orthogonal transformation.

For example, suppose a location correction $g(R)$ on standardized distributions is orthogonally equivariant, meaning $g(Q_{\#}R)=Qg(R)$, where $Q_{\#}R$ is the distribution of $QZ$ when $Z\sim R$. Then $$ T(P)=T_0(P)+S_0(P)^{1/2}g(R_P) $$ is affine equivariant. Indeed, for the transformed distribution, $$ \begin{aligned} T(P^{A,b}) &=T_0'+(S_0')^{1/2}g(Q_{\#}R_P)\\ &=AT_0+b+(S_0')^{1/2}Qg(R_P)\\ &=AT_0+b+A S_0^{1/2}g(R_P)\\ &=A T(P)+b. \end{aligned} $$ The third line follows from $(S_0')^{1/2}Q=A S_0^{1/2}$, which follows directly from the definition of $Q$. This argument shows why an orthogonally well-behaved procedure on standardized data can be lifted to an affine-equivariant location procedure.

Key takeaway: Affine equivariance is a compatibility condition between a statistical procedure and changes of coordinates. Location must transform as $AT+b$; scatter must transform as $ASA^T$. Coordinate-dependent constructions can violate these laws, while standardization reduces the construction problem to controlling orthogonal transformations.

Common Pitfall

A frequent error is to say that an affine-equivariant estimate is “unchanged” when the data are transformed. That would assert invariance, not equivariance. For a translation $b$, a location estimate must acquire $b$; for a linear map $A$, it must be multiplied by $A$. Likewise, scatter is not generally unchanged: its transformed value is $ASA^T$. Only particular scalar summaries, such as the Mahalanobis distance when the point, location, and scatter are all transformed consistently, remain invariant.

Check Your Understanding

Use the transformation laws to answer each question.

State the affine transformation laws for a location functional and a scatter functional.
If $Y=AX+b$ and $X$ has finite second moments, derive $E[Y]$ and $\operatorname{Cov}(Y)$.
Why does a nonsingular $A$ preserve positive definiteness of a transformed scatter matrix $ASA^T$?
For the three-point example, calculate the coordinatewise median after applying the shear and compare it with the transformed original median.
Explain the difference between an affine-equivariant vector estimate and an affine-invariant scalar summary.
In the standardization argument, what property of $Q=(S_0')^{-1/2}AS_0^{1/2}$ allows orthogonal equivariance to be used?