Whitening Gaussian Vectors and Mahalanobis Geometry

Why Whitening Reveals Gaussian Geometry

A Gaussian vector can have coordinates with different scales and nonzero correlations. Its mean describes the center, while its covariance determines how the distribution spreads and in which directions. Whitening changes coordinates to remove both scale differences and correlations. In those new coordinates, the covariance is the identity matrix, so the geometry becomes ordinary Euclidean geometry.

This transformation also clarifies the meaning of Mahalanobis distance: it measures how far a point is from the mean after accounting for the covariance structure. The result is a distance that treats a one-standard-deviation displacement consistently in every direction, rather than measuring displacement in the original coordinate units.

Whitening a Nonsingular Gaussian Vector

Definition: Let \(X\in\mathbb{R}^p\) have a multivariate normal distribution \(N_p(\mu,\Sigma)\), where \(\Sigma\) is positive definite. A whitening transformation is a matrix \(A\) satisfying \(A\Sigma A^T=I_p\). The transformed vector \(Z=A(X-\mu)\) is then called a whitened version of \(X\). A common choice is the symmetric whitener \(A=\Sigma^{-1/2}\).

Positive definiteness is the nonsingularity condition needed here: all eigenvalues of \(\Sigma\) are strictly positive, so \(\Sigma^{-1}\) exists. By the spectral theorem, write \(\Sigma=Q\Lambda Q^T\), where \(Q\) is orthogonal and \(\Lambda\) is diagonal with positive entries \(\lambda_1,\ldots,\lambda_p\). Define

\Sigma^{1/2}=Q\Lambda^{1/2}Q^T,\qquad \Sigma^{-1/2}=Q\Lambda^{-1/2}Q^T.

These are symmetric matrices, and \(\Sigma^{-1/2}\Sigma\Sigma^{-1/2}=I_p\). Whitening is not limited to this symmetric choice. If \(\Sigma=LL^T\) is a Cholesky factorization, then \(A=L^{-1}\) also whitens because

L^{-1}\Sigma L^{-T} =L^{-1}LL^TL^{-T} =I_p.

Theorem: If \(X\sim N_p(\mu,\Sigma)\) with positive-definite \(\Sigma\), then \(Z=\Sigma^{-1/2}(X-\mu)\sim N_p(0,I_p)\). More generally, every square matrix \(A\) satisfying \(A\Sigma A^T=I_p\) gives \(A(X-\mu)\sim N_p(0,I_p)\).

Proof. The density of \(X\) is

f_X(x)=\frac{1}{(2\pi)^{p/2}|\Sigma|^{1/2}} \exp\left\{-\frac12(x-\mu)^T\Sigma^{-1}(x-\mu)\right\}.

For the symmetric whitener, the inverse transformation is \(x=\mu+\Sigma^{1/2}z\), whose Jacobian determinant in absolute value is \(|\Sigma|^{1/2}\). Also,

(x-\mu)^T\Sigma^{-1}(x-\mu) =z^T\Sigma^{1/2}\Sigma^{-1}\Sigma^{1/2}z =z^Tz.

The change-of-variables formula therefore gives

f_Z(z)=f_X(\mu+\Sigma^{1/2}z)|\Sigma|^{1/2} =\frac{1}{(2\pi)^{p/2}}\exp\left(-\frac12z^Tz\right),

which is the density of \(N_p(0,I_p)\). For any other square \(A\) with \(A\Sigma A^T=I_p\), the transformed vector is Gaussian because it is an affine transformation of a Gaussian vector; its mean is zero and its covariance is \(A\Sigma A^T=I_p\). A Gaussian distribution is determined by its mean and covariance, so it too is \(N_p(0,I_p)\). \(\square\)

Worked Whitening Calculations

Worked Example: Cholesky Whitening With Correlated Coordinates

Suppose \(X\sim N_2(\mu,\Sigma)\), with \(\mu=(1,-1)^T\) and \(\Sigma=\begin{pmatrix}4&2\\2&2\end{pmatrix}\). Whiten the observed point \(x=(3,2)^T\) using a Cholesky factor, and verify its squared Mahalanobis distance.

A Cholesky factor is \(L=\begin{pmatrix}2&0\\1&1\end{pmatrix}\), since \(LL^T=\begin{pmatrix}4&2\\2&2\end{pmatrix}\). The centered observation is \(x-\mu=(2,3)^T\). To find \(z=L^{-1}(x-\mu)\), solve \(Lz=(2,3)^T\):

2z_1=2,\qquad z_1+z_2=3, \qquad\text{so}\qquad z=(1,2)^T.

Its squared Euclidean length is \(z^Tz=1^2+2^2=5\). Directly, \(|\Sigma|=4\) and \(\Sigma^{-1}=\frac14\begin{pmatrix}2&-2\\-2&4\end{pmatrix}\). Thus

(x-\mu)^T\Sigma^{-1}(x-\mu) =(2,3)\frac14 \begin{pmatrix}2&-2\\-2&4\end{pmatrix} \begin{pmatrix}2\\3\end{pmatrix} =5,

as required. The two original coordinates were correlated; the Cholesky transformation turns this particular centered observation into coordinates whose squared length is exactly its covariance-adjusted squared distance.

Mahalanobis Distance as Euclidean Distance

Definition: For a positive-definite covariance matrix \(\Sigma\), the squared Mahalanobis distance of \(x\in\mathbb{R}^p\) from a center \(\mu\) is \(d_M^2(x,\mu)=(x-\mu)^T\Sigma^{-1}(x-\mu)\). Its Mahalanobis distance is the nonnegative square root \(d_M(x,\mu)\).

The definition is directly connected to whitening. With \(z=\Sigma^{-1/2}(x-\mu)\),

d_M^2(x,\mu) =(x-\mu)^T\Sigma^{-1}(x-\mu) =z^Tz=\|z\|_2^2.

Thus Mahalanobis distance from the mean is Euclidean distance from the origin in whitened coordinates. More generally, for two points \(x\) and \(y\), define \(d_M(x,y)=\{(x-y)^T\Sigma^{-1}(x-y)\}^{1/2}\). Applying the same linear transformation to both gives \(d_M(x,y)=\|\Sigma^{-1/2}(x-y)\|_2\). Because \(\Sigma\) is positive definite, this is a genuine distance: it is nonnegative, is zero only when \(x=y\), is symmetric, and satisfies the triangle inequality inherited from the Euclidean norm.

Theorem: The squared Mahalanobis distance is unchanged by the choice of whitening matrix. If \(A_1\Sigma A_1^T=I_p\) and \(A_2\Sigma A_2^T=I_p\), then \(\|A_1(x-\mu)\|_2^2=\|A_2(x-\mu)\|_2^2=d_M^2(x,\mu)\).

Proof. Every such square matrix \(A_i\) is invertible, since \(A_i\Sigma A_i^T=I_p\) has full rank. Set \(Q=A_2A_1^{-1}\). From \(A_1\Sigma A_1^T=I_p\), it follows that \(\Sigma=A_1^{-1}A_1^{-T}\). Therefore

QQ^T=A_2A_1^{-1}A_1^{-T}A_2^T =A_2\Sigma A_2^T=I_p.

So \(Q\) is orthogonal and \(A_2=QA_1\). Orthogonal transformations preserve Euclidean length, hence

\|A_2(x-\mu)\|_2^2 =\|QA_1(x-\mu)\|_2^2 =\|A_1(x-\mu)\|_2^2.

Taking \(A_1=\Sigma^{-1/2}\) identifies this common squared length with \(d_M^2(x,\mu)\). \(\square\)

Worked Example: Two Whiteners, One Distance

Let \(\Sigma=\begin{pmatrix}2&1\\1&2\end{pmatrix}\) and consider the centered vector \(v=(1,1)^T\). The eigenvalue in the direction \((1,1)^T\) is \(3\), so the symmetric whitener maps \(v\) to \(z_{\mathrm{sym}}=(1/\sqrt{3},1/\sqrt{3})^T\). Its squared length is \(2/3\).

For comparison, a Cholesky factor is \(L=\begin{pmatrix}\sqrt{2}&0\\1/\sqrt{2}&\sqrt{3/2}\end{pmatrix}\). Solving \(Lz_{\mathrm{chol}}=v\) gives \(z_{\mathrm{chol}}=(1/\sqrt{2},1/\sqrt{6})^T\), whose squared length is \(1/2+1/6=2/3\). The coordinate vectors differ, but their lengths agree. Indeed, \(\Sigma^{-1}=\frac13\begin{pmatrix}2&-1\\-1&2\end{pmatrix}\), and \(v^T\Sigma^{-1}v=2/3\). The difference between whitened coordinates is an orthogonal change of orientation, not a change in Mahalanobis distance.

Covariance Ellipsoids Become Spheres

A set of points at a fixed Mahalanobis distance \(c\) from \(\mu\) satisfies \((x-\mu)^T\Sigma^{-1}(x-\mu)=c^2\). Under symmetric whitening, this equation becomes \(\|z\|_2^2=c^2\), the equation of a sphere. In the original coordinates it is generally an ellipsoid: its orientation and axis lengths reflect the eigenvectors and eigenvalues of \(\Sigma\).

Worked Example: Reading a Mahalanobis Ellipse

Let \(X\sim N_2(0,\Sigma)\) with \(\Sigma=\begin{pmatrix}9&0\\0&4\end{pmatrix}\), and examine \(x=(3,2)^T\). The symmetric whitener is \(\Sigma^{-1/2}=\begin{pmatrix}1/3&0\\0&1/2\end{pmatrix}\), so the whitened point is \(z=(1,1)^T\). Therefore

d_M^2(x,0)=1^2+1^2=2 =\frac{3^2}{9}+\frac{2^2}{4}.

The set at this same squared distance is \(x_1^2/9+x_2^2/4=2\). It is an ellipse with horizontal and vertical semiaxis lengths \(3\sqrt{2}\) and \(2\sqrt{2}\). Whitening maps it to the unit-radius-\(\sqrt{2}\) circle. In the Gaussian density, the exponent at \(x\) is \(-d_M^2(x,0)/2=-1\); this distance describes the density contour, not by itself the probability of being inside that contour.

Common Pitfall and Why the Geometry Matters

A frequent error is to standardize each coordinate separately and assume that the resulting vector is whitened. That procedure divides by marginal standard deviations, but does not remove correlation. For example, if \(\Sigma=\begin{pmatrix}4&2\\2&2\end{pmatrix}\), dividing the coordinates by their standard deviations produces covariance \(\begin{pmatrix}1&1/\sqrt{2}\\1/\sqrt{2}&1\end{pmatrix}\), not \(I_2\). The off-diagonal entries remain nonzero. A full matrix transformation, such as \(L^{-1}\) or \(\Sigma^{-1/2}\), is needed to remove both marginal scale and dependence.

Another error is to use \(\Sigma\) in place of \(\Sigma^{-1}\) in the distance formula. The inverse is essential: directions with high variance count less per unit of displacement, while directions with low variance count more. For instance, with diagonal covariance \(\operatorname{diag}(9,4)\), a displacement of \(3\) along the first coordinate contributes \(3^2/9=1\), not \(9\), to squared Mahalanobis distance.

Whitening is useful whenever a problem becomes simpler in standardized, uncorrelated coordinates. In this tutorial the central geometric consequence is that covariance-adjusted distances, and therefore Gaussian density contours, can be understood with ordinary Euclidean lengths after transformation. The choice of symmetric or Cholesky whitening affects the coordinates, but not the distance they represent.

Check Your Understanding

Use the definitions and derivations above to answer these questions.

What condition on \(\Sigma\) guarantees that the symmetric whitening transformation exists as an invertible matrix?
If \(A\Sigma A^T=I_p\), what are the mean and covariance of \(A(X-\mu)\) when \(X\sim N_p(\mu,\Sigma)\)?
Explain why two valid whitening transformations can produce different coordinate vectors but the same squared Mahalanobis distance.
For \(\Sigma=\operatorname{diag}(16,1)\), what is the squared Mahalanobis distance of the centered point \((4,1)^T\)?
Why does dividing each coordinate by its own standard deviation fail to whiten a vector with nonzero correlations?