Affine Standardization of Multivariate Densities

Why Standardize a Multivariate Distribution?

Centering and scaling put coordinates on a common reference scale: a centered value is measured relative to its mean, and a standardized value is measured in standard-deviation units. For a random vector, however, transforming the coordinates also transforms the joint density and its support. The density must change by the correct determinant factor so that total probability remains one.

The key distinction is between standardizing each coordinate and removing all dependence among coordinates. Coordinate-by-coordinate standardization makes each marginal mean zero and variance one when those moments exist and the variances are positive. It does not, by itself, make the coordinates independent or make their joint distribution normal.

Affine Density Transformation

Definition: Let $X$ be a random vector in $\mathbb{R}^d$ with Lebesgue density $f_X$, and let $Y=A(X-c)$, where $c\in\mathbb{R}^d$ and $A$ is a nonsingular $d\times d$ matrix. Then $X=c+A^{-1}Y$, and a density of $Y$ is $$ f_Y(y)=\frac{1}{|\det A|}f_X(c+A^{-1}y). $$ The support is transformed by the same map: if $X$ has density zero outside a set $S$, then $Y$ has density zero outside $A(S-c)$, apart from any choices of support on boundary sets of probability zero.

The absolute value is essential: a determinant can be negative when a transformation reverses orientation, but a density cannot acquire a negative factor. Equivalently, the inverse map $x=c+A^{-1}y$ has Jacobian determinant with absolute value $1/|\det A|$. A transformation that expands volume by a factor $|\det A|$ therefore lowers density by that factor.

Theorem: Under the conditions in the definition, the displayed formula is a density for $Y$. For any Borel set $B\subseteq\mathbb{R}^d$, $$ P(Y\in B)=\int_B \frac{1}{|\det A|}f_X(c+A^{-1}y)\,dy. $$

Proof. Since $Y=A(X-c)$, the event $Y\in B$ is equivalent to $X\in c+A^{-1}B$. By the definition of a density and the change-of-variables formula for the nonsingular affine map $y=A(x-c)$, $$ P(Y\in B)=\int_{c+A^{-1}B} f_X(x)\,dx =\int_B f_X(c+A^{-1}y)\frac{1}{|\det A|}\,dy. $$ This identity for every Borel set $B$ establishes the claimed density. Taking $B=\mathbb{R}^d$ also shows that its integral is one, since it equals $P(Y\in\mathbb{R}^d)=1$. $\square$

Coordinate-by-Coordinate Standardization

Suppose $X=(X_1,\ldots,X_d)^\mathsf{T}$ has finite coordinate means $\mu_i=E[X_i]$ and finite, positive coordinate variances $\sigma_i^2=\operatorname{Var}(X_i)$. Define $D$ to be the diagonal matrix with entries $\sigma_1,\ldots,\sigma_d$. The standardized vector is $Z=D^{-1}(X-\mu)$, where $\mu=(\mu_1,\ldots,\mu_d)^\mathsf{T}$. Its coordinates are $Z_i=(X_i-\mu_i)/\sigma_i$.

Because each $\sigma_i$ is positive, $D$ is nonsingular and $|\det D|=\prod_{i=1}^d\sigma_i$. The affine density formula gives the direct expression below.

Standardization formula: If $X$ has density $f_X$, then $$ f_Z(z)=\left(\prod_{i=1}^d\sigma_i\right) f_X\bigl(\mu_1+\sigma_1z_1,\ldots,\mu_d+\sigma_dz_d\bigr). $$ The support of $Z$ is obtained by applying $D^{-1}(x-\mu)$ to the support of $X$.

The factor here is the product of the standard deviations, not its reciprocal. In the general affine formula the multiplier is $1/|\det A|$; for standardization $A=D^{-1}$, so $1/|\det A|=|\det D|=\prod_i\sigma_i$. Equivalently, the inverse transformation from $z$ back to $x$ expands volume by $\prod_i\sigma_i$.

Worked Example: Standardizing a Uniform Rectangle

Let $X_1$ and $X_2$ be independent, with $X_1$ uniform on $(-1,3)$ and $X_2$ uniform on $(0,6)$. Find the joint density after standardizing each coordinate.

The joint density is $1/(4\cdot 6)=1/24$ on the rectangle $(-1,3)\times(0,6)$. A uniform variable on an interval of width $w$ has variance $w^2/12$. Thus $E[X_1]=1$, $\operatorname{Var}(X_1)=16/12=4/3$, and $\sigma_1=2/\sqrt{3}$. Similarly, $E[X_2]=3$, $\operatorname{Var}(X_2)=36/12=3$, and $\sigma_2=\sqrt{3}$.

Set $Z_1=(X_1-1)/(2/\sqrt{3})$ and $Z_2=(X_2-3)/\sqrt{3}$. The endpoints transform to $-\sqrt{3}$ and $\sqrt{3}$ for both coordinates. Also, the determinant of the inverse standardization map is $$ \left|\det\begin{pmatrix}2/\sqrt{3}&0\\0&\sqrt{3}\end{pmatrix}\right| =\frac{2}{\sqrt{3}}\sqrt{3}=2. $$

Consequently, the transformed density is $(1/24)\cdot 2=1/12$ on $(-\sqrt{3},\sqrt{3})^2$, and zero outside. As a normalization check, this square has area $(2\sqrt{3})^2=12$, so the density integrates to $(1/12)\cdot12=1$.

Effect on Means and Covariances

Theorem: If $X$ has finite second moments and $Y=A(X-c)$ for a fixed matrix $A$ and vector $c$, then $$ E[Y]=A(E[X]-c),\qquad \operatorname{Cov}(Y)=A\operatorname{Cov}(X)A^\mathsf{T}. $$ If every coordinate variance of $X$ is finite and positive, coordinate standardization gives $E[Z]=0$, $\operatorname{Var}(Z_i)=1$, and $$ \operatorname{Cov}(Z_i,Z_j)= \frac{\operatorname{Cov}(X_i,X_j)}{\sigma_i\sigma_j}. $$

Proof. Linearity of expectation gives $E[Y]=E[A(X-c)]=A(E[X]-c)$. Write $m=E[X]$. Then $Y-E[Y]=A(X-m)$, and, using the definition of covariance, $$ \operatorname{Cov}(Y) =E\left[A(X-m)(X-m)^\mathsf{T}A^\mathsf{T}\right] =A\operatorname{Cov}(X)A^\mathsf{T}. $$ For coordinate standardization, take $A=D^{-1}$ and $c=\mu$. The mean formula gives $E[Z]=D^{-1}(\mu-\mu)=0$. The $i$th diagonal entry of $D^{-1}\operatorname{Cov}(X)D^{-1}$ is $\sigma_i^2/\sigma_i^2=1$. Its $(i,j)$ entry is $\operatorname{Cov}(X_i,X_j)/(\sigma_i\sigma_j)$, as stated. $\square$

Worked Example: Standardizing a Correlated Bivariate Normal

Suppose $X=(X_1,X_2)^\mathsf{T}$ has a bivariate normal density with mean $\mu=(2,-1)^\mathsf{T}$ and covariance matrix $$ \Sigma=\begin{pmatrix}12&3\\3&3\end{pmatrix}. $$ Its determinant is $12\cdot3-3\cdot3=27>0$, so this covariance matrix is positive definite. The marginal standard deviations are $\sigma_1=\sqrt{12}$ and $\sigma_2=\sqrt{3}$. The covariance divided by the product of the standard deviations is $3/(\sqrt{12}\sqrt{3})=1/2$.

Define $Z_1=(X_1-2)/\sqrt{12}$ and $Z_2=(X_2+1)/\sqrt{3}$. The standardized vector has mean $(0,0)^\mathsf{T}$ and covariance matrix $$ \begin{pmatrix}1&1/2\\1/2&1\end{pmatrix}. $$ In this case standardization changes the marginal scales but retains the correlation $1/2$.

The determinant of the inverse map from $z$ to $x$ is $\sqrt{12}\sqrt{3}=6$. Applying the density formula to the bivariate normal gives $$ f_Z(z_1,z_2)=\frac{1}{2\pi\sqrt{1-(1/2)^2}} \exp\left\{-\frac{z_1^2-z_1z_2+z_2^2}{2(1-(1/2)^2)}\right\}. $$ For example, at $x=(6,2)^\mathsf{T}$, the standardized coordinates are $$ z_1=\frac{6-2}{\sqrt{12}}=\frac{2}{\sqrt{3}}\approx1.155,\qquad z_2=\frac{2-(-1)}{\sqrt{3}}=\sqrt{3}\approx1.732. $$ Substituting these values into the exponent gives $z_1^2-z_1z_2+z_2^2=4/3-2+3=7/3$, and $2(1-(1/2)^2)=3/2$. The exponent is therefore $-14/9$, so $$ f_Z(2/\sqrt{3},\sqrt{3})=\frac{1}{\pi\sqrt{3}}\exp(-14/9). $$

The transformed coordinates are still dependent: their covariance is $1/2$, not zero. Here they remain jointly normal because an invertible affine transformation of a multivariate normal vector is multivariate normal; standardization alone does not have that consequence for an arbitrary starting distribution.

Standardization Can Change the Shape of the Support

A rectangular support remains rectangular under diagonal standardization, but a nonrectangular support generally remains nonrectangular. The density formula applies on the transformed support, not automatically on all of $\mathbb{R}^d$. Its constant value, if the original density is constant, must be paired with the transformed region.

Worked Example: Standardizing a Uniform Distribution on a Triangle

Let $(X,Y)$ have density $2$ on the triangle $0<x<y<1$, and density zero elsewhere. The triangle has area $1/2$, so this is a normalized uniform density. Find the coordinate-standardized density and support.

Integrating over the other coordinate gives the marginal densities $f_X(x)=2(1-x)$ for $0<x<1$ and $f_Y(y)=2y$ for $0<y<1$. Hence $$ E[X]=2\int_0^1x(1-x)\,dx=\frac{1}{3},\qquad E[X^2]=2\int_0^1x^2(1-x)\,dx=\frac{1}{6}, $$ so $\operatorname{Var}(X)=1/6-(1/3)^2=1/18$. For $Y$, $$ E[Y]=2\int_0^1y^2\,dy=\frac{2}{3},\qquad E[Y^2]=2\int_0^1y^3\,dy=\frac{1}{2}, $$ so $\operatorname{Var}(Y)=1/2-(2/3)^2=1/18$. Both standard deviations are $1/\sqrt{18}$.

Set $Z_1=\sqrt{18}(X-1/3)$ and $Z_2=\sqrt{18}(Y-2/3)$. The inverse map is $X=1/3+Z_1/\sqrt{18}$, $Y=2/3+Z_2/\sqrt{18}$; its determinant has absolute value $1/18$. The original inequalities $0<X<Y<1$ become $$ -\sqrt{2}<Z_1<2\sqrt{2},\qquad -2\sqrt{2}<Z_2<\sqrt{2},\qquad Z_1-Z_2<\sqrt{2}. $$ Thus the standardized density is $2/18=1/9$ on this transformed triangular region and zero elsewhere. The original triangle's area $1/2$ is multiplied by $18$ under the forward map, so the transformed area is $9$; its density integrates to $(1/9)\cdot9=1$.

Common Mistakes and a Practical Check

Using the determinant in the wrong direction. For $Y=A(X-c)$, multiply the original density by $1/|\det A|$. For standardization, $A=D^{-1}$, so the multiplier is instead $|\det D|=\prod_i\sigma_i$.
Scaling by variance instead of standard deviation. A standardized coordinate is $(X_i-\mu_i)/\sigma_i$, where $\sigma_i=\sqrt{\operatorname{Var}(X_i)}$, not $(X_i-\mu_i)/\operatorname{Var}(X_i)$.
Leaving out the transformed support. A formula for the density on the whole plane is wrong when the original vector is supported only on a bounded or otherwise restricted region. Transform the inequalities defining that region.
Concluding that standardization makes coordinates independent. It sets each marginal variance to one but preserves the dependence structure in transformed covariance form. Zero covariance also does not generally imply independence.

A reliable calculation identifies the map, writes its inverse, computes the absolute determinant of that inverse, and transforms the support before stating the new density. As a final check, integrate the resulting density or verify its total mass using the transformed volume. These steps catch both scale-factor errors and support errors.

Check Your Understanding

For each question, keep track of the inverse map as well as the transformed coordinates.

If $Y=A(X-c)$ in three dimensions and $|\det A|=5$, what factor multiplies $f_X(c+A^{-1}y)$ in the density of $Y$?
A bivariate vector has coordinate standard deviations $2$ and $3$. What is the determinant factor for the inverse standardization map?
If the covariance between two coordinates is nonzero, can coordinate standardization make their covariance zero? Explain using the covariance formula.
Why must the support be transformed when a uniform density on a triangle is standardized?
What assumptions ensure that coordinate standardization is defined using finite, positive marginal standard deviations?