Standard Multivariate Normal Vectors

Why the Standard Multivariate Normal Is a Useful Starting Point

A multivariate normal model describes several random quantities jointly, so the behavior of each coordinate is only part of the story. A particularly important baseline model begins with independent standard normal coordinates. Its joint distribution has a simple density, zero mean, identity covariance, and no preferred direction: rotating or reflecting the coordinate system leaves the distribution unchanged.

The construction matters because it separates what is assumed from what follows. Independence and standard normal distributions are assumptions about the coordinates. The joint density, mean vector, covariance matrix, and spherical symmetry can then be derived from them.

Definition and Joint Density

Definition: Let \(d\) be a positive integer, and let \(Z_1,\ldots,Z_d\) be independent random variables, each with the standard normal distribution \(N(0,1)\). The random vector \(\mathbf Z=(Z_1,\ldots,Z_d)^\mathsf T\) is a standard multivariate normal vector in dimension \(d\). Its distribution is denoted \(N_d(\mathbf 0,I_d)\), where \(\mathbf 0\) is the \(d\)-dimensional zero vector and \(I_d\) is the \(d\)-by-\(d\) identity matrix.

Each coordinate has density \(\phi(z)=(2\pi)^{-1/2}\exp(-z^2/2)\). Independence implies that the joint density is the product of the coordinate densities. To justify this density, the probability of every rectangle \(A_1\times\cdots\times A_d\), with each \(A_i\) a Borel subset of the real line, factors as the product of the coordinate probabilities. Integrating the product density over that rectangle gives exactly the same product. Such rectangles determine the joint distribution, so the product is indeed a density of \(\mathbf Z\).

f_{\mathbf Z}(\mathbf z) =\prod_{i=1}^d\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{z_i^2}{2}\right) =(2\pi)^{-d/2}\exp\left(-\frac{\|\mathbf z\|^2}{2}\right), \qquad \mathbf z\in\mathbb R^d,

Here \(\|\mathbf z\|^2=\sum_{i=1}^d z_i^2\) is the squared Euclidean length. The density integrates to one because it is a product of \(d\) one-dimensional densities. Its dependence on \(\mathbf z\) is only through \(\|\mathbf z\|\), a feature that will yield spherical symmetry.

Worked Example: Writing the Density in Three Dimensions

Suppose \(Z_1,Z_2,Z_3\) are independent standard normal variables. For \(\mathbf Z=(Z_1,Z_2,Z_3)^\mathsf T\), the density is

f_{\mathbf Z}(z_1,z_2,z_3) =(2\pi)^{-3/2}\exp\left(-\frac{z_1^2+z_2^2+z_3^2}{2}\right).

At the point \((1,0,-1)\), the sum of the squared coordinates is \(1^2+0^2+(-1)^2=2\). Substitution gives

f_{\mathbf Z}(1,0,-1) =(2\pi)^{-3/2}\exp(-2/2) =(2\pi)^{-3/2}e^{-1}.

The same density value occurs at any other point at squared distance \(2\) from the origin. For example, \((0,\sqrt{2},0)\) has the same squared length and therefore the same density. This is a statement about density values, not a claim that the probability of any single point is positive; a continuous random vector assigns probability zero to each point.

Mean Vector and Covariance Matrix

The standard normal distribution has mean zero and variance one. Since the coordinates are independent, their cross-covariances are zero. Together these facts determine the first two moments of the vector.

Theorem: If \(\mathbf Z\sim N_d(\mathbf 0,I_d)\) is defined by independent standard normal coordinates, then \(E[\mathbf Z]=\mathbf 0\) and \(\operatorname{Cov}(\mathbf Z)=I_d\). In coordinate form, \(E[Z_i]=0\), \(\operatorname{Var}(Z_i)=1\), and \(\operatorname{Cov}(Z_i,Z_j)=0\) whenever \(i\ne j\).

Proof. For each \(i\), the one-dimensional standard normal moments give \(E[Z_i]=0\) and \(E[Z_i^2]=1\). Thus \(E[\mathbf Z]=\mathbf 0\), and the diagonal entries of the covariance matrix are \(\operatorname{Var}(Z_i)=E[Z_i^2]-(E[Z_i])^2=1\). For \(i\ne j\), independence and integrability give

E[Z_iZ_j]=E[Z_i]E[Z_j]=0, \qquad \operatorname{Cov}(Z_i,Z_j)=E[Z_iZ_j]-E[Z_i]E[Z_j]=0.

All entries are therefore those of the identity matrix, proving the theorem. \(\square\)

The identity covariance says that each coordinate has unit variance and every pair of distinct coordinates has zero covariance. In this construction the coordinates are also independent, by definition. Zero covariance by itself, without the independence assumption or additional distributional conditions, does not generally imply independence.

Worked Example: Reading the Covariance Matrix

Let \(\mathbf Z=(Z_1,Z_2,Z_3)^\mathsf T\) have independent standard normal coordinates. Its mean vector and covariance matrix are

E[\mathbf Z]= \begin{pmatrix}0\\0\\0\end{pmatrix}, \qquad \operatorname{Cov}(\mathbf Z)= \begin{pmatrix} 1&0&0\\ 0&1&0\\ 0&0&1 \end{pmatrix}.

For example, the entry in row 1, column 3 is \(\operatorname{Cov}(Z_1,Z_3)\). Independence gives \(E[Z_1Z_3]=E[Z_1]E[Z_3]=0\), so this entry is zero. The entry in row 2, column 2 is \(\operatorname{Var}(Z_2)=1\). The same reasoning determines every entry of the matrix.

Spherical Symmetry

Definition: A random vector \(\mathbf X\) in \(\mathbb R^d\) is spherically symmetric if \(Q\mathbf X\) has the same distribution as \(\mathbf X\) for every orthogonal \(d\)-by-\(d\) matrix \(Q\), that is, every matrix satisfying \(Q^\mathsf TQ=I_d\). Orthogonal matrices represent rotations and reflections and preserve Euclidean length.

Theorem: The standard multivariate normal vector \(\mathbf Z\sim N_d(\mathbf 0,I_d)\) is spherically symmetric.

Proof. Fix an orthogonal matrix \(Q\). Since \(Q^\mathsf TQ=I_d\), it preserves length: for every \(\mathbf z\),

\|Q\mathbf z\|^2 =(Q\mathbf z)^\mathsf T(Q\mathbf z) =\mathbf z^\mathsf TQ^\mathsf TQ\mathbf z =\mathbf z^\mathsf T\mathbf z =\|\mathbf z\|^2.

Also, \(|\det Q|=1\), since taking determinants in \(Q^\mathsf TQ=I_d\) gives \((\det Q)^2=1\). The density transformation for the invertible map \(\mathbf z\mapsto Q\mathbf z\) therefore gives the density of \(Q\mathbf Z\) at \(\mathbf x\):

f_{Q\mathbf Z}(\mathbf x) =f_{\mathbf Z}(Q^\mathsf T\mathbf x)|\det Q^\mathsf T| =(2\pi)^{-d/2} \exp\left(-\frac{\|Q^\mathsf T\mathbf x\|^2}{2}\right) =(2\pi)^{-d/2}\exp\left(-\frac{\|\mathbf x\|^2}{2}\right).

The last equality uses length preservation by \(Q^\mathsf T\), which is also orthogonal. The resulting density is exactly \(f_{\mathbf Z}(\mathbf x)\) at every \(\mathbf x\). Equal densities determine equal distributions, so \(Q\mathbf Z\) and \(\mathbf Z\) have the same distribution for every orthogonal \(Q\). \(\square\)

Spherical symmetry is stronger than the fact that all coordinates have the same marginal distribution: it says that the joint law is unchanged by any rotation or reflection of the entire space. The density formula makes the reason visible: any two points at the same distance from the origin have the same density.

Worked Example: Equal Probabilities for the Four Quadrants

Let \((Z_1,Z_2)^\mathsf T\sim N_2(\mathbf 0,I_2)\). A rotation by a right angle maps each open quadrant to the next. By spherical symmetry, the four open quadrants have equal probability. The coordinate axes have probability zero: for example, \(P(Z_1=0)=0\) because \(Z_1\) has a continuous density, and likewise \(P(Z_2=0)=0\). The four quadrant probabilities therefore sum to one. If their common value is \(p\), then \(4p=1\), so

P(Z_1>0,\ Z_2>0)=p=\frac14.

This agrees with the direct calculation from coordinate independence: \(P(Z_1>0,Z_2>0)=P(Z_1>0)P(Z_2>0)=(1/2)(1/2)=1/4\). The rotational argument highlights a different implication of the joint distribution.

Worked Probability Using Independence

Worked Example: A Rectangular Probability in Two Dimensions

For \((Z_1,Z_2)^\mathsf T\sim N_2(\mathbf 0,I_2)\), calculate \(P(|Z_1|\le 1,\ |Z_2|\le 2)\). The event is the intersection of one condition on each coordinate. Independence makes its probability the product of the two marginal probabilities:

P(|Z_1|\le 1,\ |Z_2|\le 2) =P(-1\le Z_1\le 1)P(-2\le Z_2\le 2).

Write \(\Phi\) for the standard normal distribution function. By symmetry, \(P(-a\le Z_i\le a)=\Phi(a)-\Phi(-a)=2\Phi(a)-1\) for \(a>0\). Hence

P(|Z_1|\le 1,\ |Z_2|\le 2) =\{2\Phi(1)-1\}\{2\Phi(2)-1\}.

Using \(\Phi(1)\approx0.841344746\) and \(\Phi(2)\approx0.977249868\), the two factors are approximately \(0.682689492\) and \(0.954499736\). Their product is approximately \(0.651627\), or \(0.652\) to three decimal places. Keeping adequate precision until the final rounding is important here.

Common Mistake and Why the Construction Matters

A common mistake is to infer that a vector is standard multivariate normal merely because every coordinate has a standard normal marginal distribution. Marginals do not determine the dependence structure. For a counterexample, let \(W\sim N(0,1)\) and set \(\mathbf X=(W,W)^\mathsf T\). Each coordinate separately is standard normal, but the coordinates are identical, not independent. Directly,

\operatorname{Cov}(\mathbf X) = \begin{pmatrix} \operatorname{Var}(W)&\operatorname{Cov}(W,W)\\ \operatorname{Cov}(W,W)&\operatorname{Var}(W) \end{pmatrix} = \begin{pmatrix}1&1\\1&1\end{pmatrix} \ne I_2.

The vector \(\mathbf X\) lies on the line where its two coordinates are equal, whereas the standard two-dimensional vector is defined using two independent coordinates and has the joint density given above. The correction is to check the joint construction or joint distribution, not just the coordinate marginals.

The standard multivariate normal is a useful reference model because its density, moments, and symmetries are all explicit. The density provides a direct way to calculate probabilities over regions, independence gives products for rectangular events, and spherical symmetry explains why direction does not affect density at a fixed distance from the origin. Together these properties make it a natural baseline for studying more general multivariate normal distributions.

Check Your Understanding

Try these questions using the definition and results developed above.

Write the joint density of a standard multivariate normal vector in dimension \(d\), and explain which feature of the formula depends only on distance from the origin.
For independent standard normal coordinates \(Z_1,\ldots,Z_d\), what are \(E[Z_i]\), \(\operatorname{Var}(Z_i)\), and \(\operatorname{Cov}(Z_i,Z_j)\) when \(i\ne j\)?
Why does an orthogonal matrix preserve the standard multivariate normal density?
In dimension two, calculate \(P(Z_1>0,Z_2<0)\) using coordinate independence.
Why do standard normal marginal distributions alone not establish that a vector is standard multivariate normal?