The Multivariate Normal Sample Mean

Why the Vector Sample Mean Has an Exact Law

A sample mean can summarize several measurements at once. For example, each observation might record a patient’s systolic and diastolic blood pressure, or several outcomes from the same experiment. When the observations are independent and share a multivariate normal distribution, the mean vector also has an exact multivariate normal distribution—not merely an approximation for large samples. Its covariance matrix records both the uncertainty in each component and the dependence between components.

The key operation is averaging independent random vectors. Independence makes the characteristic function of their sum a product; the quadratic form in the multivariate normal characteristic function then shows exactly how averaging changes the covariance. The result is useful for calculating standard errors and the distributions of specified linear combinations, while keeping track of correlations among measurements.

Setup and Notation

Definition: A random vector $X$ in $\mathbb{R}^p$ has a multivariate normal distribution with mean vector $\mu$ and covariance matrix $\Sigma$, written $X\sim N_p(\mu,\Sigma)$, if its characteristic function is $$ \varphi_X(t)=E[\exp(i t^\mathsf{T}X)] =\exp\!\left(i t^\mathsf{T}\mu-\frac{1}{2}t^\mathsf{T}\Sigma t\right), \qquad t\in\mathbb{R}^p. $$ Here $\mu\in\mathbb{R}^p$, and $\Sigma$ is a symmetric positive semidefinite $p\times p$ covariance matrix. Positive semidefinite matrices are allowed to be singular.

Consider independent random vectors $X_1,\ldots,X_n$, each with distribution $N_p(\mu,\Sigma)$, where $n\geq 1$. Their sample mean is the vector $$ \bar X=\frac{1}{n}\sum_{i=1}^n X_i. $$ The entries of $\bar X$ are the ordinary sample means of the corresponding components. The matrix $\Sigma$ describes the covariance among components within one observation; independence here is across observations.

The Exact Sampling Distribution

Theorem: If $X_1,\ldots,X_n$ are independent and each has distribution $N_p(\mu,\Sigma)$, then $$ \bar X\sim N_p\!\left(\mu,\frac{\Sigma}{n}\right). $$ In particular, $E[\bar X]=\mu$, and the $(j,k)$ entry of $\operatorname{Cov}(\bar X)$ is $\Sigma_{jk}/n$.

Proof. For any $t\in\mathbb{R}^p$, independence gives a product of characteristic functions for the sum. Applying the definition to each $X_i$, with argument $t/n$, yields $$ \begin{aligned} \varphi_{\bar X}(t) &=E\!\left[\exp\!\left(i t^\mathsf{T}\frac{1}{n}\sum_{i=1}^n X_i\right)\right]\\ &=\prod_{i=1}^n E\!\left[\exp\!\left(i (t/n)^\mathsf{T}X_i\right)\right]\\ &=\left\{\exp\!\left(i (t/n)^\mathsf{T}\mu -\frac{1}{2}(t/n)^\mathsf{T}\Sigma(t/n)\right)\right\}^{n}\\ &=\exp\!\left(i t^\mathsf{T}\mu-\frac{1}{2n}t^\mathsf{T}\Sigma t\right). \end{aligned} $$ This is the characteristic function of $N_p(\mu,\Sigma/n)$, so it identifies the exact distribution of $\bar X$.

The covariance statement can also be checked directly. Each $X_i$ has finite second moments, and distinct observations are independent, so their cross-covariance matrices are zero. Therefore $$ \begin{aligned} \operatorname{Cov}(\bar X) &=\frac{1}{n^2}\operatorname{Cov}\!\left(\sum_{i=1}^n X_i\right)\\ &=\frac{1}{n^2}\sum_{i=1}^n\operatorname{Cov}(X_i) =\frac{1}{n^2}(n\Sigma) =\frac{\Sigma}{n}. \end{aligned} $$ The expectation follows from linearity of expectation: $E[\bar X]=n^{-1}\sum_{i=1}^n\mu=\mu$. These calculations agree with the parameters identified by the characteristic function. $\square$

Components, Contrasts, and Density

The diagonal entry $\Sigma_{jj}/n$ is the variance of the sample mean of component $j$; its standard error is $\sqrt{\Sigma_{jj}/n}$. An off-diagonal entry $\Sigma_{jk}/n$ is the covariance between the sample means of components $j$ and $k$. Dividing the entire covariance matrix by $n$ leaves component correlations unchanged when both component variances are positive.

Corollary: For any fixed vector $a\in\mathbb{R}^p$, the linear contrast $a^\mathsf{T}\bar X$ has distribution $$ a^\mathsf{T}\bar X\sim N\!\left(a^\mathsf{T}\mu,\frac{a^\mathsf{T}\Sigma a}{n}\right). $$ If $a^\mathsf{T}\Sigma a=0$, this is a degenerate normal distribution concentrated at $a^\mathsf{T}\mu$.

Derivation. A linear transformation of a multivariate normal vector is normal. More explicitly, the characteristic function of $a^\mathsf{T}\bar X$, evaluated at a real number $s$, is $$ \begin{aligned} E[\exp(i s a^\mathsf{T}\bar X)] &=\varphi_{\bar X}(sa)\\ &=\exp\!\left(i s a^\mathsf{T}\mu -\frac{s^2}{2n}a^\mathsf{T}\Sigma a\right). \end{aligned} $$ This is the characteristic function of the stated univariate normal distribution, including the constant case when its variance is zero. $\square$

When $\Sigma$ is positive definite, $\bar X$ has a density with respect to $p$-dimensional volume: $$ f_{\bar X}(x) =\frac{n^{p/2}}{(2\pi)^{p/2}|\Sigma|^{1/2}} \exp\!\left[-\frac{n}{2}(x-\mu)^\mathsf{T}\Sigma^{-1}(x-\mu)\right], \qquad x\in\mathbb{R}^p. $$ If $\Sigma$ is singular, this ordinary density does not exist on all of $\mathbb{R}^p$, but the characteristic-function statement and the contrast result still apply.

Worked Calculations

Worked Example: A Correlated Bivariate Sample Mean

Suppose $n=16$ independent observations each have distribution $N_2\!\left((10,20)^\mathsf{T},\begin{pmatrix}4&1\\1&9\end{pmatrix}\right)$. Find the exact distribution and the standard errors of the two component means.

The theorem gives the mean vector unchanged and divides every covariance entry by $16$: $$ \bar X\sim N_2\!\left( \begin{pmatrix}10\\20\end{pmatrix}, \frac{1}{16}\begin{pmatrix}4&1\\1&9\end{pmatrix} \right) =N_2\!\left( \begin{pmatrix}10\\20\end{pmatrix}, \begin{pmatrix}1/4&1/16\\1/16&9/16\end{pmatrix} \right). $$ Thus the first component mean has standard error $\sqrt{1/4}=1/2$, and the second has standard error $\sqrt{9/16}=3/4$. Their covariance is $1/16$, not zero. The original correlation is $1/(2\cdot3)=1/6$; for the two sample means it is $$ \frac{1/16}{(1/2)(3/4)}=\frac{1}{6}. $$ Averaging reduces the marginal variances but does not remove the correlation.

Worked Example: An Exact Distribution for a Contrast

Let $n=25$ independent observations have distribution $N_2\!\left((3,1)^\mathsf{T},\begin{pmatrix}4&1\\1&9\end{pmatrix}\right)$. Find the distribution of $C=\bar X_1-2\bar X_2$, and calculate $P(C>2.2)$.

Use $a=(1,-2)^\mathsf{T}$. The contrast has mean $$ a^\mathsf{T}\mu=(1)(3)+(-2)(1)=1. $$ For its variance, first calculate $$ \Sigma a= \begin{pmatrix}4&1\\1&9\end{pmatrix} \begin{pmatrix}1\\-2\end{pmatrix} = \begin{pmatrix}2\\-17\end{pmatrix}, \qquad a^\mathsf{T}\Sigma a=(1)(2)+(-2)(-17)=36. $$ Consequently, $$ C\sim N(1,36/25)=N(1,36/25). $$ Its standard deviation is $6/5$. Standardizing the cutoff gives $$ P(C>2.2) =1-\Phi\!\left(\frac{2.2-1}{6/5}\right) =1-\Phi(1) \approx 0.1587, $$ where $\Phi$ is the standard normal distribution function.

Worked Example: The Mean of a Sum of Correlated Components

Suppose $n=9$ independent observations have distribution $N_3(\mu,\Sigma)$, where $$ \mu=\begin{pmatrix}1\\2\\0\end{pmatrix}, \qquad \Sigma=\begin{pmatrix}1&1/2&0\\1/2&4&-1\\0&-1&9\end{pmatrix}. $$ Find the mean and variance of $S=\bar X_1+\bar X_2+\bar X_3$.

Take $a=(1,1,1)^\mathsf{T}$. The expected sum of the three sample means is $$ a^\mathsf{T}\mu=1+2+0=3. $$ To include all covariances in its variance, calculate $$ \begin{aligned} a^\mathsf{T}\Sigma a &=1+4+9+2(1/2+0-1)\\ &=14-1=13. \end{aligned} $$ The terms in parentheses are the three distinct off-diagonal covariances; each contributes twice to the quadratic form. Hence $$ S\sim N(3,13/9). $$ In particular, adding only the three marginal variances would give $14/9$, which is incorrect because it omits the negative covariance between components two and three and the positive covariance between components one and two.

Why the Assumptions Matter

The exact normal law relies on both normality and independence across observations. If the observations are independent but not multivariate normal, the covariance calculation still holds when second moments exist, but the sample mean need not have an exact normal distribution. A normal approximation may be justified under additional conditions, but that is a different claim from the finite-sample result proved here.

Independence also matters for the covariance scaling. In general, if the observations have common covariance $\Sigma$ but are dependent, then $$ \operatorname{Cov}(\bar X) =\frac{1}{n^2}\left\{n\Sigma+ \sum_{i\ne k}\operatorname{Cov}(X_i,X_k)\right\}. $$ The cross-covariance terms need not vanish, so the answer need not be $\Sigma/n$. Similarly, using $n\Sigma$ for the covariance of the mean confuses the sample mean with the sample total: the covariance of the total is $n\Sigma$ under independence, while that of the mean is $\Sigma/n$.

A final point is that the theorem concerns the distribution of the mean vector alone. It does not by itself assert independence between the mean and a sample covariance matrix, or give the distribution of that sample covariance. Those are separate distributional results with their own conditions.

Key takeaway: For independent observations from one $p$-variate normal distribution, the sample mean has the same mean vector and $1/n$ times the population covariance matrix. For any linear contrast, calculate its variance with the full quadratic form $a^\mathsf{T}\Sigma a/n$, including covariance terms.

Check Your Understanding

Use the theorem and contrast result to answer these questions.

If each observation has covariance matrix $\Sigma$, what is the covariance matrix of the sample mean from $n$ independent observations?
For a fixed vector $a$, state the mean and variance of $a^\mathsf{T}\bar X$.
Why does the exact multivariate normal law of the sample mean not require $n$ to be large?
What assumption makes the cross-covariance terms between distinct observations vanish in the covariance calculation?
When $\Sigma$ is singular, which distributional statements in this tutorial still apply, and what changes about the density?