Independence of the Normal Sample Mean and Variance

Why Orthogonal Decomposition Matters

In a normal sample, the sample mean summarizes the component of the data in the direction of the constant vector, while the residual sum of squares measures variation in directions perpendicular to it. The central fact is that these two parts are independent—not merely uncorrelated. That distinction supports exact finite-sample calculations, but the independence depends on the normal model.

We prove the result by changing to orthogonal coordinates. One coordinate records the sample mean; the remaining coordinates describe the residual contrasts. For a spherical multivariate normal vector, the joint density separates across these coordinates, which gives independence directly.

Projection Formulation

Definition: Let $X_1,\ldots,X_n$ be independent and identically distributed as $N(\mu,\sigma^2)$, where $\mu\in\mathbb{R}$, $\sigma^2>0$, and $n\geq 2$. Write $X=(X_1,\ldots,X_n)^\mathsf{T}$, let $\mathbf{1}$ be the $n$-vector of ones, and define $$ P=\frac{1}{n}\mathbf{1}\mathbf{1}^\mathsf{T},\qquad M=I_n-P. $$ The sample mean is $\bar X=n^{-1}\mathbf{1}^\mathsf{T}X$, and the residual sum of squares is $R=\sum_{i=1}^n(X_i-\bar X)^2$.

The matrix $P$ is the orthogonal projection onto the one-dimensional space spanned by $\mathbf{1}$, and $M$ projects onto its orthogonal complement. In particular, $P^\mathsf{T}=P$, $M^\mathsf{T}=M$, $P+M=I_n$, and $PM=0$. Since $PX=\bar X\mathbf{1}$, the complementary projection gives the residual vector:

MX=X-\bar X\mathbf{1} \qquad\text{and}\qquad R=(MX)^\mathsf{T}(MX)=X^\mathsf{T}MX.

Thus, the mean is determined by the projected component $PX$, and $R$ by the perpendicular component $MX$. Orthogonality alone is not a general proof of independence. The normal distribution supplies the stronger fact needed here.

Main Result and Proof

Theorem: Under the stated iid normal model, $\bar X$ and $R$ are independent. Consequently, $\bar X$ is independent of every measurable function of $R$, including any fixed constant multiple of $R$.

To see how the normal assumption produces independence, standardize the vector: $Z=(X-\mu\mathbf{1})/\sigma$. Its coordinates are independent $N(0,1)$ variables. Choose an orthogonal matrix $Q$ whose first row is $\mathbf{1}^\mathsf{T}/\sqrt n$, and write $Y=QZ$. Such a matrix exists by extending the unit vector $\mathbf{1}/\sqrt n$ to an orthonormal basis.

The following density argument proves that the coordinates of $Y$ are independent. The density of $Z$ at $z\in\mathbb{R}^n$ is $(2\pi)^{-n/2}\exp(-z^\mathsf{T}z/2)$. Because $Q$ is orthogonal, its determinant has absolute value one and $z^\mathsf{T}z=(Qz)^\mathsf{T}(Qz)$. The change of variables $y=Qz$ therefore gives

f_Y(y)=(2\pi)^{-n/2}\exp\left(-\frac12\sum_{j=1}^n y_j^2\right) =\prod_{j=1}^n \frac{1}{\sqrt{2\pi}}\exp\left(-\frac{y_j^2}{2}\right).

This product density shows that $Y_1,\ldots,Y_n$ are independent standard normal variables. The first coordinate is $Y_1=\sqrt n\,\mathbf{1}^\mathsf{T}Z/n$, so

\bar X=\mu+\frac{\sigma}{\sqrt n}Y_1.

The remaining coordinates span the residual subspace. Since $Q$ is orthogonal and its first row points in the $\mathbf{1}$ direction, the squared length of the residual projection is the sum of squares of coordinates $2,\ldots,n$. Hence

R=\sigma^2\sum_{j=2}^n Y_j^2.

The mean is a measurable function of $Y_1$, while $R$ is a measurable function of $Y_2,\ldots,Y_n$. These coordinate groups are independent, so their measurable functions are independent. Therefore $\bar X$ and $R$ are independent. A measurable function of $R$ remains independent of $\bar X$, proving the final statement. $\square$

Worked Orthogonal Decompositions

Worked Example: A Normal Sample of Size Two

Suppose $X_1,X_2$ are independent $N(3,4)$ variables. Set $Z_i=(X_i-3)/2$, so $Z_1,Z_2$ are independent standard normal variables. Apply the orthogonal transformation

A=\frac{Z_1+Z_2}{\sqrt2},\qquad B=\frac{Z_1-Z_2}{\sqrt2}.

The coefficient rows are orthogonal unit vectors, so the density factorization in the proof shows that $A$ and $B$ are independent standard normal variables. Substituting $X_i=3+2Z_i$ gives

\bar X=\frac{X_1+X_2}{2} =3+Z_1+Z_2=3+\sqrt2 A, \qquad R=(X_1-\bar X)^2+(X_2-\bar X)^2 =\frac{(X_1-X_2)^2}{2}=4B^2.

The first quantity depends only on $A$, and the second only on $B$. They are therefore independent. As a check, $\operatorname{Var}(\bar X)=2=\sigma^2/n$, and $R$ is nonnegative, as a sum of squared residuals must be.

Worked Example: Three Observations and Two Residual Contrasts

Let $X_1,X_2,X_3$ be iid $N(-1,9)$. Write $Z_i=(X_i+1)/3$, and use the orthonormal rows

q_1=\frac{(1,1,1)}{\sqrt3},\qquad q_2=\frac{(1,-1,0)}{\sqrt2},\qquad q_3=\frac{(1,1,-2)}{\sqrt6}.

Each row has length one. Their pairwise dot products are $(1-1+0)/\sqrt6=0$, $(1+1-2)/\sqrt{18}=0$, and $(1-1+0)/\sqrt{12}=0$. Thus the rows form an orthonormal basis. Define $Y_j=q_jZ$. The orthogonal change of coordinates makes $Y_1,Y_2,Y_3$ independent standard normals. Since the first coordinate records the constant direction,

\bar X=-1+\frac{3}{\sqrt3}Y_1=-1+\sqrt3Y_1, \qquad R=9(Y_2^2+Y_3^2).

The mean depends only on $Y_1$, and the residual sum of squares only on $Y_2,Y_3$, so they are independent. This decomposition also checks the scale: $\operatorname{Var}(\bar X)=3$, equal to $9/3$.

Worked Example: Four Observations with Explicit Residual Coordinates

Suppose $X_1,\ldots,X_4$ are iid $N(10,1)$, and put $Z_i=X_i-10$. Use the following orthonormal rows:

q_1=\frac{(1,1,1,1)}{2},\quad q_2=\frac{(1,-1,0,0)}{\sqrt2},\quad q_3=\frac{(0,0,1,-1)}{\sqrt2},\quad q_4=\frac{(1,1,-1,-1)}{2}.

Each row has unit length. The first row has dot product zero with the other three because their entries sum to zero. The dot products among $q_2,q_3,q_4$ are respectively $0$, $(1-1)/ (2\sqrt2)=0$, and $(-1+1)/(2\sqrt2)=0$. If $Y_j=q_jZ$, the coordinates are therefore independent standard normals. Directly from these coordinates,

\bar X=10+\frac{Y_1}{2}, \qquad R=Y_2^2+Y_3^2+Y_4^2.

The first expression uses only $Y_1$; the second uses only $Y_2,Y_3,Y_4$. Their independence follows from the independence of the coordinate groups. In particular, the argument does not require listing or conditioning on possible residual values.

What the Result Does—and Does Not—Say

Independence is useful because probabilities involving the mean and residual variation can be separated when their joint distribution is needed. It also means that replacing $R$ by a fixed multiple does not change the independence: for $c\in\mathbb{R}$, $\bar X$ is independent of $cR$. For example, when $n\geq2$, this applies to the usual scaled residual sum of squares $R/(n-1)$.

The model assumptions matter. The density factorization works because the standardized sample has a multivariate normal density that is unchanged in form by orthogonal transformations. For a nonnormal sample, orthogonal components need not be independent. Neither orthogonality of two vectors nor zero covariance, on its own, establishes independence.

Common Pitfall

Pitfall: “The sample mean and residual sum of squares are orthogonal, so they are independent for any sample.” This confuses a geometric property with a probabilistic conclusion. Orthogonal projection components are independent here because the sample is jointly normal, not because perpendicular vectors are automatically independent.

A simple nonnormal check makes the distinction concrete. Let $X_1,X_2$ be iid, each taking $1$ and $-1$ with probability $1/2$. Then $\bar X$ is $1,-1,0,0$ across the four equally likely outcomes, while $R=(X_1-X_2)^2/2$ is $0,0,2,2$. In this example $R=2(1-\bar X^2)$, so $R$ is determined by $\bar X$ and the two are not independent. Yet $\operatorname{Cov}(\bar X,R)=0$: symmetry gives $E[\bar X R]=0$, while $E[\bar X]=0$. Zero covariance is not enough.

Check Your Understanding

Use the projection and orthogonal-coordinate argument to answer the following.

What subspace does $P$ project onto, and what component of $X$ does $MX$ represent?
In the proof, why does the density of $QZ$ factor into a product of one-dimensional standard normal densities?
For an iid $N(\mu,\sigma^2)$ sample, which orthogonal coordinates determine $\bar X$, and which determine $R$?
Why does independence of $\bar X$ and $R$ imply independence of $\bar X$ and $R/(n-1)$ when $n\geq2$?
What does the Rademacher example show about the claim that zero covariance implies independence?