Independence of Orthogonal Gaussian Projections

Why Orthogonal Projections Matter

A projection separates a vector into a component lying in a chosen subspace and a component perpendicular to that subspace. For a Gaussian vector with spherical covariance, projections onto perpendicular subspaces are independent—not merely uncorrelated. Their squared lengths are therefore independent quadratic forms, and the dimension of each subspace determines the degrees of freedom of its chi-square law.

The covariance condition is essential. Geometrically orthogonal directions need not produce independent projections when the Gaussian covariance is not spherical. Here we focus on a Gaussian vector whose covariance is a scalar multiple of the identity, and we derive exactly what orthogonality implies.

Projection Matrices and Gaussian Vectors

Definition: A matrix $P$ is an orthogonal projection matrix if $P^\top=P$ and $P^2=P$. It maps a vector to its orthogonal projection onto the subspace $\mathcal{R}(P)$, the range of $P$. For two orthogonal projection matrices $P$ and $Q$, their projected subspaces are orthogonal exactly when $PQ=0$. In that case $QP=0$ as well, since $Q=P^\top$ and $P=P^\top,\ Q=Q^\top$.

Let $Z$ be a $p$-dimensional Gaussian vector with mean $\mu$ and covariance $\sigma^2 I_p$, where $\sigma^2>0$. Thus $Z\sim N_p(\mu,\sigma^2 I_p)$. Its projections are the vectors $PZ$ and $QZ$. Their squared lengths can be written as quadratic forms:

\|PZ\|^2=Z^\top PZ,\qquad \|QZ\|^2=Z^\top QZ.

These identities use symmetry and idempotence: $(PZ)^\top(PZ)=Z^\top P^\top PZ=Z^\top PZ$. The rank of a projection is the dimension of its range. It will become the degrees of freedom in the corresponding chi-square distribution.

Definition: A random variable has a noncentral chi-square distribution with $r$ degrees of freedom and noncentrality parameter $\lambda$, written $\chi^2_r(\lambda)$, if it is distributed as $\sum_{j=1}^r W_j^2$, where the $W_j$ are independent normal random variables with variance $1$ and means $\delta_j$ satisfying $\sum_{j=1}^r\delta_j^2=\lambda$. When $\lambda=0$, this is the central chi-square distribution $\chi^2_r$.

Orthogonal Projections Give Independent Gaussian Vectors

Proposition: Suppose $Z\sim N_p(\mu,\sigma^2 I_p)$, and $P,Q$ are orthogonal projection matrices satisfying $PQ=0$. Then the random vectors $PZ$ and $QZ$ are independent.

Proof. The pair $(PZ,QZ)$ is jointly Gaussian because it is a linear transformation of a Gaussian vector. For vectors $s,t\in\mathbb{R}^p$, its joint characteristic function is the characteristic function of $s^\top PZ+t^\top QZ$. Using $P^\top=P$, $Q^\top=Q$, and the covariance $\sigma^2 I_p$, this is

\begin{aligned} E\!\left[\exp\{i(s^\top PZ+t^\top QZ)\}\right] &=\exp\!\left\{i(s^\top P\mu+t^\top Q\mu) -\frac{\sigma^2}{2}\|Ps+Qt\|^2\right\}. \end{aligned}

The squared norm in the exponent separates. Indeed,

\|Ps+Qt\|^2 =\|Ps\|^2+\|Qt\|^2+2s^\top PQt =\|Ps\|^2+\|Qt\|^2,

because $PQ=0$. The joint characteristic function therefore factors into the characteristic function of $PZ$ at $s$ and the characteristic function of $QZ$ at $t$. Factorization of the joint characteristic function proves independence. $\square$

This argument uses more than zero covariance: it uses that the projected vectors are jointly Gaussian, so the factorization establishes independence. The spherical covariance also makes the cross term vanish whenever the projection spaces are orthogonal.

Chi-Square Laws and Their Ranks

Theorem: Under the conditions of the proposition, let $r=\operatorname{rank}(P)$ and $s=\operatorname{rank}(Q)$. Then $Z^\top PZ$ and $Z^\top QZ$ are independent, and $$ \frac{Z^\top PZ}{\sigma^2}\sim\chi^2_r\!\left(\frac{\mu^\top P\mu}{\sigma^2}\right), \qquad \frac{Z^\top QZ}{\sigma^2}\sim\chi^2_s\!\left(\frac{\mu^\top Q\mu}{\sigma^2}\right). $$ In particular, if $\mu=0$, the two scaled forms have central chi-square distributions.

Proof. Because the ranges of $P$ and $Q$ are orthogonal, choose an orthonormal basis of $\mathbb{R}^p$ whose first $r$ vectors span $\mathcal{R}(P)$, whose next $s$ vectors span $\mathcal{R}(Q)$, and whose remaining vectors span the orthogonal complement of both. In these coordinates, the projections retain the first $r$ coordinates and the next $s$ coordinates, respectively. If $U$ is the orthogonal matrix formed from this basis, then $U^\top Z\sim N_p(U^\top\mu,\sigma^2 I_p)$; its coordinates are independent normals, each with variance $\sigma^2$.

Consequently, $Z^\top PZ/\sigma^2$ is the sum of the squares of the first $r$ transformed coordinates after division by $\sigma$. Those standardized coordinates have variance $1$ and means equal to the corresponding coordinates of $U^\top\mu/\sigma$. The sum of the squares of those means is $\mu^\top P\mu/\sigma^2$, giving the stated noncentral chi-square law. The same reasoning using the next $s$ coordinates gives the law for $Q$. The two forms depend on disjoint groups of independent coordinates, so they are independent. $\square$

If a projection has rank zero, its range is $\{0\}$, its projected vector is identically zero, and its quadratic form is the corresponding degenerate zero variable. The theorem includes this boundary case by interpreting a chi-square sum with zero terms as zero.

Worked Examples

Worked Example: Coordinate Projections in Three Dimensions

Let $Z=(Z_1,Z_2,Z_3)^\top\sim N_3(0,I_3)$, and take

P=\begin{pmatrix}1&0&0\\0&1&0\\0&0&0\end{pmatrix}, \qquad Q=\begin{pmatrix}0&0&0\\0&0&0\\0&0&1\end{pmatrix}.

Both matrices are symmetric and idempotent. Their product is zero, so the projected vectors are independent. The ranks are $\operatorname{rank}(P)=2$ and $\operatorname{rank}(Q)=1$. Directly,

Z^\top PZ=Z_1^2+Z_2^2\sim\chi^2_2,\qquad Z^\top QZ=Z_3^2\sim\chi^2_1.

The forms are independent because they use disjoint independent coordinates. The ranks match the number of coordinates contributing to each squared length.

Worked Example: Noncoordinate Projections and a Nonzero Mean

Let $Z\sim N_3(\mu,I_3)$, where $\mu=(2,0,1)^\top$. Define the unit vectors

a=\frac{1}{\sqrt{2}}(1,1,0)^\top,\qquad b=\frac{1}{\sqrt{6}}(1,-1,2)^\top.

Their lengths are $1$, and $a^\top b=(1-1+0)/\sqrt{12}=0$. Therefore $P=aa^\top$ and $Q=bb^\top$ are rank-one orthogonal projections with $PQ=0$. The projected quadratic forms are $(a^\top Z)^2$ and $(b^\top Z)^2$, and they are independent. Their noncentrality parameters are

\lambda_P=(a^\top\mu)^2 =\left(\frac{2}{\sqrt{2}}\right)^2=2,\qquad \lambda_Q=(b^\top\mu)^2 =\left(\frac{2+2}{\sqrt{6}}\right)^2=\frac{8}{3}.

Thus $Z^\top PZ\sim\chi^2_1(2)$ and $Z^\top QZ\sim\chi^2_1(8/3)$, independently. The mean changes the noncentrality parameters but does not destroy independence.

Worked Example: A Common Variance Scale

Suppose $X\sim N_4(\mu,4I_4)$, where $\mu=(1,-1,2,0)^\top$. Let $P$ project onto the span of the first two coordinate vectors and $Q$ project onto the span of the third coordinate vector. Their ranges are perpendicular, their ranks are $2$ and $1$, and $PQ=0$. The theorem applies with $\sigma^2=4$. For $P$, the noncentrality parameter is

\frac{\mu^\top P\mu}{4} =\frac{1^2+(-1)^2}{4}=\frac12.

For $Q$, it is

\frac{\mu^\top Q\mu}{4} =\frac{2^2}{4}=1.

Hence $X^\top PX/4\sim\chi^2_2(1/2)$ and $X^\top QX/4\sim\chi^2_1(1)$, independently. The unscaled forms are four times these chi-square variables; omitting the division by the variance would give the wrong scale.

Conditions and a Common Mistake

Common mistake: “The projection spaces are perpendicular, so the projected vectors must be independent for any Gaussian covariance.” The reasoning omits the covariance assumption. For a general covariance matrix $\Sigma$, the cross-covariance is $P\Sigma Q$, which need not vanish even if $PQ=0$. In the spherical case $\Sigma=\sigma^2 I_p$, it reduces to $\sigma^2PQ=0$, as required by the proof.

For a concrete check, let $X\sim N_2(0,\Sigma)$, where

\Sigma=\begin{pmatrix}1&\rho\\\rho&1\end{pmatrix},\qquad 0<|\rho|<1, \quad P=\begin{pmatrix}1&0\\0&0\end{pmatrix}, \quad Q=\begin{pmatrix}0&0\\0&1\end{pmatrix}.

The coordinate projections are geometrically orthogonal, but their cross-covariance is $P\Sigma Q\ne0$. In fact, the two projected squared lengths are $X_1^2$ and $X_2^2$. For this bivariate normal vector, $E[X_1^2X_2^2]=1+2\rho^2$, whereas $E[X_1^2]E[X_2^2]=1$. Their covariance is $2\rho^2>0$, so these quadratic forms are not independent. Orthogonality alone is not enough when the covariance is not spherical.

The practical procedure is to check the matrix assumptions first, then read the ranks and means from the projection spaces. In particular, for the central case $\mu=0$, each projected squared length divided by $\sigma^2$ has a central chi-square law with degrees of freedom equal to the projection rank. For nonzero $\mu$, the same rank determines the degrees of freedom, while the squared length of the projected mean, divided by $\sigma^2$, determines noncentrality.

Check Your Understanding

Use the assumptions and results above to answer these questions.

What matrix conditions define an orthogonal projection, and what additional condition makes two projection ranges orthogonal?
If $Z\sim N_5(0,9I_5)$ and $P$ is a rank-two projection, what is the distribution of $Z^\top PZ/9$?
For $Z\sim N_p(\mu,\sigma^2I_p)$, how do the degrees of freedom and noncentrality parameter for $Z^\top PZ/\sigma^2$ depend on $P$ and $\mu$?
Why does orthogonality of the ranges not by itself establish independence for a Gaussian vector with a general covariance matrix?
In the noncoordinate example, verify $a^\top\mu$ and $b^\top\mu$, then state the two noncentrality parameters.