Why the Mean Matters in a Sum-of-Squares Decomposition
Orthogonal projection decomposes a vector into perpendicular pieces, and the squared lengths of those pieces add. For Gaussian data, this geometry also gives a distributional decomposition: each squared length has a chi-square distribution, and perpendicular components are independent. But the chi-square distributions need not be central. Their status depends on how the mean vector sits relative to each projection space.
The key test is not whether the mean vector itself belongs to a component’s projection space. A component is central precisely when the projection of the mean onto that space is zero—that is, when the mean is orthogonal to the space. A nonzero projected mean shifts the Gaussian coordinates and produces a noncentral chi-square distribution.
Projection Components and Noncentrality
Let \(Y\) be an \(n\)-dimensional random vector with \(Y\sim N_n(\mu,\sigma^2 I_n)\), where \(\sigma^2>0\). Consider symmetric idempotent matrices \(P_1,\ldots,P_m\) satisfying \(P_iP_j=0\) for \(i\ne j\). Each \(P_j\) is the orthogonal projection onto its range, denoted \(\mathcal{R}(P_j)\). The matrices need not sum to \(I_n\); if they do, their ranges form a complete orthogonal decomposition of \(\mathbb{R}^n\).
The equality \(\mu'P_j\mu=\|P_j\mu\|^2\) follows from symmetry and idempotence: \((P_j\mu)'(P_j\mu)=\mu'P_j'P_j\mu=\mu'P_j\mu\). Thus the parameter is the squared length of the mean component in the projection space, measured in units of \(\sigma^2\). We use the convention that a noncentral chi-square distribution with \(r\) degrees of freedom and parameter \(\lambda\) is the distribution of \(\sum_{k=1}^r Z_k^2\), where the \(Z_k\) are independent normal random variables with variance 1 and with means whose squared sum is \(\lambda\).
Proof. For each \(j\), choose an orthonormal basis for \(\mathcal{R}(P_j)\). Because the ranges are mutually orthogonal, the chosen bases together are orthonormal. Extend them, if necessary, to an orthonormal basis of \(\mathbb{R}^n\), and let \(U\) be the orthogonal matrix whose columns are those basis vectors. For a column \(u\) in the basis of \(\mathcal{R}(P_j)\), the corresponding coordinate of \(U'Y\) is \(u'Y\). It has variance \(\sigma^2\) and mean \(u'\mu\). The coordinates of \(U'Y\) are jointly Gaussian, and their covariance matrix is \(U'(\sigma^2 I_n)U=\sigma^2 I_n\). They are therefore independent.
The projection \(P_jY\) retains exactly the coordinates corresponding to the \(r_j\) basis vectors for \(\mathcal{R}(P_j)\). Consequently, \(Y'P_jY\) is the sum of their squares. After dividing each coordinate by \(\sigma\), \(Q_j\) is a sum of \(r_j\) independent squared unit-variance normal variables. The sum of the squared coordinate means after this scaling is \(\sum_{u} (u'\mu)^2/\sigma^2=\|P_j\mu\|^2/\sigma^2=\lambda_j\), where the sum is over that basis. This gives the stated noncentral chi-square distribution. Disjoint projection spaces use disjoint coordinates of \(U'Y\), so the resulting sums are independent.
Finally, \(\lambda_j=0\) if and only if \(\|P_j\mu\|^2=0\), which holds if and only if \(P_j\mu=0\). For an orthogonal projection, this is equivalent to \(\mu\) being orthogonal to its range. Hence \(\lambda_j=0\) exactly when the component is central. \(\square\)
How to Classify the Components
For each component, first identify its projection space and compute the projection of \(\mu\) onto that space. Its rank gives the degrees of freedom, while the squared length of the projected mean divided by \(\sigma^2\) gives the noncentrality parameter. A zero projected mean gives a central component; any nonzero projected mean gives a noncentral one. A component can be central even when the overall mean vector is nonzero, provided that mean lies entirely in other, orthogonal spaces.
Find the range of \(P_j\), its rank \(r_j\), and the action of \(P_j\) on vectors.
Calculate \(P_j\mu\). This determines which part of the mean contributes to this sum of squares.
Calculate \(\lambda_j=\|P_j\mu\|^2/\sigma^2\). The component is central exactly when this value is zero.
Worked Example: Two Coordinate Components
Let \(Y\sim N_2((2,0)',4I_2)\), and let \(P_1\) project onto the first coordinate axis and \(P_2\) onto the second: \(P_1=\begin{pmatrix}1&0\\0&0\end{pmatrix}\) and \(P_2=\begin{pmatrix}0&0\\0&1\end{pmatrix}\). Find the distributions of \(Q_1=Y'P_1Y/4\) and \(Q_2=Y'P_2Y/4\).
Both projections have rank 1. For the first component, \(P_1\mu=(2,0)'\), so \(\lambda_1=\|(2,0)'\|^2/4=4/4=1\). Thus \(Q_1\sim\chi^2_1(1)\), which is noncentral. For the second, \(P_2\mu=(0,0)'\), so \(\lambda_2=0/4=0\) and \(Q_2\sim\chi^2_1\), which is central. The coordinates are independent under the assumed spherical normal distribution, so \(Q_1\) and \(Q_2\) are independent. The nonzero overall mean does not make both components noncentral.
Worked Example: A Central Component Despite a Nonzero Mean
Let \(Y\sim N_3(\mu, I_3)\), where \(\mu=(1,1,3)'\). Use the orthonormal directions \(u_1=(1,1,0)'/\sqrt{2}\), \(u_2=(1,-1,0)'/\sqrt{2}\), and \(u_3=(0,0,1)'\). Let \(P_j=u_ju_j'\), so each projection has rank 1. Find the three noncentrality parameters.
For a rank-one projection \(P_j=u_ju_j'\), the projected mean is \(P_j\mu=u_j(u_j'\mu)\), and its squared length is \((u_j'\mu)^2\). The three coordinates of the mean along these directions are \(u_1'\mu=(1+1)/\sqrt{2}=\sqrt{2}\), \(u_2'\mu=(1-1)/\sqrt{2}=0\), and \(u_3'\mu=3\). Since \(\sigma^2=1\), the parameters are \(\lambda_1=2\), \(\lambda_2=0\), and \(\lambda_3=9\). Therefore \(Q_1\sim\chi^2_1(2)\), \(Q_2\sim\chi^2_1\), and \(Q_3\sim\chi^2_1(9)\), independently. The mean vector is nonzero, but its coordinate in the \(u_2\) direction vanishes, making that component central.
Worked Example: A Mean Orthogonal to One Projection Space
Let \(Y\sim N_3((1,0,2)',4I_3)\). Let \(P_1\) project onto the plane spanned by \(e_1=(1,0,0)'\) and \(e_2=(0,1,0)'\), and let \(P_2\) project onto the line spanned by \(e_3=(0,0,1)'\). Find the component distributions.
The plane projection is \(P_1=\operatorname{diag}(1,1,0)\), with rank 2. It gives \(P_1\mu=(1,0,0)'\), so \(\lambda_1=1^2/4=1/4\). Thus \(Q_1\sim\chi^2_2(1/4)\). The line projection is \(P_2=\operatorname{diag}(0,0,1)\), with rank 1, and \(P_2\mu=(0,0,2)'\). Hence \(\lambda_2=2^2/4=1\), giving \(Q_2\sim\chi^2_1(1)\). These components are independent because the plane and line are orthogonal. Although the mean is not in the plane, its nonzero projection onto that plane makes the plane component noncentral. Centrality requires orthogonality, not simply failure of the mean to belong to the projection space.
Moments as a Check on the Parameter
The noncentrality parameter also determines how much the mean shifts the component’s expected size. If \(Q\sim\chi^2_r(\lambda)\), write \(Q=\sum_{k=1}^r Z_k^2\) with independent \(Z_k\sim N(a_k,1)\) and \(\sum_k a_k^2=\lambda\). For one such coordinate, \(E[Z_k^2]=1+a_k^2\), so adding the expectations gives \(E[Q]=r+\lambda\).
To verify the variance, write \(Z_k=a_k+\varepsilon_k\), where \(\varepsilon_k\sim N(0,1)\). Then \(Z_k^2=a_k^2+2a_k\varepsilon_k+\varepsilon_k^2\). The random terms have zero covariance because \(E[\varepsilon_k^3]=E[\varepsilon_k]=0\). Using \(E[\varepsilon_k^2]=1\) and \(E[\varepsilon_k^4]=3\), we obtain \(\operatorname{Var}(Z_k^2)=4a_k^2+2\). Independence across coordinates gives \(\operatorname{Var}(Q)=\sum_{k=1}^r(4a_k^2+2)=4\lambda+2r=2(r+2\lambda)\). Thus a proposed parameter can be checked against the component’s mean and variance. The larger the squared projected mean, the larger both the expected sum of squares and its variance.
Common Mistakes and Why the Classification Matters
- Using the overall mean instead of the projected mean. A nonzero \(\mu\) does not make every component noncentral. Compute \(P_j\mu\) separately for each component.
- Confusing membership with orthogonality. A mean that is not in \(\mathcal{R}(P_j)\) can still have a nonzero projection onto it. Centrality requires \(P_j\mu=0\), not merely \(\mu\notin\mathcal{R}(P_j)\).
- Forgetting the variance scale. The parameter is \(\|P_j\mu\|^2/\sigma^2\), not just \(\|P_j\mu\|^2\). Changing the measurement scale of the noise changes the noncentrality.
- Using the wrong degrees of freedom. The degrees of freedom are \(\operatorname{rank}(P_j)\), not necessarily the number of observations or the number of symbols used to describe the model.
This classification matters because central and noncentral components have different distributions, even when they arise from the same orthogonal decomposition. Under a complete decomposition, every part of the mean contributes to the noncentrality of its own component; a part with zero projection contributes none. Consequently, checking projected means is essential before using a central chi-square distribution for a Gaussian sum of squares.
Check Your Understanding
Use the projection criterion and the definition of the noncentrality parameter to answer these questions.
- For \(Y\sim N_n(\mu,\sigma^2I_n)\), state the condition on \(P_j\mu\) that makes \(Y'P_jY/\sigma^2\) central.
- If a projection has rank 3 and \(\|P_j\mu\|^2=12\), with \(\sigma^2=4\), what are its degrees of freedom and noncentrality parameter?
- Can a component be central when \(\mu\ne 0\)? Explain using the projection space.
- For a rank-one projection onto a unit vector \(u\), express the noncentrality parameter in terms of \(u\), \(\mu\), and \(\sigma^2\).
- Why are components from mutually orthogonal projection spaces independent under a spherical multivariate normal distribution?