Centering and Orthogonal Contrasts for Normal Vectors

Introduction

For a vector of measurements, centering removes the part that points in the constant-vector direction: the direction in which every coordinate changes by the same amount. The remaining component consists of contrasts, which compare coordinates while summing to zero. For an isotropic normal vector, these two pieces are not merely geometrically perpendicular; they are independent, and their squared lengths have chi-square distributions after scaling.

This connection is useful whenever a Gaussian sum of squares is split into a component along a specified direction and a component orthogonal to it. The key is to identify the two projection matrices, then express the normal vector in an orthonormal coordinate system adapted to those projections.

The Constant-Vector Projection and Centering

Let \(n\geq 2\), and let \(\mathbf{1}\in\mathbb{R}^n\) denote the vector whose entries are all one. The unit vector in the constant-vector direction is \(e=\mathbf{1}/\sqrt{n}\). The orthogonal projection onto this direction and the projection onto its orthogonal complement are

The matrix \(P\) is symmetric and idempotent: \(P'=P\) and \(P^2=P\). The same is true of \(M\). Also \(PM=MP=0\) and \(P+M=I_n\). The range of \(P\) is the one-dimensional space spanned by \(\mathbf{1}\); the range of \(M\) is the \((n-1)\)-dimensional space of vectors whose coordinates sum to zero.

These are orthogonal components, so \(x' x=\|Px\|^2+\|Mx\|^2\). In particular, \(\|Px\|^2=n\bar{x}^{\,2}\), while \(\|Mx\|^2=\sum_{i=1}^n(x_i-\bar{x})^2\). The latter is a squared length in a space of dimension \(n-1\), not \(n\).

Orthonormal Contrasts

Choose an \(n\times(n-1)\) matrix \(C\) whose columns form an orthonormal basis of the space perpendicular to \(\mathbf{1}\). Thus \(C'C=I_{n-1}\) and \(C'\mathbf{1}=0\). Such a basis exists by choosing a basis for that subspace and orthonormalizing it. Its projection matrix is \(CC'=M\). Consequently, the contrast coordinates \(C'x\) determine the centered component, and

This gives \(n-1\) orthonormal coordinates for the centered component. The particular contrast basis may vary, but the projection \(M\) and the squared length \(\|Mx\|^2\) do not.

Independent Chi-Square Laws for a Normal Vector

The following result applies more generally than the equal-mean case. It identifies how the mean vector determines the noncentrality in each subspace.

and these two random variables are independent. In particular, if \(\eta=\mu\mathbf{1}\), then \(M\eta=0\) and \(\|P\eta\|^2=n\mu^2\), giving

independently. The centered component has a central chi-square law even when the common mean \(\mu\) is nonzero.

Proof. Form the orthogonal matrix \(Q=[e\ C]\), whose first column is \(e\) and whose remaining columns are the orthonormal contrasts. Set \(Z=Q'X\). An orthogonal transformation of a multivariate normal vector is multivariate normal, so

A multivariate normal vector with diagonal covariance matrix has independent coordinates. More explicitly, its density factors into the product of the univariate normal densities, since its covariance is \(\sigma^2 I_n\). Thus \(Z_1=e'X\) and the contrast coordinates \(Z_2,\ldots,Z_n\) are independent, with variance \(\sigma^2\) each.

The projection onto the constant direction is \(PX=eZ_1\), so \(\|PX\|^2=Z_1^2\). Its scaled value is the square of a variance-one normal coordinate with mean \(e'\eta/\sigma\). By the definition above, it has a \(\chi^2_1((e'\eta)^2/\sigma^2)\) distribution. Since \(P=ee'\), \((e'\eta)^2=\|P\eta\|^2\).

The centered projection has contrast coordinates \(C'X=(Z_2,\ldots,Z_n)'\), and \(\|MX\|^2=\|C'X\|^2\). After division by \(\sigma^2\), it is the sum of squares of \(n-1\) independent variance-one normal coordinates with means given by \(C'\eta/\sigma\). Their squared means sum to

This gives the stated noncentral chi-square distribution for the centered component. The constant-direction coordinate and the contrast coordinates are independent, so their functions \(\|PX\|^2\) and \(\|MX\|^2\) are independent as well. If \(\eta=\mu\mathbf{1}\), then \(M\eta=0\), while \(P\eta=\eta\) and \(\|\eta\|^2=n\mu^2\). Substitution gives the equal-mean case. \(\square\)

Why the Decomposition Matters

The projection view separates geometry from distribution. The matrix \(P\) identifies the direction shared by all coordinates; \(M\) identifies the contrasts that remain after removing that direction. Under covariance \(\sigma^2I_n\), an orthonormal change of coordinates preserves the covariance matrix, so the coordinates in these perpendicular subspaces are independent. The resulting degrees of freedom are the dimensions of the subspaces: one for the constant direction and \(n-1\) for the contrasts.

A common mistake is to argue that perpendicular projections are independent simply because their covariance is zero. Zero covariance alone does not imply independence for arbitrary random vectors. In this result, independence follows because the vector is jointly normal and its covariance is isotropic; in the proof, the adapted normal coordinates have diagonal covariance and their joint density factors. Without the stated normal and covariance assumptions, the chi-square laws and independence need not hold.

Another error is to assign \(n\) degrees of freedom to the centered squared length. The centered vectors satisfy one linear constraint, namely that their entries sum to zero, so they occupy an \((n-1)\)-dimensional space. Finally, \(\|PX\|^2\) is \(n\) times the square of the coordinate average, not just the square of that average.