Whitening and the Geometry of GLS

Why Whitening Clarifies GLS

In generalized least squares, observations with correlated or differently scaled errors do not have the geometry of ordinary least squares in the original coordinates. The estimator minimizes a covariance-adjusted residual criterion rather than the ordinary sum of squared residuals. A transformation called whitening makes that adjustment visible: it maps the model to one with spherical error covariance, where the familiar Euclidean projection picture applies.

Whitening is more than a computational trick. It shows why GLS residuals satisfy a weighted orthogonality condition, how fitted values are projected onto a transformed design space, and why a particular factorization can be used without changing the GLS estimate.

Definition: Consider the linear model \(y=X\beta+u\), where \(y\) is an \(n\times1\) response vector, \(X\) is an \(n\times k\) design matrix of full column rank, and \(\beta\) is a \(k\times1\) parameter vector. Suppose \(E(u\mid X)=0\) and \(\operatorname{Var}(u\mid X)=\sigma^2\Omega\), where \(\Omega\) is known, symmetric, and positive definite. A whitening factor is a nonsingular matrix \(C\) satisfying \(C'C=\Omega^{-1}\). The transformed response, design matrix, and error are \(y^*=Cy\), \(X^*=CX\), and \(u^*=Cu\).

The transformed model is \(y^*=X^*\beta+u^*\). Its conditional error mean is zero and its conditional covariance is

E(u^*\mid X)=0,\qquad \operatorname{Var}(u^*\mid X) =C(\sigma^2\Omega)C' =\sigma^2 I_n.

The last equality follows from \(C\Omega C'=I_n\), which is implied by \(C'C=\Omega^{-1}\) for nonsingular \(C\). Thus the transformed errors have equal variance and zero covariance across distinct coordinates. This is the sense in which they are spherical; it does not by itself assert that the errors are normally distributed.

GLS as Least Squares on Transformed Data

For a candidate coefficient vector \(b\), the GLS criterion is the residual quadratic form \((y-Xb)'\Omega^{-1}(y-Xb)\). Since \(C'C=\Omega^{-1}\), this criterion is exactly the ordinary squared length of the transformed residual:

(y-Xb)'\Omega^{-1}(y-Xb) =(y-Xb)'C'C(y-Xb) =\|Cy-CXb\|^2 =\|y^*-X^*b\|^2.

Therefore minimizing the GLS criterion is equivalent to running ordinary least squares (OLS) on \(y^*\) and \(X^*\). Because \(C\) is nonsingular and \(X\) has full column rank, \(X^*=CX\) also has full column rank. The OLS normal equations for the transformed model give the GLS estimator.

Result: Under the stated model and rank conditions, the GLS estimator can be written either in the original coordinates or as OLS on whitened data:

\hat{\beta}_{GLS} =(X'\Omega^{-1}X)^{-1}X'\Omega^{-1}y =(X^{*'}X^*)^{-1}X^{*'}y^*.

Worked Example: Whitening an Intercept-Only Model

Suppose a hypothetical three-observation model has an intercept only, \(X=(1,1,1)'\), and observed response \(y=(1,4,3)'\). Let

C= \begin{pmatrix} 1&0&0\\ -1&1&0\\ 0&0&1 \end{pmatrix}.

First calculate the inverse covariance factor:

\Omega^{-1}=C'C= \begin{pmatrix} 2&-1&0\\ -1&1&0\\ 0&0&1 \end{pmatrix}.

This matrix is positive definite: its leading principal minors are \(2\), \(2(1)-(-1)^2=1\), and \(1\), all positive. Its inverse is \(\Omega\), so the specified covariance structure meets the positive-definiteness requirement.

Transform the response and design:

y^*=Cy= \begin{pmatrix} 1\\ -1(1)+4\\ 3 \end{pmatrix} = \begin{pmatrix} 1\\3\\3 \end{pmatrix}, \qquad X^*=CX= \begin{pmatrix} 1\\-1(1)+1\\1 \end{pmatrix} = \begin{pmatrix} 1\\0\\1 \end{pmatrix}.

The transformed intercept estimate is the OLS coefficient \((X^{*'}X^*)^{-1}X^{*'}y^*\). Substitution gives

\hat{\beta}_{GLS} =\frac{1(1)+0(3)+1(3)}{1^2+0^2+1^2} =\frac{4}{2}=2.

In the original coordinates, \(\Omega^{-1}X=(1,0,1)'\), so the same estimate is \[ \frac{X'\Omega^{-1}y}{X'\Omega^{-1}X} =\frac{1(1)+0(4)+1(3)}{1(1)+1(0)+1(1)} =\frac{4}{2}=2. \] Both calculations agree, as they must. The transformed calculation makes clear that the intercept is fitted to the whitened response using the whitened design, not by simply averaging the original observations.

The Projection Geometry

Set \(A=X^*=CX\). The columns of \(A\) span the transformed design space. OLS projects \(y^*\) onto this column space. The projection matrix and its residual-making counterpart are

P_A=A(A'A)^{-1}A', \qquad M_A=I_n-P_A.

The transformed fitted vector is \(\hat{y}^*=A\hat{\beta}_{GLS}=P_Ay^*\), and the transformed residual is \(r^*=y^*-A\hat{\beta}_{GLS}=M_Ay^*\). Since \(P_A\) projects onto the columns of \(A\), \(r^*\) is perpendicular to every column of \(A\). In matrix form, the transformed normal equations are \(A'r^*=0\).

Geometric interpretation: GLS projects the whitened response \(y^*\) onto the column space of the whitened design \(X^*=CX\). Its whitened residual is perpendicular to that space in ordinary Euclidean geometry. In original coordinates, the corresponding residual is perpendicular to the columns of \(X\) under the \(\Omega^{-1}\)-weighted inner product.

To see the original-coordinate condition, let \(e=y-X\hat{\beta}_{GLS}\). Then \(r^*=Ce\) and \(A=CX\), so

A'r^*=(CX)'Ce=X'C'Ce=X'\Omega^{-1}e=0.

This is the GLS normal equation for the residual. It is not generally the ordinary condition \(X'e=0\). The distinction is geometric: the relevant inner product for original-coordinate residuals is \(\langle a,b\rangle_{\Omega^{-1}}=a'\Omega^{-1}b\), because \(\langle a,b\rangle_{\Omega^{-1}}=(Ca)'(Cb)\).

Worked Example: A Two-Parameter Projection

Continue with the factor \(C\) above, but now suppose the hypothetical design contains an intercept and a linear trend: \(X=\begin{pmatrix}1&0\\1&1\\1&2\end{pmatrix}\), with the same response \(y=(1,4,3)'\). The transformed response remains \(y^*=(1,3,3)'\), while

A=X^*=CX= \begin{pmatrix} 1&0\\ 0&1\\ 1&2 \end{pmatrix}.

Calculate the transformed cross-products:

A'A= \begin{pmatrix}2&2\\2&5\end{pmatrix}, \qquad A'y^*= \begin{pmatrix}4\\9\end{pmatrix}, \qquad (A'A)^{-1} =\frac{1}{6}\begin{pmatrix}5&-2\\-2&2\end{pmatrix}.

The determinant is \(2(5)-2(2)=6\), so the inverse is well-defined. Hence

\hat{\beta}_{GLS} =\frac{1}{6} \begin{pmatrix}5&-2\\-2&2\end{pmatrix} \begin{pmatrix}4\\9\end{pmatrix} =\frac{1}{6} \begin{pmatrix}20-18\\-8+18\end{pmatrix} = \begin{pmatrix}1/3\\5/3\end{pmatrix}.

The transformed fitted vector is \(A\hat{\beta}_{GLS}=(1/3,5/3,11/3)'\), and the transformed residual is \(r^*=(2/3,4/3,-2/3)'\). Check perpendicularity to both design columns:

\begin{pmatrix}1&0&1\\0&1&2\end{pmatrix} \begin{pmatrix}2/3\\4/3\\-2/3\end{pmatrix} = \begin{pmatrix}2/3-2/3\\4/3-4/3\end{pmatrix} = \begin{pmatrix}0\\0\end{pmatrix}.

Returning to original coordinates gives \(e=C^{-1}r^*=(2/3,2,-2/3)'\). The transformed residual is perpendicular to the transformed design; equivalently \(X'\Omega^{-1}e=0\). Here \(X'e=(2,2/3)'\), which is not zero. That is not a contradiction: ordinary Euclidean orthogonality in the original coordinates is not the GLS residual condition.

Why the Choice of Factor Does Not Change the Geometry

A whitening factor need not be unique. Suppose \(C_1'C_1=C_2'C_2=\Omega^{-1}\), with both factors nonsingular. Define \(Q=C_2C_1^{-1}\). Then

Q'Q =C_1^{-T}C_2'C_2C_1^{-1} =C_1^{-T}C_1'C_1C_1^{-1} =I_n.

Thus \(Q\) is orthogonal, and \(C_2y=Q(C_1y)\), \(C_2X=Q(C_1X)\). An orthogonal transformation preserves lengths and angles, so the two whitened least-squares problems have the same residual criterion and coefficient estimate. Their transformed coordinates may look different, but they represent the same GLS geometry.

Worked Example: Checking Both Residual Conditions

A small two-observation example makes the distinction between ordinary and GLS orthogonality explicit. Let \(X=(1,1)'\), \(y=(2,5)'\), and

\Omega^{-1}= \begin{pmatrix}2&-1\\-1&1\end{pmatrix}.

This matrix is positive definite because its leading principal minors are \(2\) and \(2(1)-(-1)^2=1\). The GLS intercept estimate is

\hat{\beta}_{GLS} =\frac{X'\Omega^{-1}y}{X'\Omega^{-1}X} =\frac{(1,1)\begin{pmatrix}2&-1\\-1&1\end{pmatrix} \begin{pmatrix}2\\5\end{pmatrix}} {(1,1)\begin{pmatrix}2&-1\\-1&1\end{pmatrix} \begin{pmatrix}1\\1\end{pmatrix}} =\frac{2}{1}=2.

The residual is \(e=y-X\hat{\beta}_{GLS}=(0,3)'\). Its ordinary dot product with the intercept column is \(X'e=0+3=3\), so it is not ordinarily orthogonal. But

\Omega^{-1}e= \begin{pmatrix}-3\\3\end{pmatrix}, \qquad X'\Omega^{-1}e=(1,1)\begin{pmatrix}-3\\3\end{pmatrix}=0.

This verifies the GLS residual condition directly. In contrast, the vector \(e=(-1,1)'\) is ordinarily orthogonal to the intercept column \(X=(1,1)'\), since \(X'e=-1+1=0\). Whether a residual is ordinarily orthogonal must be checked by its actual dot product; GLS requires the separate weighted check \(X'\Omega^{-1}e=0\).

Common Pitfalls and Practical Meaning

The projection view is useful when checking derivations or interpreting what GLS has done to the data. The estimator does not project the original response onto the original design using ordinary Euclidean distances. It first changes coordinates according to the covariance structure, then performs an ordinary projection. Returning to the original coordinates preserves the fitted model form \(X\hat{\beta}_{GLS}\), while the residual orthogonality is weighted.

Using \(C\) on only the response. The same transformation must be applied to \(y\), \(X\), and the error. Otherwise the transformed equation is not \(y^*=X^*\beta+u^*\).
Reversing the factor order. With the convention \(C'C=\Omega^{-1}\), the transformed criterion is \(\|C(y-Xb)\|^2\). A factorization using \(CC'=\Omega^{-1}\) needs the corresponding transformation arranged consistently.
Claiming ordinary residual orthogonality in the original model. The GLS condition is \(X'\Omega^{-1}e=0\). It reduces to \(X'e=0\) only in special cases, such as \(\Omega=I_n\), or when the particular residual also happens to satisfy the ordinary condition.
Interpreting whitening as making the data independent. The covariance of \(u^*\) is \(\sigma^2I_n\), so its components are uncorrelated. Independence follows only with additional assumptions, such as joint normality.

Whitening therefore supplies a compact diagnostic: verify \(C'C=\Omega^{-1}\), form \(y^*=Cy\) and \(X^*=CX\), solve the ordinary least-squares problem, and check \(X^{*'}r^*=0\). Equivalently, check \(X'\Omega^{-1}e=0\) in the original coordinates. These checks connect the calculation to the geometry and can catch both factorization and arithmetic errors.

Check Your Understanding

Answer each question using the model and geometric definitions given above.

Why must \(C\) be nonsingular when it is used to whiten a positive-definite covariance structure?
Show in one line that the GLS quadratic criterion equals the squared Euclidean length of the transformed residual.
What is the transformed-design column space onto which \(y^*\) is projected?
State the GLS residual orthogonality condition in original coordinates, and contrast it with the transformed-coordinate condition.
For \(X=(1,1)'\) and \(e=(-1,1)'\), calculate \(X'e\). What does this calculation establish, and what does it not establish about GLS orthogonality?