GLS Transformations for AR(1) Errors

Why Transform a Regression With Serially Correlated Errors?

In a time-ordered regression, errors may be correlated across observations even after the conditional mean has been modeled. When that dependence follows a stationary first-order autoregressive structure, the covariance matrix has a particularly useful form: a simple difference between adjacent observations removes the serial correlation after the first observation. The first observation needs a separate adjustment because it has no preceding observation to difference against.

This transformation is not a change to the substantive regression model. It rewrites the same GLS criterion as ordinary least squares on transformed outcomes and regressors. The distinction matters: the transformation depends on the assumed error covariance, and the covariance assumptions must hold for the design being analyzed.

Model and Covariance Assumptions

For this tutorial, treat the regressors as fixed and assume the error covariance is the stationary AR(1) covariance just stated. Equivalently, with random regressors, impose the corresponding conditional moment assumptions explicitly: \(E(u\mid X)=0\) and \(\operatorname{Var}(u\mid X)=\sigma_\varepsilon^2R/(1-\rho^2)\), where \(R_{st}=\rho^{|t-s|}\). An unconditional covariance statement alone does not establish this conditional covariance. This qualification ensures that the GLS calculation is justified for the design at hand.

Write \(\Omega=\operatorname{Var}(u)=\sigma_\varepsilon^2R/(1-\rho^2)\) for a fixed design. GLS minimizes the error quadratic form \((y-X\beta)'\Omega^{-1}(y-X\beta)\). The goal is to find a matrix transformation \(T\) for which \(T\Omega T'=\sigma_\varepsilon^2 I\), so the transformed errors have equal variance and zero covariance across observations.

Deriving the AR(1) Transformation

For each observation after the first, subtract \(\rho\) times the preceding observation. The regression equation becomes \(y_t-\rho y_{t-1}=(x_t-\rho x_{t-1})'\beta+(u_t-\rho u_{t-1})\). Because the AR(1) equation gives \(u_t-\rho u_{t-1}=\varepsilon_t\), these later transformed errors are the innovations. But the first observation is not an innovation: \(u_1\) has stationary variance \(\sigma_\varepsilon^2/(1-\rho^2)\). Multiplying the first equation by \(\sqrt{1-\rho^2}\) gives it the same error variance as the innovations.

In equations, if \(y^*=Ty\) and \(X^*=TX\), the rows are

and each regressor column is transformed by exactly the same rule. In particular, an intercept column becomes \(\sqrt{1-\rho^2}\) in the first row and \(1-\rho\) in each later row. It does not remain a column of ones.

Why the Transformation Gives GLS

The key justification is that the transformation whitens the specified covariance, rather than merely making adjacent observations look less correlated. To see the covariance calculation directly, \(v_1=\sqrt{1-\rho^2}u_1\) has variance \(\sigma_\varepsilon^2\). For \(t\geq2\), \(v_t=u_t-\rho u_{t-1}=\varepsilon_t\), whose variance is also \(\sigma_\varepsilon^2\). Under the stationary AR(1) model, the initial error is uncorrelated with subsequent innovations, and the innovations are uncorrelated with one another. Hence the transformed errors have covariance \(\sigma_\varepsilon^2 I\).

The GLS equivalence follows by substituting \(\Omega^{-1}=\sigma_\varepsilon^{-2}T'T\) into its criterion. The positive scalar \(\sigma_\varepsilon^{-2}\) does not change the minimizer, leaving the transformed sum of squared residuals.

Therefore, minimizing the GLS criterion over \(\beta\) is equivalent to minimizing \(\|y^*-X^*\beta\|^2\). This proves the coefficient-estimate equivalence. It does not claim that a covariance model is correct merely because a transformation has been applied; the assumed covariance structure remains essential.

Calculating GLS Estimates

The following example computes the coefficients rather than stopping at the transformation. The data are hypothetical and ordered in time. Assume the regression has an intercept and a linear time regressor, and use a known AR(1) covariance parameter \(\rho=0.5\). As above, the regressors are treated as fixed and the error covariance is assumed to be the stationary AR(1) covariance. These conditions make the transformed least-squares estimate the GLS estimate.

Interpretation, Use, and a Common Pitfall

The transformation is useful because it converts a particular GLS problem into a familiar least-squares calculation while preserving the GLS coefficient estimate. Its simplicity depends on the AR(1) structure and on using the correct \(\rho\). A different covariance pattern generally calls for a different whitening transformation. Also, the transformed regression is a computational representation of the GLS objective, not evidence by itself that the original errors follow a stationary AR(1) process.

A common mistake is to difference every row using \(y_t-\rho y_{t-1}\), including the first observation, or to leave the first observation unadjusted while treating all transformed errors as having variance \(\sigma_\varepsilon^2\). The first observation has no predecessor, and under stationarity its original error variance is \(\sigma_\varepsilon^2/(1-\rho^2)\), not \(\sigma_\varepsilon^2\). Multiplication by \(\sqrt{1-\rho^2}\) is what brings its transformed variance to the innovation variance.

A second important pitfall is to infer a conditional covariance model from an unconditional one when regressors are random. GLS conditional on \(X\) requires the conditional covariance to have the stipulated form (and the conditional mean to be correctly specified). State whether the design is fixed or give the conditional moment assumptions; do not substitute an unconditional covariance statement for them.