Why Two Transformations?
When regression errors follow a first-order autoregressive process, neighboring errors are correlated. Generalized least squares (GLS) can account for that dependence by transforming the regression so its errors are uncorrelated and have a common variance. Two familiar procedures use the same transformation for later observations but handle the first observation differently: Cochrane–Orcutt omits it, while Prais–Winsten transforms and retains it.
The difference is consequential in short time series: omitting an observation can change the fitted coefficients, and it always discards some information. The Prais–Winsten first-row adjustment is not an arbitrary correction; under a stationary AR(1) error model, it follows directly from the variance of the initial error.
Model and Assumptions
The transformations below are exact when \(\rho\) is known and the stated covariance model holds. The zero conditional mean assumption is needed for the transformed regression to estimate the same \(\beta\); the AR(1) covariance structure is what makes its transformed errors uncorrelated with common variance. Normality is not needed for the algebra or for GLS estimation itself.
In practice, \(\rho\) is often estimated and then substituted into the transformation. The resulting feasible GLS estimates depend on that estimate, so the calculations below treat \(\rho\) as given to show precisely what each transformation does.
Transforming the Later Observations
For any \(t\geq 2\), multiply the equation at \(t-1\) by \(\rho\) and subtract it from the equation at \(t\):
Thus each later outcome and each regressor is quasi-differenced by subtracting \(\rho\) times its preceding value. The transformed error is \(\epsilon_t\), with mean zero and variance \(\sigma^2\); the innovations are uncorrelated across time. For an intercept, the original regressor is 1, so its transformed value in every later row is \(1-\rho\), not 1.
This formula requires no value of \(y_0\), which is one reason Cochrane–Orcutt can be applied using observations \(2,\ldots,T\). But the original first observation has not disappeared from the data-generating model. Its treatment distinguishes the two procedures.
The First Observation: Prais–Winsten
At \(t=1\), stationarity gives a different error variance from that of an innovation: \(\operatorname{Var}(u_1)=\sigma^2/(1-\rho^2)\). Multiplying the first regression equation by \(\sqrt{1-\rho^2}\) gives
The first transformed error has variance \((1-\rho^2)\sigma^2/(1-\rho^2)=\sigma^2\), matching the later transformed errors. This is the Prais–Winsten transformation. It transforms the entire sample, applying the square-root factor to every first-row regressor, including the intercept.
Cochrane–Orcutt instead fits the quasi-differenced regression only for \(t=2,\ldots,T\). It avoids specifying or using a transformed first row, but uses one fewer observation. When the stationary AR(1) assumptions hold, Prais–Winsten is the transformation that retains the information in the initial observation while putting all rows on the same error-variance scale.
Worked Example: Transforming a Four-Observation Series
Worked Example: Transforming a Four-Observation Series
Suppose a hypothetical regression records \(x_t\), hours of training, and \(y_t\), a performance score in points. The four observations are \((x_t,y_t)=(0,2),(1,4),(2,5.5),(3,7)\). Use the assumed value \(\rho=0.5\) to calculate both transformations. The data and parameter value are illustrative, not from a real study.
For \(t=2\), the quasi-differenced values are \(y_2^*=4-0.5(2)=3\), \(x_2^*=1-0.5(0)=1\), and the transformed intercept is \(1-0.5=0.5\). For \(t=3\), they are \(y_3^*=5.5-0.5(4)=3.5\), \(x_3^*=2-0.5(1)=1.5\), and intercept \(0.5\). For \(t=4\), they are \(y_4^*=7-0.5(5.5)=4.25\), \(x_4^*=3-0.5(2)=2\), and intercept \(0.5\).
For Prais–Winsten, the initial-row multiplier is \(\sqrt{1-0.5^2}=\sqrt{0.75}\approx0.8660\). The first transformed row is therefore intercept \(0.8660\), regressor \(0.8660(0)=0\), and outcome \(0.8660(2)\approx1.732\). Cochrane–Orcutt uses only the three later rows. In both cases, the transformed intercept is a regressor column whose coefficient is the original \(\beta_0\); it should not be replaced by a standard column of ones without adjusting how the intercept coefficient is interpreted.
Worked Example: Comparing the Coefficient Estimates
Worked Example: Comparing the Coefficient Estimates
Continue with the same hypothetical data and \(\rho=0.5\), fitting an intercept and the training-hours regressor. For Cochrane–Orcutt, the transformed rows have intercept column \(0.5\), regressor values \(1,1.5,2\), and outcomes \(3,3.5,4.25\). Regressing those outcomes on the two stated columns estimates the original intercept and slope.
The mean regressor is \(\bar{x}^*=(1+1.5+2)/3=1.5\), and the mean outcome is \(\bar{y}^*=(3+3.5+4.25)/3=3.5833\). The slope is $$ \hat{\beta}_{1,\mathrm{CO}} =\frac{(1-1.5)(3-3.5833)+(1.5-1.5)(3.5-3.5833)+(2-1.5)(4.25-3.5833)} {(1-1.5)^2+(1.5-1.5)^2+(2-1.5)^2} =\frac{0.625}{0.5}=1.25. $$ The fitted constant in the ordinary centered calculation is \(3.5833-1.25(1.5)=1.7083\). Since the transformed intercept column is \(0.5\), this constant equals \(0.5\hat{\beta}_{0,\mathrm{CO}}\). Hence \(\hat{\beta}_{0,\mathrm{CO}}=1.7083/0.5=3.4167\).
For Prais–Winsten, use all four transformed rows. With design columns \(a_t\) for the transformed intercept and \(z_t\) for transformed training hours, the rows are \((a_t,z_t,y_t^*)=(0.8660,0,1.7321),(0.5,1,3),(0.5,1.5,3.5),(0.5,2,4.25)\). The least-squares normal equations are $$ \begin{pmatrix} \sum a_t^2 & \sum a_tz_t\\ \sum a_tz_t & \sum z_t^2 \end{pmatrix} \begin{pmatrix}\hat{\beta}_{0,\mathrm{PW}}\\\hat{\beta}_{1,\mathrm{PW}}\end{pmatrix} = \begin{pmatrix}\sum a_ty_t^*\\\sum z_ty_t^*\end{pmatrix} = \begin{pmatrix}6.875\\16.75\end{pmatrix}, \qquad \begin{pmatrix} 1.5&2.25\\ 2.25&7.25 \end{pmatrix} \begin{pmatrix}\hat{\beta}_{0,\mathrm{PW}}\\\hat{\beta}_{1,\mathrm{PW}}\end{pmatrix} = \begin{pmatrix}6.875\\16.75\end{pmatrix}. $$ The determinant is \(1.5(7.25)-2.25^2=5.8125\). Solving gives \(\hat{\beta}_{0,\mathrm{PW}}=(6.875(7.25)-2.25(16.75))/5.8125\approx2.091\) points and \(\hat{\beta}_{1,\mathrm{PW}}=(1.5(16.75)-2.25(6.875))/5.8125\approx1.661\) points per training hour.
The estimates differ: Cochrane–Orcutt gives a slope of 1.25 points per hour, while Prais–Winsten gives about 1.661 points per hour. Both use the same \(\rho\) and the same later transformed rows; the difference arises because Prais–Winsten also fits the transformed first row. This small example demonstrates that dropping one observation can matter, but it does not imply that either estimate must always be larger.
Why the First-Row Adjustment Is GLS
The justification can be seen by considering the transformation matrix \(A\) applied to the vector of errors. Its first row has \(\sqrt{1-\rho^2}\) in the first position; each later row has \(-\rho\) in the preceding position and 1 in its own position. Therefore \(Au\) has first component \(\sqrt{1-\rho^2}u_1\) and later components \(u_t-\rho u_{t-1}=\epsilon_t\).
The first component has variance \(\sigma^2\) by stationarity, and each later component has variance \(\sigma^2\) by the innovation assumption. The first component is uncorrelated with later innovations under the stationary AR(1) model, and the innovations are mutually uncorrelated. Consequently,
Thus least squares on the fully transformed regression is GLS for the stated covariance model: the transformation makes the error covariance spherical. Cochrane–Orcutt instead applies only the later rows of \(A\), so it minimizes the transformed residual sum of squares over a reduced set of observations. It is not the same full-sample GLS calculation.
Worked Example: Quantifying the First Observation's Scale
Worked Example: Quantifying the First Observation's Scale
Suppose a hypothetical stationary AR(1) model has \(\rho=0.8\). The first-row Prais–Winsten multiplier is \(\sqrt{1-0.8^2}=\sqrt{0.36}=0.6\). Its transformed error is \(0.6u_1\). Since \(\operatorname{Var}(u_1)=\sigma^2/0.36\), the transformed error variance is \(0.6^2(\sigma^2/0.36)=\sigma^2\), exactly the variance of each transformed innovation.
For comparison, the variance of the untransformed initial error is \(\sigma^2/0.36\approx2.778\sigma^2\), not \(\sigma^2\). Simply appending the original first equation to the quasi-differenced later rows would therefore give the first row a different error variance and would not produce the equal-variance GLS transformation. Cochrane–Orcutt avoids that mismatch by excluding the row; Prais–Winsten rescales it correctly and retains it.
Practical Comparison and Common Pitfall
| Feature | Prais–Winsten | Cochrane–Orcutt |
|---|---|---|
| Rows used | First row plus transformed rows \(2,\ldots,T\) | Transformed rows \(2,\ldots,T\) |
| First-row treatment | Multiply all first-row variables, including the intercept, by \(\sqrt{1-\rho^2}\) | Omit the first row |
| Under the stationary AR(1) model | Full-sample GLS transformation | Least squares on the later transformed equations |
| Information retained | Uses all \(T\) observations | Uses \(T-1\) transformed observations |
Common mistake: “The first observation cannot be used because the AR(1) transformation requires a lag, so both procedures must drop it.” The lagged subtraction formula does require a preceding observation and therefore applies directly only for \(t\geq2\). But under stationarity the initial error has a known variance, which yields the separate Prais–Winsten first-row adjustment. Another related error is to multiply only \(y_1\) by \(\sqrt{1-\rho^2}\); every first-row regressor must be multiplied by the same factor.
Neither procedure automatically resolves an incorrect error model or endogeneity. The interpretation of estimated coefficients remains conditional on the regression's identifying assumptions. Also, when \(\rho\) is estimated rather than known, the transformed regression uses an estimated covariance parameter; the numerical comparison then depends on the estimate and on the implementation used to obtain it.
Check Your Understanding
Answer each question using the stationary AR(1) model and the transformations defined above.
- For \(\rho=0.4\), what is the Prais–Winsten multiplier for the first row, and what transformed variance does it produce?
- Why is the transformed intercept in a later row \(1-\rho\), rather than 1?
- Which observation does Cochrane–Orcutt omit, and why does Prais–Winsten not need to omit it under stationarity?
- If a researcher scales the first outcome but not the first-row regressors, what has gone wrong?
- In the four-observation example, which procedure produced the larger estimated training-hours slope, and what feature of the calculations explains why the estimates differ?