Yule-Walker Equations For AR(1)

Why the AR(1) Yule-Walker Equations Matter

An AR(1) model describes a time series whose current value depends linearly on its immediately preceding value and a new random innovation. Its dependence is summarized by a particularly direct relationship: the lag-one autocorrelation equals the autoregressive coefficient. The Yule-Walker equations make that relationship precise and also connect the process variance to the innovation variance.

These equations are useful both for understanding a model and for recovering its parameters from autocovariances. The key is to distinguish the variance of the observed process from the variance of its innovations: they are generally not the same, because past innovations continue to contribute to the process.

Definition: A causal stationary AR(1) process with mean $\mu$ satisfies $$ X_t-\mu=\phi(X_{t-1}-\mu)+\varepsilon_t, $$ where $\phi$ is the autoregressive coefficient and $\{\varepsilon_t\}$ is white noise with mean zero and variance $\sigma_\varepsilon^2$. White noise has zero covariance between distinct times, and each innovation is uncorrelated with earlier process values. For a causal stationary AR(1), $|\phi|<1$.

Write $Z_t=X_t-\mu$ for the centered process. Its autocovariance at lag $k$ is $\gamma_k=\operatorname{Cov}(Z_t,Z_{t-k})$, and its autocorrelation at lag $k$ is $\rho_k=\gamma_k/\gamma_0$, where $\gamma_0=\operatorname{Var}(Z_t)>0$. For a real-valued stationary process, $\gamma_{-k}=\gamma_k$.

Deriving the AR(1) Yule-Walker Equations

The derivation uses the centered model and the fact that an innovation is uncorrelated with the process values that came before it. For any $k\geq 1$, multiply the model at time $t$ by $Z_{t-k}$ and take expectations. The innovation term has zero covariance with $Z_{t-k}$, leaving a simple recurrence for the autocovariances.

Start with the centered model.
$Z_t=\phi Z_{t-1}+\varepsilon_t$, with $E[Z_t]=E[\varepsilon_t]=0$.

Multiply by the lagged value and take expectations.
For $k\geq1$, $E[Z_tZ_{t-k}]=\phi E[Z_{t-1}Z_{t-k}]+E[\varepsilon_tZ_{t-k}]$.

Use stationarity and the innovation assumption.
$E[Z_tZ_{t-k}]=\gamma_k$, $E[Z_{t-1}Z_{t-k}]=\gamma_{k-1}$, and $E[\varepsilon_tZ_{t-k}]=0$. Thus $\gamma_k=\phi\gamma_{k-1}$.

Use the model at lag zero.
Multiplying $Z_t=\phi Z_{t-1}+\varepsilon_t$ by $Z_t$ and taking expectations gives $\gamma_0=\phi\gamma_1+\sigma_\varepsilon^2$, because $E[\varepsilon_tZ_t]=\sigma_\varepsilon^2$.

At lag one, the recurrence says $\gamma_1=\phi\gamma_0$. Dividing by the positive process variance $\gamma_0$ gives the central result. The lag-zero equation then recovers the innovation variance.

Theorem: For a causal stationary AR(1) process with $\gamma_0>0$, $$ \gamma_k=\phi\gamma_{k-1}\quad(k\geq1),\qquad \gamma_1=\phi\gamma_0,\qquad \rho_1=\phi, $$ and $$ \sigma_\varepsilon^2=\gamma_0-\phi\gamma_1 =\gamma_0(1-\phi^2). $$ Consequently, $\gamma_k=\phi^k\gamma_0$ and $\rho_k=\phi^k$ for $k\geq0$.

The last equality for the innovation variance follows by substituting $\gamma_1=\phi\gamma_0$ into $\gamma_0=\phi\gamma_1+\sigma_\varepsilon^2$. It also shows why the observed process variance exceeds the innovation variance when $0<|\phi|<1$: the process includes accumulated effects of earlier innovations. The derivation needs uncorrelated innovations and the stated stationary AR(1) model; the relationship $\rho_1=\phi$ is not a general rule for arbitrary time series.

Recovering Parameters From Autocovariances

If the autocovariances are supplied, use $\gamma_1/\gamma_0$ to obtain $\phi$, then use either $\gamma_0-\phi\gamma_1$ or $\gamma_0(1-\phi^2)$ to obtain $\sigma_\varepsilon^2$. The two calculations should agree. A proposed pair of autocovariances for a causal stationary AR(1) must also satisfy $|\gamma_1/\gamma_0|<1$.

Worked Example: Positive Dependence and Innovation Variance

A centered, stationary series is modeled as a causal AR(1). Its theoretical autocovariances are $\gamma_0=16$ and $\gamma_1=10$, in squared measurement units. Recover $\phi$ and $\sigma_\varepsilon^2$, and check the result.

Method. Apply the lag-one Yule-Walker equation $\gamma_1=\phi\gamma_0$, followed by the lag-zero equation. This avoids treating the process variance as if it were the innovation variance.

First, $$ \phi=\frac{\gamma_1}{\gamma_0}=\frac{10}{16}=0.625. $$ Since $|0.625|<1$, the coefficient is consistent with a causal stationary AR(1).

Next, using the lag-zero equation, $$ \sigma_\varepsilon^2 =\gamma_0-\phi\gamma_1 =16-(0.625)(10) =16-6.25 =9.75. $$

Check with the equivalent formula: $$ \gamma_0(1-\phi^2) =16(1-0.625^2) =16(1-0.390625) =16(0.609375) =9.75. $$ The two methods agree. Therefore, $\phi=0.625$ and the innovation variance is $9.75$ squared measurement units.

Worked Example: Negative Lag-One Dependence

A hypothetical monthly index, after subtracting its mean, is modeled by a causal stationary AR(1). Its autocovariances are $\gamma_0=25$ and $\gamma_1=-15$, in squared index units. Find the coefficient and innovation variance, then calculate the lag-two autocovariance.

Method. The ratio of lag-one to lag-zero autocovariance determines the signed coefficient. Retaining the sign is important: a negative lag-one autocovariance indicates alternating, rather than positive, dependence.

Calculate the coefficient: $$ \phi=\frac{-15}{25}=-0.6. $$ It satisfies $|\phi|<1$, as required.

Calculate the innovation variance: $$ \sigma_\varepsilon^2 =\gamma_0-\phi\gamma_1 =25-(-0.6)(-15) =25-9 =16. $$ The product $\phi\gamma_1$ is positive because both factors are negative.

For the lag-two autocovariance, apply $\gamma_2=\phi\gamma_1$: $$ \gamma_2=(-0.6)(-15)=9. $$ A further check is $\phi^2\gamma_0=(-0.6)^2(25)=0.36(25)=9$. Thus the recovered values are $\phi=-0.6$, $\sigma_\varepsilon^2=16$, and $\gamma_2=9$.

Worked Example: Testing a Proposed Autocovariance Pattern

A centered stationary series is claimed to follow a causal AR(1), with $\gamma_0=20$, $\gamma_1=-8$, and $\gamma_2=3.2$, all in squared measurement units. Recover its parameters and determine whether the three autocovariances agree with the AR(1) recurrence.

Method. First recover the coefficient from lags zero and one. Then calculate the implied innovation variance and lag-two autocovariance. This check distinguishes a usable AR(1) autocovariance pattern from a set of values that only matches at lag one.

The coefficient is $$ \phi=\frac{\gamma_1}{\gamma_0}=\frac{-8}{20}=-0.4, $$ which satisfies $|\phi|<1$.

The innovation variance is $$ \sigma_\varepsilon^2 =20-(-0.4)(-8) =20-3.2 =16.8. $$ It is positive, as a variance must be.

The lag-two value implied by the AR(1) recurrence is $$ \gamma_2=\phi\gamma_1=(-0.4)(-8)=3.2. $$ This equals the supplied value. Equivalently, $\phi^2\gamma_0=0.16(20)=3.2$. The values are consistent with the AR(1) equations, and the recovered parameters are $\phi=-0.4$ and $\sigma_\varepsilon^2=16.8$.

Common Mistakes and Model Checks

The formula $\phi=\gamma_1/\gamma_0$ has a specific direction: the lag-one autocovariance is divided by the lag-zero autocovariance. Reversing the ratio can produce a plausible-looking number, but it does not satisfy the Yule-Walker equation.

Common exam trap: With $\gamma_0=16$ and $\gamma_1=10$, an incorrect calculation is $\phi=\gamma_0/\gamma_1=16/10=1.6$. This reverses the lag-one equation and gives a coefficient incompatible with a causal stationary AR(1). The correction is $\phi=\gamma_1/\gamma_0=10/16=0.625$. Then $\sigma_\varepsilon^2=16-(0.625)(10)=9.75$.

A second frequent error is to report $\gamma_0$ as the innovation variance. For a nonzero coefficient, the process variance includes dependence from previous time periods, and the correct calculation subtracts $\phi\gamma_1$. Always check the resulting variance is nonnegative and, for a causal stationary AR(1), that the recovered coefficient has absolute value less than one. If autocovariances are estimated from data rather than supplied as theoretical values, these checks concern the fitted estimate; finite-sample estimates need not obey every theoretical relationship exactly.

Key takeaway: For a causal stationary AR(1), the lag-one autocorrelation is the coefficient, $\phi=\gamma_1/\gamma_0=\rho_1$. The innovation variance follows from the lag-zero equation, $\sigma_\varepsilon^2=\gamma_0-\phi\gamma_1=\gamma_0(1-\phi^2)$. Use both equations to calculate and check the result.

Check Your Understanding

For each problem, assume the stated autocovariances are theoretical values unless the question says otherwise. Check the stationary AR(1) condition where relevant.

A centered series has $\gamma_0=9$ and $\gamma_1=3$. Find $\phi$ and the innovation variance. (Answer: $\phi=1/3$, $\sigma_\varepsilon^2=8$.)
A proposed causal stationary AR(1) has $\gamma_0=16$ and $\gamma_1=-12$. Find $\phi$, the innovation variance, and $\gamma_2$. (Answer: $\phi=-0.75$, $\sigma_\varepsilon^2=7$, $\gamma_2=9$.)
A model has $\gamma_0=10$, $\gamma_1=7.5$, and $\gamma_2=5.625$. Recover $\phi$ and the innovation variance, then check the supplied lag-two value. (Answer: $\phi=0.75$, $\sigma_\varepsilon^2=4.375$; the implied $\gamma_2=(0.75)(7.5)=5.625$, so it agrees.)
A claimed stationary AR(1) has $\gamma_0=6$, $\gamma_1=-3$, and $\gamma_2=1.5$. Recover the coefficient and innovation variance and test the lag-two autocovariance. (Answer: $\phi=-0.5$, $\sigma_\varepsilon^2=4.5$; the implied $\gamma_2=(-0.5)(-3)=1.5$, so the values agree.)
A proposed causal stationary AR(1) has $\gamma_0=4$ and $\gamma_1=4.4$. Calculate the coefficient implied by the lag-one equation and decide whether the values can describe a causal stationary AR(1). (Answer: $\phi=1.1$; no, because $|\phi|>1$.)

What You'll Learn