The Stationary AR(1) Process as a Linear Process

Why the AR(1) Has an Infinite Linear Representation

An autoregressive model expresses the current value partly in terms of the previous value. Repeated substitution shows that the previous value itself depends on an earlier value, and so on. When the autoregressive coefficient has magnitude below one, the influence of those increasingly distant values fades geometrically. The result is a representation of the process as a weighted sum of current and past shocks.

This representation is useful because it makes the consequences of the AR(1) coefficient transparent. The weights describe how a shock persists over time, while sums of their products give the process variance and autocovariances. We will work with the causal, covariance-stationary AR(1) and make the role of the assumptions explicit.

Model and assumptions: Let $$ y_t=c+\phi y_{t-1}+\varepsilon_t, $$ where $c$ and $\phi$ are constants. Assume the shocks $\varepsilon_t$ have mean zero, variance $\sigma_\varepsilon^2>0$, and zero covariance across distinct dates. For the causal model, also assume each shock is uncorrelated with the process's past, including $y_{t-1}$. A causal solution is constructed from current and past shocks, not future shocks. The stationary causal solution considered here has finite, time-invariant mean and variance.

The shocks need not be independent or normally distributed for the results below: zero means, finite constant variance, and the relevant lack of correlation suffice. The extra condition about the past matters. It is what makes the shocks genuine new disturbances in the causal equation and permits the variance calculations that follow.

Centering the Model and Deriving the Expansion

For a stationary solution, take expectations in the model. Since the shock has mean zero, the stationary mean $\mu=E(y_t)$ must satisfy $\mu=c+\phi\mu$. If $\phi\ne 1$, this gives $\mu=c/(1-\phi)$. Define the demeaned process $x_t=y_t-\mu$. Its equation is

x_t=\phi x_{t-1}+\varepsilon_t.

Substitute the equation for $x_{t-1}$, then continue:

x_t=\varepsilon_t+\phi\varepsilon_{t-1}+\phi^2\varepsilon_{t-2} +\cdots+\phi^{m-1}\varepsilon_{t-m+1}+\phi^m x_{t-m}.

This finite identity holds for every positive integer $m$. If $|\phi|<1$, the coefficient on the remote value $x_{t-m}$ shrinks toward zero. For a covariance-stationary process with finite variance, this remainder vanishes in mean square: $E[(\phi^m x_{t-m})^2]=\phi^{2m}E(x_{t-m}^2)=\phi^{2m}\operatorname{Var}(x_t)$, which tends to zero. Thus the limiting representation is

x_t=\sum_{j=0}^{\infty}\phi^j\varepsilon_{t-j}, \qquad y_t=\mu+\sum_{j=0}^{\infty}\phi^j\varepsilon_{t-j}.

Definition: A linear process is a constant plus a weighted sum of current and past shocks. For the causal AR(1), its weights are $1,\phi,\phi^2,\ldots$. When $|\phi|<1$, these weights are absolutely summable, since $\sum_{j=0}^{\infty}|\phi|^j=1/(1-|\phi|)$, and square summable as well.

The expansion is a limit, not an assertion that a finite number of past shocks exactly determines $x_t$. It says that truncating the sum at a sufficiently remote date leaves a remainder that becomes arbitrarily small in mean square. The condition $|\phi|<1$ ensures this decay regardless of whether $\phi$ is positive or negative.

Worked Example: Expansion and Variance With a Positive Coefficient

Suppose $y_t=4+0.6y_{t-1}+\varepsilon_t$, where the shocks satisfy the stated assumptions and have variance $4$ units squared. Find the mean, write the linear representation, and calculate the variance and first two autocovariances.

Because $|0.6|<1$, the causal stationary formulas apply. The mean is

\mu=\frac{c}{1-\phi}=\frac{4}{1-0.6}=10.

After centering, the model is $x_t=0.6x_{t-1}+\varepsilon_t$. Its infinite representation is $x_t=\varepsilon_t+0.6\varepsilon_{t-1}+0.36\varepsilon_{t-2}+\cdots$, so $y_t$ is this sum plus 10.

The shock terms at distinct dates are uncorrelated. Therefore the variance is the shock variance times the sum of squared weights:

\operatorname{Var}(y_t)=\operatorname{Var}(x_t) =4\sum_{j=0}^{\infty}(0.6)^{2j} =\frac{4}{1-0.6^2} =\frac{4}{0.64} =6.25.

As shown below, $\gamma(h)=\sigma_\varepsilon^2\phi^h/(1-\phi^2)$ for nonnegative $h$. Hence $\gamma(1)=6.25(0.6)=3.75$ and $\gamma(2)=6.25(0.6)^2=2.25$, in squared units of $y_t$. The corresponding autocorrelations are $0.6$ and $0.36$.

Variance and Geometric Autocovariance

Write $\gamma(h)=\operatorname{Cov}(x_t,x_{t-h})$ for the lag-$h$ autocovariance. Centering does not change autocovariances, so these are also the autocovariances of $y_t$. For the variance, use the linear representation and the fact that shocks at distinct dates have zero covariance. Only the terms involving the same shock contribute:

\gamma(0)=\operatorname{Var}(x_t) =\sigma_\varepsilon^2\sum_{j=0}^{\infty}\phi^{2j} =\frac{\sigma_\varepsilon^2}{1-\phi^2}.

To derive the covariance at a positive lag $h$, pair the shocks in $x_t$ and $x_{t-h}$. Their dates agree only when $j=h+k$, where $j$ indexes the shocks in $x_t$ and $k$ indexes those in $x_{t-h}$. All other pairs have covariance zero. Thus, for $h\ge 0$,

\begin{aligned} \gamma(h) &=\operatorname{Cov}\left( \sum_{j=0}^{\infty}\phi^j\varepsilon_{t-j}, \sum_{k=0}^{\infty}\phi^k\varepsilon_{t-h-k} \right)\\ &=\sigma_\varepsilon^2\sum_{k=0}^{\infty}\phi^{h+k}\phi^k\\ &=\sigma_\varepsilon^2\phi^h\sum_{k=0}^{\infty}\phi^{2k} =\frac{\sigma_\varepsilon^2\phi^h}{1-\phi^2}. \end{aligned}

For negative lags, covariance symmetry gives $\gamma(-h)=\gamma(h)$. Equivalently, for any integer $h$, $\gamma(h)=\sigma_\varepsilon^2\phi^{|h|}/(1-\phi^2)$. Dividing by $\gamma(0)$ gives the autocorrelation function $\rho(h)=\phi^{|h|}$. The sign of $\phi$ therefore matters: positive $\phi$ gives positive autocorrelation at every positive lag, while negative $\phi$ gives alternating signs. The magnitude $|\phi|$ controls the rate at which the autocorrelation decays.

Theorem: Under the causal AR(1) assumptions with $|\phi|<1$, the stationary mean is $\mu=c/(1-\phi)$, the variance is $\sigma_\varepsilon^2/(1-\phi^2)$, and the autocovariance and autocorrelation are $$ \gamma(h)=\frac{\sigma_\varepsilon^2\phi^{|h|}}{1-\phi^2}, \qquad \rho(h)=\phi^{|h|}. $$

Worked Example: Negative Coefficient and Alternating Dependence

Suppose $y_t=2-0.5y_{t-1}+\varepsilon_t$, with shock variance $9$ units squared. Assume the shocks satisfy the model conditions. Find the stationary mean, variance, and autocovariances at lags one and two.

Here $|\phi|=0.5<1$, so the causal stationary formulas apply. The mean is $2/(1-(-0.5))=2/1.5=1.333$ units. The variance is

\gamma(0)=\frac{9}{1-(-0.5)^2} =\frac{9}{0.75}=12

in squared units. At lag one, $\gamma(1)=12(-0.5)=-6$, and at lag two, $\gamma(2)=12(-0.5)^2=3$. The autocorrelations are $-0.5$ and $0.25$, respectively. This alternating pattern means that adjacent deviations from the mean tend to have opposite signs, while deviations two periods apart have positive correlation. It does not imply that every adjacent pair must move in opposite directions.

Why the Coefficient Restriction Matters

For the causal representation, $|\phi|<1$ is the stability condition. It makes the powers $\phi^j$ tend to zero and ensures that the geometric sums used for the variance and autocovariance are finite. If $|\phi|=1$, the weights do not decay; if $|\phi|>1$, they grow in magnitude. The proposed sum of past shocks then does not give the convergent causal linear representation derived above.

The restriction also follows from the causal variance equation. Since $\varepsilon_t$ is uncorrelated with $x_{t-1}$, taking variances in $x_t=\phi x_{t-1}+\varepsilon_t$ gives

\operatorname{Var}(x_t) =\phi^2\operatorname{Var}(x_{t-1})+\sigma_\varepsilon^2.

If the process is stationary with finite variance $\gamma(0)$, this becomes $\gamma(0)=\phi^2\gamma(0)+\sigma_\varepsilon^2$, or $(1-\phi^2)\gamma(0)=\sigma_\varepsilon^2$. Because $\sigma_\varepsilon^2>0$ and a variance cannot be negative, this equation requires $\phi^2<1$. It also yields the variance formula above.

This conclusion concerns a causal stationary solution, which is the usual stable AR(1) interpretation. It is not a claim that no stationary process can ever obey an AR(1) equation with $|\phi|>1$. If future shocks are allowed into the representation, a noncausal stationary solution can be constructed for $|\phi|>1$, but the shocks are then correlated with the process's past. For example, with a two-sided white-noise sequence, the centered solution can be written $x_t=-\sum_{j=1}^{\infty}\phi^{-j}\varepsilon_{t+j}$. It is stationary, but it depends on future shocks and does not satisfy the causal innovation condition used here.

Worked Example: Quantifying the Error From Truncating the Expansion

Let $x_t=0.8x_{t-1}+\varepsilon_t$, with shock variance $1$ squared unit. Approximate $x_t$ using only the current and three most recent shocks, and calculate the variance omitted by this truncation.

The four-term approximation is $x_t\approx\varepsilon_t+0.8\varepsilon_{t-1}+0.64\varepsilon_{t-2}+0.512\varepsilon_{t-3}$. Since these shocks are uncorrelated, its variance is

$$ 1(1+0.8^2+0.8^4+0.8^6) =1+0.64+0.4096+0.262144 =2.311744. $$

The full variance is $1/(1-0.8^2)=1/0.36=2.7778$ squared units. The omitted terms begin with $\varepsilon_{t-4}$, so their total variance is

\sum_{j=4}^{\infty}0.8^{2j} =\frac{0.8^8}{1-0.8^2} =\frac{0.16777216}{0.36} =0.4660.

The two pieces sum to $2.311744+0.4660338=2.777778$, up to rounding. The truncation keeps most, but not all, of the process variance; a finite expansion should not be described as exact.

Common Mistake and Practical Interpretation

A common mistake is to see that $\phi$ appears in the equation and conclude that the autocorrelation at lag $h$ is always $\phi^h$, including negative $h$. The standard autocorrelation function is defined at integer lags and is symmetric: $\rho(-h)=\rho(h)$. Its full expression is $\phi^{|h|}$. Another mistake is to say that $|\phi|<1$ proves every AR(1) equation is stationary. The derivation requires the causal assumptions; the equation alone does not rule out noncausal representations or establish that a particular data-generating process meets those assumptions.

Within the causal model, $\phi$ summarizes persistence in deviations from the mean. When $\phi$ is near one, the autocorrelation decays slowly and shocks have effects that persist for many periods. When $\phi$ is near zero, those effects fade quickly. A negative coefficient produces alternating responses. These are statements about the model's dependence structure, not evidence that an observed association is causal.

When using the formulas, check the model conditions first: the shocks have constant finite variance, are uncorrelated across dates and with the past, and the causal coefficient satisfies $|\phi|<1$. Then distinguish the mean level from the centered process, and report variance in squared units and autocovariances in squared units. This prevents the coefficient, the mean, and the scale of the shocks from being conflated.

Check Your Understanding

Answer each question using the causal AR(1) results and assumptions developed above.

For $y_t=3+0.4y_{t-1}+\varepsilon_t$, with shock variance $2$, what are the stationary mean and variance?
If $\phi=-0.7$, what are the autocorrelations at lags one, two, and three? Describe their signs.
In the finite substitution formula, why does the remainder $\phi^m x_{t-m}$ vanish in mean square when $|\phi|<1$ and the process has finite stationary variance?
Why does the causal variance equation rule out a finite-variance stationary solution when $|\phi|>1$ and the shock variance is positive?
How can a stationary solution exist for $|\phi|>1$ without contradicting the causal restriction explained here?