Deriving the ARMA(1,1) Autocovariance

Why Lag One Needs Special Treatment

An ARMA(1,1) process combines a first-order autoregressive term with a current and a lagged innovation. Its autocovariances have a simple pattern from lag two onward, but lag one depends on both model parameters in a way that is easy to overlook. Deriving the moments from the defining equation makes that distinction clear and provides a reliable check on calculations.

We work with a process $X_t$ having mean $\mu$, and center it by defining $Z_t=X_t-\mu$. The convention for the moving-average parameter matters, so the model equation is stated explicitly below.

Definition: A causal ARMA(1,1) process with mean $\mu$ satisfies $$ X_t-\mu=\phi(X_{t-1}-\mu)+\varepsilon_t+\theta\varepsilon_{t-1}, $$ where $\{\varepsilon_t\}$ is white noise with mean zero and variance $\sigma_\varepsilon^2$, and $|\phi|<1$. White noise here means that distinct innovations are uncorrelated and each has the same finite variance. The condition $|\phi|<1$ gives a causal stationary solution. No invertibility condition on $\theta$ is needed for the autocovariance results derived here.

For an integer lag $k$, define the autocovariance by $\gamma_k=\operatorname{Cov}(Z_t,Z_{t-k})$. Weak stationarity means that the mean and variance are constant over time and that the covariance between two observations depends only on their separation. Consequently, $\gamma_{-k}=\gamma_k$, and $\gamma_0=\operatorname{Var}(X_t)$. The autocorrelation at lag $k$ is $\rho_k=\gamma_k/\gamma_0$.

The Infinite Moving-Average Representation

Write the centered equation as $Z_t=\phi Z_{t-1}+\varepsilon_t+\theta\varepsilon_{t-1}$. Repeated substitution gives $Z_t$ in terms of current and past innovations. Since $|\phi|<1$, the resulting geometric series converges in mean square, giving the causal representation

Z_t=\varepsilon_t+(\phi+\theta)\sum_{j=1}^{\infty}\phi^{j-1}\varepsilon_{t-j}.

The coefficient of $\varepsilon_t$ is 1. For each $j\geq 1$, the coefficient of $\varepsilon_{t-j}$ is $(\phi+\theta)\phi^{j-1}$. Because distinct white-noise innovations are uncorrelated, the variance is the innovation variance multiplied by the sum of the squared coefficients. The geometric sum converges because $\phi^2<1$.

\begin{aligned} \gamma_0 &=\sigma_\varepsilon^2\left[1+(\phi+\theta)^2 \sum_{j=1}^{\infty}\phi^{2(j-1)}\right]\\ &=\sigma_\varepsilon^2\left[1+\frac{(\phi+\theta)^2}{1-\phi^2}\right]\\ &=\sigma_\varepsilon^2\frac{1-\phi^2+(\phi+\theta)^2}{1-\phi^2}\\ &=\sigma_\varepsilon^2\frac{1+2\phi\theta+\theta^2}{1-\phi^2}. \end{aligned}

This derivation uses only the white-noise mean, variance, and lack of correlation across time; normality of the innovations is not required. The same infinite representation also lets us identify which innovations overlap when calculating a covariance.

Deriving Lag One and the Higher Lags

Start with the model equation and take covariance with $Z_{t-1}$. The innovation $\varepsilon_t$ is uncorrelated with $Z_{t-1}$, which depends only on present and earlier innovations. Meanwhile, the coefficient of $\varepsilon_{t-1}$ in $Z_{t-1}$ is 1, so $\operatorname{Cov}(\varepsilon_{t-1},Z_{t-1})=\sigma_\varepsilon^2$. Thus the moving-average term contributes $\theta\sigma_\varepsilon^2$, not zero.

\begin{aligned} \gamma_1 &=\operatorname{Cov}(Z_t,Z_{t-1})\\ &=\phi\gamma_0+ \operatorname{Cov}(\varepsilon_t+\theta\varepsilon_{t-1},Z_{t-1})\\ &=\phi\gamma_0+\theta\sigma_\varepsilon^2\\ &=\sigma_\varepsilon^2 \frac{(\phi+\theta)(1+\phi\theta)}{1-\phi^2}. \end{aligned}

The final equality follows by substituting the variance expression and collecting terms: $\phi(1+2\phi\theta+\theta^2)+\theta(1-\phi^2)=(\phi+\theta)(1+\phi\theta)$. This special lag-one expression includes the direct contribution from the lagged innovation.

For $k\geq 2$, take covariance of the model equation with $Z_{t-k}$. Both $\varepsilon_t$ and $\varepsilon_{t-1}$ are uncorrelated with $Z_{t-k}$: that earlier observation is built only from innovations dated $t-k$ and earlier. The two innovation terms therefore contribute zero, leaving a recurrence.

\gamma_k=\phi\gamma_{k-1}\quad(k\geq2), \qquad \gamma_k=\phi^{k-1}\gamma_1\quad(k\geq1).

The second formula follows by repeated application of the recurrence, starting with the separately derived value of $\gamma_1$. It does not say that $\gamma_1=\phi\gamma_0$: that would omit the lagged innovation’s covariance contribution. Dividing by $\gamma_0$ gives the corresponding autocorrelations:

\rho_1=\frac{(\phi+\theta)(1+\phi\theta)}{1+2\phi\theta+\theta^2}, \qquad \rho_k=\phi^{k-1}\rho_1\quad(k\geq2).

Worked Moment Calculations

Worked Example: A Positive Lag-One Covariance

Suppose $X_t$ follows the centered ARMA(1,1) equation with mean $\mu=10$, $\phi=0.5$, and $\theta=0.25$. Let the innovations be independent $N(0,4)$, so $\sigma_\varepsilon^2=4$. Find the variance and autocovariances at lags one, two, and three. The method is to calculate $\gamma_0$, use the special lag-one equation, and then apply the higher-lag recurrence. Independence is stronger than the white-noise assumption needed for these covariance calculations.

Variance. Substitute the parameters into the variance formula:

\begin{aligned} \gamma_0 &=4\frac{1+2(0.5)(0.25)+(0.25)^2}{1-(0.5)^2}\\ &=4\frac{1+0.25+0.0625}{0.75}\\ &=4\frac{1.3125}{0.75}=4(1.75)=7. \end{aligned}

Lag one. The direct covariance formula gives

\gamma_1=\phi\gamma_0+\theta\sigma_\varepsilon^2 =(0.5)(7)+(0.25)(4)=3.5+1=4.5.

Lags two and three. Apply the recurrence once for each additional lag:

\gamma_2=(0.5)(4.5)=2.25, \qquad \gamma_3=(0.5)(2.25)=1.125.

The check is that the variance is positive and the higher-lag values halve at each step, as required by $\gamma_k=\phi\gamma_{k-1}$. The method avoids treating the lag-one covariance as just $\phi\gamma_0$; that incorrect calculation would give $3.5$, missing the additional $1$ from the lagged innovation.

Worked Example: A Negative Autoregressive Coefficient

Let the mean be $\mu=-2.5$, $\phi=-0.5$, and $\theta=1$. Suppose the innovations are independent $N(0,3)$, so their variance is 3. Find $\gamma_0$ through $\gamma_3$, and the autocorrelation at lag one. Use the variance formula first and retain the sign of $\phi$ when calculating later lags.

Variance. Here $2\phi\theta=2(-0.5)(1)=-1$, so the numerator $1+2\phi\theta+\theta^2$ is $1-1+1=1$. The denominator is $1-\phi^2=1-0.25=0.75$. Therefore,

\gamma_0=3\frac{1}{0.75}=4.

Lag one. Use the special expression in its direct form:

\gamma_1=(-0.5)(4)+(1)(3)=-2+3=1.

Lags two and three. Each step multiplies the previous autocovariance by $-0.5$:

\gamma_2=(-0.5)(1)=-0.5, \qquad \gamma_3=(-0.5)(-0.5)=0.25.

The lag-one autocorrelation is $\rho_1=\gamma_1/\gamma_0=1/4=0.25$. The checks are that $\gamma_0=4$ is positive and the signs of the higher-lag covariances alternate, consistent with a negative $\phi$. The mean does not enter these centered covariance calculations.

Worked Example: Converting Autocovariances to Correlations

Consider a process with mean $\mu=6$, $\phi=0.5$, and $\theta=0.5$. Let its innovations be independent $N(0,2)$. Calculate $\gamma_0,\gamma_1,\gamma_2,\gamma_3$, then calculate $\rho_1$ and $\rho_3$. The method is to find the autocovariances before dividing by the variance; the common mistake avoided is using $\gamma_1=\phi\gamma_0$.

Variance. The numerator factor is $1+2(0.5)(0.5)+(0.5)^2=1+0.5+0.25=1.75$, and the denominator is $1-0.25=0.75$. Hence

\gamma_0=2\frac{1.75}{0.75} =2\frac{7}{3}=\frac{14}{3}.

Lag one. The lagged innovation term contributes $(0.5)(2)=1$:

\gamma_1=(0.5)\frac{14}{3}+(0.5)(2) =\frac{7}{3}+1=\frac{10}{3}.

Higher lags. Repeatedly multiply by $\phi=0.5$:

\gamma_2=(0.5)\frac{10}{3}=\frac{5}{3}, \qquad \gamma_3=(0.5)\frac{5}{3}=\frac{5}{6}.

Correlations. Divide each requested autocovariance by $\gamma_0=14/3$:

\rho_1=\frac{10/3}{14/3}=\frac{5}{7}, \qquad \rho_3=\frac{5/6}{14/3}=\frac{5}{28}.

As a check, $\rho_3=(0.5)^2\rho_1=5/28$, in agreement with the higher-lag formula. The incorrect shortcut $\gamma_1=\phi\gamma_0$ would give $7/3$, rather than the correct $10/3$.

Common Mistakes and Practical Checks

The defining equation identifies exactly which terms contribute to each covariance. In particular, at lag one, $\varepsilon_{t-1}$ is also part of $Z_{t-1}$, so the two terms are correlated. At lag two and beyond, both innovations in the equation are later than the innovations that form $Z_{t-k}$, so they contribute zero under the causal representation.

Dropping the moving-average contribution at lag one. The correct equation is $\gamma_1=\phi\gamma_0+\theta\sigma_\varepsilon^2$, not $\gamma_1=\phi\gamma_0$.
Applying the higher-lag recurrence too early. The recurrence $\gamma_k=\phi\gamma_{k-1}$ starts at $k=2$. The lag-one value must be calculated separately.
Changing the sign convention for $\theta$. These formulas use $+\theta\varepsilon_{t-1}$. If a model is written with a minus sign, first rewrite it in this convention before substituting.
Confusing the mean and the variance calculation. Centering removes $\mu$ from the covariance formulas; it does not mean the original process has mean zero.
Assuming invertibility is needed for the moments. The stated covariance derivation uses $|\phi|<1$ and white-noise innovations. It does not require an invertibility restriction on $\theta$.

Key result: For the causal stationary model $X_t-\mu=\phi(X_{t-1}-\mu)+\varepsilon_t+\theta\varepsilon_{t-1}$, the variance is $\gamma_0=\sigma_\varepsilon^2(1+2\phi\theta+\theta^2)/(1-\phi^2)$. Lag one is $\gamma_1=\phi\gamma_0+\theta\sigma_\varepsilon^2$, while every lag $k\geq2$ satisfies $\gamma_k=\phi\gamma_{k-1}$. These formulas are aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Check Your Understanding

For each problem, use the stated plus-sign convention for $\theta$. Answers are given so you can check your calculations.

A centered ARMA(1,1) process has $\phi=0.2$, $\theta=0.3$, and independent $N(0,4)$ innovations. Find $\gamma_0,\gamma_1,\gamma_2$. (Answer: $\gamma_0=121/24$, $\gamma_1=53/24$, $\gamma_2=53/120$.)
A process has $\mu=3$, $\phi=-0.25$, $\theta=0.5$, and independent $N(0,4)$ innovations. Find $\gamma_0,\gamma_1,\gamma_2$. (Answer: $\gamma_0=64/15$, $\gamma_1=14/15$, $\gamma_2=-7/30$.)
For $\phi=0.6$, $\theta=0.2$, and independent $N(0,2)$ innovations, find the variance, the lag-one autocovariance, and the lag-three autocovariance. (Answer: $\gamma_0=4$, $\gamma_1=2.8$, $\gamma_3=1.008$.)
A process has $\phi=0.4$, $\theta=-0.4$, and independent $N(0,5)$ innovations. Calculate $\gamma_0,\gamma_1,\gamma_2$ using the formulas, and state what happens to the lag-two covariance. (Answer: $\gamma_0=5$, $\gamma_1=0$, $\gamma_2=0$.)
For $\phi=-0.2$, $\theta=0.4$, and independent $N(0,2)$ innovations, calculate $\gamma_0,\gamma_1,\gamma_3$, and $\rho_1$. (Answer: $\gamma_0=25/12$, $\gamma_1=23/60$, $\gamma_3=23/1500$, $\rho_1=23/125$.)

What You'll Learn