Causality and Invertibility in ARMA Models

Why Causality and Invertibility Matter

An ARMA model relates a time series to its past values and to a sequence of unobserved innovations. Causality and invertibility describe two different directions in that relationship. Causality asks whether the current observation can be expressed using present and past innovations. Invertibility asks whether the current innovation can be recovered using present and past observations. These properties give the model representations that make its dependence on past shocks and its innovations interpretable.

The distinction matters in applications such as hypothetical insurance claim-frequency or financial-risk series. A fitted model may describe dependence through its equations, but the infinite representations clarify which historical shocks drive current observations and whether the shocks can be reconstructed from observed data. Neither property should be confused with the other, or with stationarity alone.

Definition: Consider a mean-zero ARMA($p,q$) process $$ \phi(B)X_t=\theta(B)\varepsilon_t, $$ where $B X_t=X_{t-1}$, $\phi(B)=1-\phi_1B-\cdots-\phi_pB^p$, and $\theta(B)=1+\theta_1B+\cdots+\theta_qB^q$. The innovations $\varepsilon_t$ are white noise with mean zero and variance $\sigma^2$. The process is causal if it has a representation $X_t=\sum_{j=0}^{\infty}\psi_j\varepsilon_{t-j}$ with absolutely summable coefficients. It is invertible if it has a representation $\varepsilon_t=\sum_{j=0}^{\infty}\pi_jX_{t-j}$ with absolutely summable coefficients. In both definitions, the representations are understood in mean square.

Here “mean square” means convergence in the sense that the expected squared difference between the infinite series and its limit tends to zero. Absolute summability, such as $\sum_{j=0}^{\infty}|\psi_j|<\infty$, is a standard sufficient condition for these stable representations. The definitions concern the direction and convergence of the representations: causality uses innovations to express observations, while invertibility uses observations to express innovations.

How the Representations Follow From the Model

If the AR polynomial can be inverted as a convergent power series, the ARMA equation gives $X_t=\{\theta(B)/\phi(B)\}\varepsilon_t$. The coefficients in this quotient are the causal moving-average coefficients. Conversely, if the MA polynomial can be inverted as a convergent power series, then $\varepsilon_t=\{\phi(B)/\theta(B)\}X_t$. The resulting coefficients express the innovations in terms of current and past observations.

For causality, invert the AR polynomial.
Expand $1/\phi(B)$ as a power series and multiply by $\theta(B)$. The resulting coefficients must define a convergent representation in present and past innovations.

For invertibility, invert the MA polynomial.
Expand $1/\theta(B)$ as a power series and multiply by $\phi(B)$. The resulting coefficients must define a convergent representation in present and past observations.

Check the relevant polynomial.
Causality concerns $\phi$, the AR polynomial; invertibility concerns $\theta$, the MA polynomial. A condition on one does not establish the other.

For the usual ARMA parameterization, the standard conditions are that the zeros of $\phi(z)$ lie outside the unit circle for a causal representation, and the zeros of $\theta(z)$ lie outside the unit circle for an invertible representation. A zero on or inside the unit circle prevents the corresponding stable one-sided power-series inverse. The focus here is the meaning of these conditions and the representations they support.

Worked Example: Both Representations for an ARMA(1,1)

Suppose a hypothetical centered quarterly risk index follows $$ (1-0.5B)X_t=(1+0.4B)\varepsilon_t, $$ where $\varepsilon_t$ is white noise with variance $4$. Determine the causal moving-average coefficients, the invertible autoregressive coefficients, and the variance of $X_t$. State which common mistake the calculations avoid.

Method: power-series expansion. For causality, expand the reciprocal of the AR polynomial. Since $|0.5|<1$, $$ \frac{1}{1-0.5B}=1+0.5B+0.25B^2+0.125B^3+\cdots. $$ Multiplying by $1+0.4B$ gives $$ \frac{1+0.4B}{1-0.5B} =1+0.9B+0.45B^2+0.225B^3+\cdots. $$ Thus $\psi_0=1$, and for $j\geq 1$, $\psi_j=0.9(0.5)^{j-1}$. The absolute sum is $1+0.9/(1-0.5)=2.8$, so the coefficients are absolutely summable and the process is causal.

For invertibility, solve for the innovations: $$ \varepsilon_t=\frac{1-0.5B}{1+0.4B}X_t. $$ Because $|-0.4|<1$, expand $1/(1+0.4B)=1-0.4B+0.16B^2-0.064B^3+\cdots$. Multiplying by $1-0.5B$ gives $$ \varepsilon_t=X_t-0.9X_{t-1}+0.36X_{t-2}-0.144X_{t-3}+\cdots. $$ For $j\geq1$, $\pi_j=-0.9(-0.4)^{j-1}$, whose absolute values have finite sum. The process is invertible as well.

Finally, the causal representation has uncorrelated innovation terms, so $$ \operatorname{Var}(X_t)=4\sum_{j=0}^{\infty}\psi_j^2 =4\left(1+\frac{0.9^2}{1-0.5^2}\right) =4\left(1+\frac{0.81}{0.75}\right) =8.32. $$ The variance is $8.32$. This result checks the calculation using the full coefficient sequence, not just its first few terms. The common mistake avoided is treating the same polynomial as the test for both properties: the AR polynomial establishes causality here, while the MA polynomial establishes invertibility.

Deriving the Coefficient Recursion

The infinite moving-average coefficients are not arbitrary. Their recurrence follows by substituting the representation into the ARMA equation and matching powers of the lag operator. This also explains why the AR and MA coefficients both affect the early terms of the representation.

Derivation: Suppose $X_t=\sum_{j=0}^{\infty}\psi_j\varepsilon_{t-j}$, with $\psi_0=1$. Then $\phi(B)\sum_{j=0}^{\infty}\psi_jB^j=\theta(B)$. Equating the coefficient of $B^j$, and taking $\phi_k=0$ for $k>p$ and $\theta_j=0$ for $j>q$, gives $$ \psi_j-\sum_{k=1}^{\min(p,j)}\phi_k\psi_{j-k}=\theta_j,\qquad j\geq1. $$ The left side is the coefficient of $B^j$ in $\phi(B)\psi(B)$, and the right side is the coefficient of $B^j$ in $\theta(B)$. This coefficient matching proves the recursion.

For an ARMA(1,1), the recurrence is $\psi_j-\phi_1\psi_{j-1}=\theta_j$. For $j>1$, $\theta_j=0$, so the later coefficients satisfy $\psi_j=\phi_1\psi_{j-1}$. A decaying geometric sequence results when $|\phi_1|<1$. At $j=1$, however, the MA coefficient also enters: $\psi_1=\phi_1+\theta_1$. Ignoring that initial contribution gives an incorrect expansion.

Worked Example: Reconstructing Innovations From Observations

A hypothetical monthly index is modeled as $X_t=(1+0.5B)\varepsilon_t$, with white-noise innovations. Find the invertibility expansion. Then use $X_t=12$, $X_{t-1}=10$, $X_{t-2}=8$, $X_{t-3}=6$, and $X_{t-4}=4$ to calculate the estimate obtained by retaining terms through $X_{t-4}$. Name the method and the limitation of this calculation.

Method: invert the MA polynomial and truncate the convergent series. Since $|0.5|<1$, $$ \varepsilon_t=\frac{1}{1+0.5B}X_t =(1-0.5B+0.25B^2-0.125B^3+0.0625B^4-\cdots)X_t. $$ The coefficient magnitudes form a geometric series with ratio $0.5$, so the infinite representation converges and the model is invertible.

Substitute the five supplied observations into the terms through lag four: $$ 12-(0.5)(10)+(0.25)(8)-(0.125)(6)+(0.0625)(4) =12-5+2-0.75+0.25 =8.5. $$ Thus the four-lag truncation is $8.5$. This is an approximation to the infinite representation, not necessarily the exact innovation: terms involving $X_{t-5},X_{t-6},\ldots$ remain. The common mistake avoided is to call a finite truncation exact merely because several observations were supplied.

Stationarity Does Not Mean Causality

Causality is a one-sided representation in past innovations. A stationary process can fail to have that representation and still have a well-defined stationary solution involving future innovations. Such a solution is not causal under the definition above. This distinction is important: stationarity concerns the behavior of the process over time, while causality concerns how it can be represented using the innovation sequence.

Worked Example: A Stationary but Noncausal AR(1)

A hypothetical centered index satisfies $X_t=2X_{t-1}+\varepsilon_t$, where the innovations are white noise with variance $1$. Determine whether the model is causal. If a stationary solution is represented using future innovations, derive that representation and its variance.

Method: test for a past-innovation expansion, then verify a future-innovation solution. The AR polynomial is $1-2B$. Its zero is $z=1/2$, inside the unit circle, so the usual causal condition fails. Directly expanding $1/(1-2B)$ as a power series in nonnegative powers of $B$ would give coefficients $1,2,4,8,\ldots$, which are not absolutely summable. Therefore this model is not causal.

Consider instead $$ X_t=-\sum_{j=1}^{\infty}2^{-j}\varepsilon_{t+j}. $$ The coefficients are square summable, since $\sum_{j=1}^{\infty}4^{-j}=1/3$, so the series converges in mean square. To check the model equation, write $$ 2X_{t-1}+\varepsilon_t =-\sum_{j=1}^{\infty}2^{1-j}\varepsilon_{t-1+j}+\varepsilon_t =-\varepsilon_t-\sum_{k=1}^{\infty}2^{-k}\varepsilon_{t+k}+\varepsilon_t =X_t. $$ The equation holds, but $X_t$ depends on future innovations, not just present and past ones. Its variance is $$ \operatorname{Var}(X_t)=\sum_{j=1}^{\infty}4^{-j} =\frac{1/4}{1-1/4} =\frac13. $$ Thus a stationary solution exists, with finite variance $1/3$, but it is noncausal. The trap is to infer causality from stationarity alone.

Common Mistakes and Practical Interpretation

Checking the wrong polynomial. Causality is governed by the AR polynomial $\phi$; invertibility is governed by the MA polynomial $\theta$. Passing one check does not imply passing the other.
Reversing the direction of the representation. Causality expresses $X_t$ in terms of current and past $\varepsilon$'s. Invertibility expresses $\varepsilon_t$ in terms of current and past $X$'s.
Using only a few terms to establish convergence. A finite list of coefficients cannot establish that the entire series converges. Check the general coefficient pattern or the relevant polynomial condition.
Using the wrong coefficient at the first lag. In an ARMA(1,1), $\psi_1=\phi_1+\theta_1$ under the convention used here. The later geometric pattern does not, by itself, give this first coefficient.
Equating stationarity and causality. A stationary process may have a representation involving future innovations rather than the past-innovation representation required for causality.

Key takeaway: Causality supplies a stable infinite moving-average representation of observations in present and past innovations; invertibility supplies a stable infinite autoregressive representation of innovations in present and past observations. Keep the two directions, and the AR-versus-MA polynomial checks, separate.

These representations are useful for interpreting shock propagation, reconstructing innovations, and calculating moments from model coefficients. The distinction is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus: a correct answer must identify which property is being tested and explain what representation that property supports.

Check Your Understanding

For each problem, identify the relevant property and use the specified polynomial or representation rather than relying on a visual pattern alone.

A centered AR(1) process satisfies $X_t=0.25X_{t-1}+\varepsilon_t$, with white-noise variance $9$. Give its causal coefficients and calculate $\operatorname{Var}(X_t)$. (Answer: $\psi_j=0.25^j$ for $j\geq0$; causal; $\operatorname{Var}(X_t)=9.6$.)
An MA(1) model is $X_t=(1-0.4B)\varepsilon_t$. State whether it is invertible and calculate the estimate of $\varepsilon_t$ through lag two when $X_t=5$, $X_{t-1}=2$, and $X_{t-2}=-1$. (Answer: Invertible; $\varepsilon_t\approx5.64$ through lag two, using $5+0.4(2)+0.16(-1)$.)
An ARMA(1,1) model has $\phi(B)=1-0.3B$ and $\theta(B)=1+0.2B$. Give $\psi_0,\psi_1,\psi_2,\psi_3$, and state whether the process is causal. (Answer: $\psi_0=1,\ \psi_1=0.5,\ \psi_2=0.15,\ \psi_3=0.045$; causal, with $\psi_j=0.5(0.3)^{j-1}$ for $j\geq1$.)
An AR(1) process satisfies $X_t=1.2X_{t-1}+\varepsilon_t$, with white-noise variance $1$. Is it causal? If a stationary solution is represented using future innovations, what is its variance? (Answer: Not causal; the future-innovation representation is $X_t=-\sum_{j=1}^{\infty}1.2^{-j}\varepsilon_{t+j}$, with variance $1/(1.2^2-1)=2.2727$ to four decimals.)
An MA(1) model is $X_t=(1+1.5B)\varepsilon_t$. Determine whether it is invertible and state what happens to the coefficients in a proposed one-sided inverse expansion. (Answer: Not invertible; the inverse coefficients are $(-1.5)^j$, which do not tend to zero and are not absolutely summable.)

What You'll Learn