Forecast Error Variances from the Innovation Weights

Why Innovation Weights Determine Forecast Uncertainty

An ARMA forecast is made using information available at a particular time, but future innovations have not yet been observed. The forecast error therefore comes from the future innovations that enter the outcome after the forecast origin. The model’s innovation weights tell us how strongly each of those future shocks affects the outcome, and their squared magnitudes determine the forecast-error variance.

This calculation answers a different question from producing a point forecast: it measures the variability of the error at a specified horizon. The answer depends on the horizon because a longer-horizon forecast is exposed to more future innovations. Throughout, assume the ARMA parameters are known; accounting for parameter estimation uncertainty is a separate issue.

Setup: The Infinite Moving-Average Representation

Let $X_t$ be a stationary, causal ARMA process with mean $\mu$. Its infinite moving-average representation is

X_t-\mu=\sum_{j=0}^{\infty}\psi_j\varepsilon_{t-j}, \qquad \psi_0=1.

Here $\varepsilon_t$ is the innovation at time $t$, with mean zero and variance $\sigma_\varepsilon^2$. For the variance calculations below, assume the innovations are uncorrelated across time, with the same variance at every time. Also assume they have conditional mean zero given past information; this makes the conditional mean forecast the sum of the known terms. If the innovations are independent, as in a Gaussian ARMA model, their conditional variances are also $\sigma_\varepsilon^2$.

Write $\mathcal{F}_t$ for the information available at time $t$, including past observations and innovations. The $h$-step-ahead forecast is $\widehat{X}_{t+h\mid t}=E(X_{t+h}\mid\mathcal{F}_t)$, for $h\geq 1$. Given the representation above, the terms involving innovations dated $t$ or earlier are known at time $t$, while terms involving later innovations are not.

Definition: The $h$-step forecast error is $e_{t+h\mid t}=X_{t+h}-\widehat{X}_{t+h\mid t}$. Under the stated innovation assumptions, its forecast-error variance is $$ \operatorname{Var}(e_{t+h\mid t})= \sigma_\varepsilon^2\sum_{j=0}^{h-1}\psi_j^2. $$ The coefficients $\psi_0,\ldots,\psi_{h-1}$ are the weights on the $h$ future innovations that can affect $X_{t+h}$.

Deriving the Forecast-Error Variance

At horizon $h$, substitute the moving-average representation at time $t+h$:

X_{t+h}-\mu =\sum_{j=0}^{h-1}\psi_j\varepsilon_{t+h-j} +\sum_{j=h}^{\infty}\psi_j\varepsilon_{t+h-j}.

In the second sum, the most recent innovation is $\varepsilon_t$, so all its terms are known given $\mathcal{F}_t$. Each term in the first sum is a future innovation: its dates run from $t+h$ down to $t+1$. Because each has conditional mean zero, the forecast consists of the mean and the known terms:

\widehat{X}_{t+h\mid t} =\mu+\sum_{j=h}^{\infty}\psi_j\varepsilon_{t+h-j}.

Subtracting this forecast from $X_{t+h}$ leaves exactly the future-innovation terms:

e_{t+h\mid t} =\sum_{j=0}^{h-1}\psi_j\varepsilon_{t+h-j}.

To find the variance, expand the variance of this weighted sum. Since distinct innovations are uncorrelated, every cross-covariance is zero. Each remaining innovation has variance $\sigma_\varepsilon^2$, giving

\begin{aligned} \operatorname{Var}(e_{t+h\mid t}) &=\sum_{j=0}^{h-1}\psi_j^2\operatorname{Var}(\varepsilon_{t+h-j})\\ &=\sigma_\varepsilon^2\sum_{j=0}^{h-1}\psi_j^2. \end{aligned}

This derivation also explains why the weights are squared: variance scales by the square of a coefficient. The formula is for forecasts from the model with its parameters treated as known. If conditional innovation variances change over time, a conditional forecast-error variance may instead need to account for those changing variances.

Worked Calculations

Worked Example: Forecast Variance for an AR(1) Process

Suppose a hypothetical monthly series $X_t$, measured in thousands of units, follows the stationary model $X_t-\mu=0.6(X_{t-1}-\mu)+\varepsilon_t$. Assume $\varepsilon_t$ are independent with mean zero and standard deviation $2$ thousand units, so $\sigma_\varepsilon^2=4$ in squared-thousands-of-units. The AR coefficient satisfies $|0.6|<1$, so the process is causal and its innovation weights are $\psi_j=0.6^j$.

At horizon three, the relevant weights are $\psi_0=1$, $\psi_1=0.6$, and $\psi_2=0.36$. Therefore,

\begin{aligned} \operatorname{Var}(e_{t+3\mid t}) &=4\left(1^2+0.6^2+0.36^2\right)\\ &=4(1+0.36+0.1296)\\ &=5.9584\;(\text{thousand units})^2. \end{aligned}

The forecast-error standard deviation is $\sqrt{5.9584}=2.441$ thousand units. The variance summarizes uncertainty around the three-month-ahead forecast; it does not say that the forecast error will equal this standard deviation.

Worked Example: How an MA(1) Variance Levels Off

Consider a hypothetical quarterly series, measured in points, with model $X_t-\mu=\varepsilon_t+0.5\varepsilon_{t-1}$. Suppose the innovations are independent with variance $9$ points squared. The moving-average weights are $\psi_0=1$, $\psi_1=0.5$, and $\psi_j=0$ for $j\geq2$.

At one quarter ahead, only $\varepsilon_{t+1}$ is unknown. At two quarters ahead, both $\varepsilon_{t+2}$ and $0.5\varepsilon_{t+1}$ contribute. Thus,

\begin{aligned} \operatorname{Var}(e_{t+1\mid t})&=9(1^2)=9\text{ points}^2,\\ \operatorname{Var}(e_{t+2\mid t})&=9(1^2+0.5^2) =9(1.25)=11.25\text{ points}^2. \end{aligned}

For every $h\geq2$, the variance remains $11.25\text{ points}^2$, because all later innovation weights are zero. This leveling off is a feature of the finite moving-average representation, not a general feature of ARMA forecasts.

Worked Example: ARMA(1,1) Weights and a Two-Step Variance

Suppose a hypothetical annual series $X_t$, measured in index points, follows $(1-0.5B)(X_t-\mu)=(1+0.2B)\varepsilon_t$, where $B$ is the lag operator, $BX_t=X_{t-1}$. Assume the innovations are independent with variance $1.44$ points squared. Since the AR coefficient has magnitude less than one, the model has a causal representation.

Expanding the lag-polynomial equation gives $X_t-\mu=0.5(X_{t-1}-\mu)+\varepsilon_t+0.2\varepsilon_{t-1}$. The current innovation has weight $\psi_0=1$. Matching the coefficient on $\varepsilon_{t-1}$ gives $\psi_1=0.5+0.2=0.7$; subsequent weights satisfy $\psi_j=0.5\psi_{j-1}$, so $\psi_2=0.35$. For a two-step forecast, the required weights are only $\psi_0$ and $\psi_1$:

\begin{aligned} \operatorname{Var}(e_{t+2\mid t}) &=1.44(1^2+0.7^2)\\ &=1.44(1.49)\\ &=2.1456\text{ points}^2. \end{aligned}

The weight $\psi_2$ does not enter this two-step variance: the two future innovations affecting $X_{t+2}$ have weights $\psi_0$ and $\psi_1$. It would enter the three-step variance.

Horizon Patterns and Long-Run Behavior

The formula yields useful checks on any calculation. At the first horizon, the forecast error is $\varepsilon_{t+1}$, since $\psi_0=1$; its variance is $\sigma_\varepsilon^2$. Moving from horizon $h$ to horizon $h+1$ adds one squared weight:

\operatorname{Var}(e_{t+h+1\mid t})- \operatorname{Var}(e_{t+h\mid t}) =\sigma_\varepsilon^2\psi_h^2\geq0.

Thus the model-based forecast-error variance cannot decrease as the horizon grows. It stays unchanged at a step only when the newly added weight is zero. In a causal stationary ARMA model the weights are square-summable, so as the horizon grows the variance converges to

\sigma_\varepsilon^2\sum_{j=0}^{\infty}\psi_j^2 =\operatorname{Var}(X_t).

The equality follows because the innovations in the infinite moving-average representation are uncorrelated and each has variance $\sigma_\varepsilon^2$. As the forecast horizon increases, fewer of the terms in $X_{t+h}$ are known at time $t$; in the limit, the forecast-error variance reaches the process variance. This limit is not the variance of every finite-horizon forecast error.

For example, in an AR(1) model with coefficient $\phi$, where $|\phi|<1$, the innovation weights are $\psi_j=\phi^j$. The finite-horizon variance is a geometric sum:

\operatorname{Var}(e_{t+h\mid t}) =\sigma_\varepsilon^2\sum_{j=0}^{h-1}\phi^{2j} =\sigma_\varepsilon^2\frac{1-\phi^{2h}}{1-\phi^2}.

As $h$ tends to infinity, this converges to $\sigma_\varepsilon^2/(1-\phi^2)$, the stationary variance of the AR(1) process. This also shows why strong persistence can make forecast uncertainty build up over many horizons.

Common Mistakes and Practical Checks

Including the wrong number of weights. For horizon $h$, use $\psi_0$ through $\psi_{h-1}$, not through $\psi_h$. The $h$ future shocks have dates $t+1,\ldots,t+h$.
Adding weights rather than their squares. Variance is $\sigma_\varepsilon^2\sum\psi_j^2$, not $\sigma_\varepsilon^2(\sum\psi_j)^2$. The latter incorrectly adds cross-products, which are zero for uncorrelated innovations.
Using the process variance at every horizon. The process variance is the long-horizon limit for a causal stationary process. At a finite horizon, some past innovations are already known, so they do not contribute to forecast error.
Ignoring units. If innovations are measured in points, variances are measured in points squared; the square root of the variance returns to points.

A reliable calculation identifies the model’s innovation weights, selects exactly the first $h$ weights, squares and sums them, and multiplies by the innovation variance. Finally, check that the result is at least $\sigma_\varepsilon^2$ and does not decrease with the horizon.

Check Your Understanding

Use the innovation-weight formula and state the units where relevant.

For an AR(1) model with $\phi=0.4$ and innovation variance $4$, calculate the two-step forecast-error variance.
An MA(1) model has weights $\psi_0=1$, $\psi_1=-0.3$, and all later weights zero. With innovation variance $25$, compare the one-step and three-step forecast-error variances.
Why does the $h$-step formula stop at $\psi_{h-1}$, rather than including $\psi_h$?
Show that the forecast-error variance from horizon $h$ to horizon $h+1$ changes by $\sigma_\varepsilon^2\psi_h^2$. What does this imply?
For a causal stationary ARMA process, what quantity does the forecast-error variance approach as the horizon tends to infinity, and why?