Forecasting ARMA Models by Recursive Substitution

Why Forecasting Requires Substitution

An ARMA equation describes a series using its own past values and current and past innovations. At a forecast origin \(T\), the past values are observed, but the future values and future innovations are not. Forecasting therefore requires a disciplined substitution: retain observed values and residuals where available, and replace future values by forecasts while assigning future innovations their conditional means.

The goal here is the conditional-mean forecast at horizon \(h\), given the information through time \(T\). It is not a prediction interval: it gives a forecast center, not a range describing forecast uncertainty. Forecast error variances and prediction intervals are separate calculations.

Model and Forecasting Information

Use the following ARMA\((p,q)\) convention, with the process written around its unconditional mean \(\mu\):

y_t-\mu = \sum_{i=1}^{p}\phi_i(y_{t-i}-\mu) +\varepsilon_t +\sum_{j=1}^{q}\theta_j\varepsilon_{t-j}.

Here \(y_t\) is the series, \(\phi_i\) are autoregressive coefficients, \(\theta_j\) are moving-average coefficients, and \(\varepsilon_t\) is an innovation with mean zero. This tutorial uses a plus sign before the moving-average terms; the sign convention matters when inserting fitted coefficients.

Definition: Let \(\mathcal F_T\) denote the information available through time \(T\), including observed values through \(T\) and the fitted residuals used as estimates of past innovations. The \(h\)-step conditional-mean forecast is \(\hat y_{T+h\mid T}=E(y_{T+h}\mid\mathcal F_T)\), under the model with its parameters treated as fixed. For forecasting, assume future innovations have conditional mean zero: \(E(\varepsilon_{T+k}\mid\mathcal F_T)=0\) for every \(k\geq1\). A standard sufficient setup is that the innovations are independent of past information, with mean zero.

In practice, \(\mu,\phi_i,\theta_j\), and the past innovations are usually replaced by estimates and fitted residuals. The resulting forecasts are conditional on those fitted quantities. Invertibility is useful for interpreting and obtaining residuals from observed data, while the recursive forecast calculation itself follows directly from the fitted equation.

The Recursive Forecast Rule

For any time index \(s\leq T\), the value \(y_s\) is already observed. For a future index \(s>T\), use the forecast already calculated for that index. Likewise, use a fitted residual \(\hat\varepsilon_s\) for an innovation at or before \(T\), and use zero for the conditional mean of an innovation after \(T\). These substitutions give a single rule for all horizons:

\hat y_{T+h\mid T} = \mu +\sum_{i=1}^{p}\phi_i(\tilde y_{T+h-i\mid T}-\mu) +\sum_{j=1}^{q}\theta_j\tilde\varepsilon_{T+h-j\mid T}, \qquad h\geq1,

where \(\tilde y_{s\mid T}=y_s\) if \(s\leq T\) and \(\tilde y_{s\mid T}=\hat y_{s\mid T}\) if \(s>T\). Also, \(\tilde\varepsilon_{s\mid T}=\hat\varepsilon_s\) if \(s\leq T\) and \(\tilde\varepsilon_{s\mid T}=0\) if \(s>T\). Calculate the forecasts in order: first \(h=1\), then \(h=2\), and so on. Each newly available forecast can enter the next recursion as a lagged value.

Worked Example: One-Step Forecast from an ARMA(1,1)

Suppose a hypothetical fitted model is

y_t-10=0.6(y_{t-1}-10)+\varepsilon_t+0.4\varepsilon_{t-1}.

At the forecast origin, suppose \(y_T=12\) and the fitted residual is \(\hat\varepsilon_T=0.5\). The parameters and mean are treated as fixed, and the future innovation has conditional mean zero. For one step ahead, both lagged terms in the equation refer to information available at \(T\):

\begin{aligned} \hat y_{T+1\mid T} &=10+0.6(y_T-10)+0.4\hat\varepsilon_T\\ &=10+0.6(12-10)+0.4(0.5)\\ &=10+1.2+0.2\\ &=11.4. \end{aligned}

The one-step conditional-mean forecast is 11.4 units of \(y\). The residual term is included because \(\varepsilon_T\) is a past innovation relevant to \(y_{T+1}\); the unobserved \(\varepsilon_{T+1}\) has conditional mean zero and does not add a term to the forecast.

Why the Substitution Works

The recursion follows by taking conditional expectations of the model equation. For a future time \(T+h\), the model gives:

y_{T+h}-\mu = \sum_{i=1}^{p}\phi_i(y_{T+h-i}-\mu) +\varepsilon_{T+h} +\sum_{j=1}^{q}\theta_j\varepsilon_{T+h-j}.

Conditional expectation given \(\mathcal F_T\) can be applied term by term because the equation is linear. A lagged series value whose index is no later than \(T\) is known; a later lagged value has conditional expectation equal to its forecast. An innovation whose index is after \(T\) has conditional expectation zero by assumption. An innovation at or before \(T\) is already represented by its observed or fitted value. Therefore the conditional expectation of the right-hand side is exactly the recursive substitution rule above. This argument justifies the procedure at every horizon, not only one step ahead.

Worked Example: Three Forecasts from an AR(1)

Consider the hypothetical model \(y_t-100=0.8(y_{t-1}-100)+\varepsilon_t\), with \(y_T=110\). Future innovations have conditional mean zero. Since this model has no moving-average terms, its forecasts use only the mean and the latest observed or forecasted value.

One step ahead:

\hat y_{T+1\mid T} =100+0.8(110-100)=108.

Two steps ahead: \(y_{T+1}\) is unavailable, so its forecast is substituted:

\hat y_{T+2\mid T} =100+0.8(108-100)=106.4.

Three steps ahead:

\hat y_{T+3\mid T} =100+0.8(106.4-100)=105.12.

The forecasts move toward the model mean of 100 because each horizon multiplies the previous forecast's deviation from the mean by 0.8. In fact, repeated substitution gives \(\hat y_{T+h\mid T}=100+0.8^h(110-100)\). This expression describes the conditional-mean path for this particular AR(1), not the uncertainty around that path.

Mixed AR and MA Terms Across Horizons

With a moving-average component, a past residual can affect a forecast while it remains among the relevant lags. A future innovation, by contrast, contributes zero to the conditional mean. The autoregressive lags continue to carry earlier deviations forward, using forecasts once the lagged time is beyond \(T\).

Worked Example: Recursive Forecasts from an ARMA(2,1)

Suppose a hypothetical fitted model is

y_t-5=0.5(y_{t-1}-5)-0.2(y_{t-2}-5)+\varepsilon_t+0.3\varepsilon_{t-1}.

At time \(T\), let \(y_T=7\), \(y_{T-1}=6\), and \(\hat\varepsilon_T=0.4\). For \(T+1\), the AR lags are observed and the MA lag is a fitted residual:

\begin{aligned} \hat y_{T+1\mid T} &=5+0.5(7-5)-0.2(6-5)+0.3(0.4)\\ &=5+1.0-0.2+0.12\\ &=5.92. \end{aligned}

For \(T+2\), the first AR lag is the one-step forecast and the second is the observed value \(y_T\). The MA term uses the conditional mean of \(\varepsilon_{T+1}\), which is zero:

\begin{aligned} \hat y_{T+2\mid T} &=5+0.5(5.92-5)-0.2(7-5)+0.3(0)\\ &=5+0.46-0.4\\ &=5.06. \end{aligned}

For \(T+3\), both AR lags are now forecasts, and both MA innovations in the equation are future innovations with conditional mean zero:

\begin{aligned} \hat y_{T+3\mid T} &=5+0.5(5.06-5)-0.2(5.92-5)+0.3(0)\\ &=5+0.03-0.184\\ &=4.846. \end{aligned}

The calculations show the change in inputs across horizons: observed series values and the residual \(\hat\varepsilon_T\) matter initially, but subsequent calculations use forecasts for future series values and zero conditional means for future innovations.

Practical Interpretation and a Common Pitfall

A forecast is conditional on the information available at its origin and on the fitted ARMA specification. It is not a claim that the future observed value will equal the forecast. A realized future innovation will generally differ from zero; setting its conditional expectation to zero is appropriate for the conditional mean, not a statement that the innovation itself is known to be zero.

A frequent error is to set every innovation term to zero when forecasting. That discards the effect of observed past residuals. In the ARMA(1,1) example, this incorrect calculation would give \(10+0.6(12-10)=11.2\), omitting the contribution \(0.4(0.5)=0.2\). The correct one-step forecast is 11.4. The opposite error is to carry the last fitted residual into every future step as if it were a future innovation; after the forecast origin, the conditional mean of each future innovation is zero.

When implementing a forecast, it helps to write down the lag inputs at each horizon explicitly. This prevents mixing an observed value with a forecast, or a past residual with the zero conditional mean assigned to a future innovation. It also makes the calculation easy to check against the signs and lag order in the fitted equation.

Key takeaway: To forecast an ARMA model recursively, use observed values and fitted residuals for lags at or before the origin, use previously calculated forecasts for future series lags, and use zero for the conditional mean of every future innovation.

Check Your Understanding

Use the recursive substitution rule to answer the following questions.

In an ARMA model with the stated plus-sign convention, which innovation values can contribute directly to the one-step forecast at time \(T\)?
For a model \(y_t-\mu=\phi(y_{t-1}-\mu)+\varepsilon_t\), explain why the second forecast uses the first forecast rather than \(y_{T+1}\).
In an ARMA(2,1), what should be substituted for a moving-average innovation whose index is \(T+1\) when forming a forecast conditional on information through \(T\)?
Suppose a one-step calculation omits a nonzero fitted residual \(\hat\varepsilon_T\) multiplied by a nonzero moving-average coefficient. What part of the forecast calculation has been lost?
Does a conditional-mean forecast state the value the series will necessarily take, or describe its expected value given information through the forecast origin?