Prediction Intervals for ARMA Forecasts

Why Prediction Intervals Need an Information Set

An ARMA forecast gives a point prediction for a future observation, but it does not describe how far that observation may fall from the prediction. A prediction interval adds that uncertainty. Its calculation depends on the distribution of the forecast error, and that distribution must be defined relative to the information available when the forecast is made.

For ARMA models, a useful representation expresses observations as weighted sums of innovations. The forecast-error variance is then built from the weights on future innovations. However, this calculation is valid for prediction from observed history only when the innovations used in the representation are recoverable from that history. Causality alone does not guarantee this; invertibility matters.

Throughout, suppose the process is a stationary, purely nondeterministic ARMA process with known parameters and mean \(\mu\), and has the innovation representation

Assume the process is causal and invertible, so that with the infinite observed past the innovations are recoverable from observed history. Also assume the innovations are independent over time, have mean zero and variance \(\sigma^2\), and are independent of the past. For Gaussian prediction intervals, add the assumption \(\varepsilon_t\sim N(0,\sigma^2)\). Known parameters are assumed here; intervals that account for parameter estimation are a separate issue.

From Forecast-Error Variance to a Gaussian Interval

Under these conditions, the conditional forecast error consists of the weighted innovations arriving after time \(T\):

These innovations are independent of \(\mathcal{F}_T^X\). Therefore the forecast error has conditional mean zero and variance

If the innovations are Gaussian, the forecast error is a weighted sum of independent Gaussian variables, so its conditional distribution is \(N(0,V_h)\). Writing \(z_{1-\alpha/2}\) for the \(1-\alpha/2\) standard normal quantile, the central \(100(1-\alpha)\%\) prediction interval is

For a 95% interval, \(z_{0.975}\) is approximately 1.96. Under the stated assumptions, this interval has 95% conditional predictive coverage: repeated future observations generated under the model and the same information at time \(T\) fall inside it with probability 0.95. It is not an interval for the unknown mean or for an ARMA parameter.

How Uncertainty Changes with the Forecast Horizon

The variance formula gives a direct comparison between successive horizons:

Thus forecast-error variance cannot decrease as the horizon grows. It increases strictly whenever the newly added weight \(\psi_h\) is nonzero; it may stay unchanged at a particular step if that weight is zero. For a stationary causal ARMA process, the weights are square-summable, so the variance approaches the finite limit \(\sigma^2\sum_{j=0}^{\infty}\psi_j^2\). This is the process variance. As the horizon grows, the forecast generally loses information from the observed past and uncertainty approaches the unconditional variation of the process.

Nonnormal Innovations: Use Predictive Quantiles

When innovations are independent but nonnormal, the weighted-sum variance formula still holds, but the forecast error need not be normal. Substituting a normal critical value then does not generally give the advertised coverage. Instead, obtain quantiles of the forecast-error distribution under the fitted innovation law. One approach simulates many independent future innovation sequences and computes the corresponding weighted sums. An innovation bootstrap does this by resampling centered fitted residuals as future innovations. These methods approximate the predictive distribution; with discrete or small samples, their realized coverage need not equal the nominal level exactly.

Common Pitfall: Causality Is Not Enough

A tempting but incorrect argument is: “The ARMA process is causal, so the past innovations are known from past observations; therefore the error variance is \(\sigma^2\sum_{j=0}^{h-1}\psi_j^2\) for an observed-history forecast.” Causality gives a representation in terms of present and past innovations, but does not by itself make past innovations recoverable from observations. Invertibility, or another appropriate condition on the information set, is needed for that step.

For example, the causal Gaussian MA(1) process \(X_t=\varepsilon_t+2\varepsilon_{t-1}\) is not invertible in this representation. Treating \(\varepsilon_{T-1}\) as known from observed history would give a one-step error variance of \(\sigma^2\). But the same observed process has an invertible representation with coefficient 0.5 and innovation variance \(4\sigma^2\), giving observed-history one-step forecast-error variance \(4\sigma^2\). The distinction is between conditioning on past innovations and conditioning on observed history. State which information set is being used before interpreting an interval.