Forecasts with Estimated ARMA Parameters

Why Estimated Parameters Matter for Forecasts

An ARMA forecast is usually calculated by substituting estimated parameters into a forecasting formula. A prediction interval then commonly uses the estimated model’s future innovation variance as though those parameter estimates were exact. That calculation accounts for uncertainty about future shocks, but not for the fact that another sample could have produced different parameter estimates and therefore a different forecast.

The distinction is important even when the fitted ARMA model is appropriate. At short horizons, the additional uncertainty may be modest; at longer horizons, small parameter changes can materially alter the forecast. The effect depends on the forecast horizon and on how sensitive the forecast is to the parameters. This tutorial focuses on incorporating that sensitivity into forecast distributions and intervals, not on choosing an ARMA order or evaluating forecasts across time origins.

Definition: Let $\theta$ denote the vector of ARMA parameters, including any fitted mean or innovation-variance parameters. Write $m_h(\theta)$ for the conditional mean forecast $h$ periods ahead given the observations through time $T$, and $V_h(\theta)$ for the corresponding forecast-error variance when $\theta$ is known. A plug-in forecast is $m_h(\hat{\theta})$. A plug-in prediction interval uses estimated parameters in the forecast and future-innovation variance but treats them as fixed; it does not directly include uncertainty in $\hat{\theta}$.

For a correctly specified, causal and invertible ARMA model with parameters in the interior of the admissible region, suppose the estimator is approximately normal in a large sample:

\hat{\theta} \ \dot{\sim}\ N(\theta,\widehat{\operatorname{Var}}(\hat{\theta})).

Here $\widehat{\operatorname{Var}}(\hat{\theta})$ can be estimated from the likelihood information matrix or another appropriate covariance estimator. The earlier material on standard errors and Wald inference for ARMA parameters gives the parameter-estimation context. For a fixed forecast horizon, the central approximation used here is that uncertainty in the forecast center is determined by how $m_h(\theta)$ changes with $\theta$.

A First-Order Approximation

Let $g_h$ be the gradient of the forecast mean with respect to the parameter vector, evaluated at the fitted estimate:

g_h(\hat{\theta})= \left.\frac{\partial m_h(\theta)}{\partial\theta}\right|_{\theta=\hat{\theta}}.

A first-order Taylor expansion gives $m_h(\hat{\theta})$ approximately equal to $m_h(\theta)+g_h(\theta)'(\hat{\theta}-\theta)$. Thus the estimated forecast center has approximate variance $g_h(\theta)'\operatorname{Var}(\hat{\theta})g_h(\theta)$, estimated by substituting the fitted quantities. This is the parameter-estimation contribution to uncertainty in the forecast center.

Approximation: Under the large-sample normal approximation for $\hat{\theta}$, and when future innovations are independent of the estimation sample, a first-order approximation to the repeated-sample forecast-error variance is $$ \widehat{\operatorname{Var}}(Y_{T+h}-m_h(\hat{\theta})) \ \approx\ \hat{V}_h+ g_h(\hat{\theta})' \widehat{\operatorname{Var}}(\hat{\theta}) g_h(\hat{\theta}), $$ where $\hat{V}_h=V_h(\hat{\theta})$ is the plug-in future-innovation forecast-error variance. The second term is nonnegative and represents forecast-center uncertainty.

This approximation has a useful variance-decomposition interpretation. If parameters are regarded as uncertain across repeated estimation samples, variation in the future outcome arises both from future innovations and from variation in the forecast center. The formula adds those two contributions to first order. It is an approximation, not an exact finite-sample identity for every fitting procedure. It is most defensible when the estimator is approximately normal, the horizon is fixed, the model is not near a parameter boundary, and the forecast function is sufficiently smooth.

Calculate the plug-in forecast.
Evaluate $m_h(\hat{\theta})$ using the fitted ARMA parameters and observed data.

Find forecast sensitivity.
Calculate or numerically approximate the gradient $g_h(\hat{\theta})$.

Estimate added variance.
Compute $g_h(\hat{\theta})'\widehat{\operatorname{Var}}(\hat{\theta})g_h(\hat{\theta})$.

Form an approximate interval.
Add that contribution to $\hat{V}_h$, take the square root, and use an appropriate quantile if a normal approximation is reasonable.

Worked Examples

Worked Example: Added Uncertainty in a Two-Step AR(1) Forecast

Suppose a hypothetical stationary AR(1) model has known mean $\mu=50$ and innovation standard deviation $\sigma=2$. Its fitted autoregressive coefficient is $\hat{\phi}=0.60$, with estimated standard error $0.10$, and the current observation is $Y_T=60$. Treat $\mu$ and $\sigma$ as known for this calculation. For horizon $h=2$, the forecast mean is $m_2(\phi)=\mu+\phi^2(Y_T-\mu)$.

Plug-in forecast. Substituting the values gives $50+(0.60)^2(60-50)=50+3.60=53.60$.

Future-innovation variance. The two-step AR(1) forecast error is the sum of the next innovation and the following innovation multiplied by $\phi$. Its plug-in variance is $4[1+(0.60)^2]=4(1.36)=5.44$ squared outcome units.

Parameter contribution. The derivative of the forecast mean is $2\phi(Y_T-\mu)$, so at the fitted value it is $2(0.60)(10)=12$. The estimated variance of $\hat{\phi}$ is $(0.10)^2=0.0100$. Therefore the added variance is $12^2(0.0100)=1.44$ squared outcome units.

The total approximate variance is $5.44+1.44=6.88$, and the standard error is $\sqrt{6.88}=2.623$ outcome units. Under a normal approximation, a 95% interval is $53.60\pm1.96(2.623)$, or approximately $[48.46,58.74]$. A plug-in interval that ignored coefficient uncertainty would instead use $5.44$, giving standard error $2.332$ and interval $[49.03,58.17]$. The wider first interval reflects uncertainty in the fitted forecast center as well as future innovations. These are approximate repeated-sample intervals under the stated model and assumptions, not probabilities assigned to a fixed parameter value.

Worked Example: Combining Uncertainty in an Estimated Mean and AR Coefficient

Consider a hypothetical AR(1) model written $Y_t-\mu=\phi(Y_{t-1}-\mu)+\varepsilon_t$. At horizon one, its forecast is $m_1(\mu,\phi)=\mu+\phi(Y_T-\mu)$. Suppose $\hat{\mu}=100$, $\hat{\phi}=0.50$, $Y_T=108$, and the fitted innovation standard deviation is $3$. The estimated covariance matrix for $(\hat{\mu},\hat{\phi})$ is

\widehat{\operatorname{Var}} \begin{pmatrix}\hat{\mu}\\\hat{\phi}\end{pmatrix} = \begin{pmatrix}1.00&0.0200\\0.0200&0.0064\end{pmatrix}.

The covariance matrix is positive definite: its determinant is $1.00(0.0064)-(0.0200)^2=0.0060>0$. The plug-in forecast is $100+0.50(108-100)=104$. The gradient, in the order $(\mu,\phi)$, is $(1-\phi,\;Y_T-\mu)=(0.50,\;8)$.

The added variance is the quadratic form $0.50^2(1.00)+2(0.50)(8)(0.0200)+8^2(0.0064)$. The three terms are $0.2500$, $0.1600$, and $0.4096$, so the contribution is $0.8196$ squared outcome units. Including the one-step innovation variance gives $3^2+0.8196=9.8196$, with standard error $\sqrt{9.8196}=3.134$ outcome units. A normal 95% approximation is $104\pm1.96(3.134)$, or approximately $[97.86,110.14]$.

The covariance term matters: uncertainty in the estimates of the mean and autoregressive coefficient need not be independent. Omitting the off-diagonal entries would give a different contribution. The calculation also assumes the fitted innovation variance is adequate; it does not separately model finite-sample uncertainty in that variance.

Parametric Bootstrap for Forecast Distributions

The delta method is convenient when the forecast is smooth and its gradient is manageable. A parametric bootstrap offers a simulation-based alternative, especially when the forecast distribution may be asymmetric or when the forecast function is awkward to differentiate. It can reproduce parameter estimation uncertainty by simulating new samples and refitting the model. To include future shocks as well, each refitted model must also generate future observations.

A basic procedure is:

Fit the chosen ARMA model to the observed series and save $\hat{\theta}$.
For each bootstrap replication, simulate a series of the observed sample size from the fitted model, using the fitted initialization or an adequate burn-in.
Refit the same model to that simulated series using the same estimation method and constraints, obtaining $\hat{\theta}^{*(b)}$.
Using that refit, calculate the forecast center and simulate future innovations to generate a future value $Y_{T+h}^{*(b)}$.
Use the simulated forecast errors $Y_{T+h}^{*(b)}-m_h(\hat{\theta}^{*(b)})$, or the simulated future values themselves, to estimate forecast-error quantiles and form an interval.

The simulated series represent repeated samples under the fitted model, so refitting is essential: simply simulating future innovations while holding $\hat{\theta}$ fixed estimates only the plug-in forecast distribution. The bootstrap is conditional on the chosen model and fitted parameter values. It does not automatically account for uncertainty about which ARMA order is correct, structural change, or model misspecification.

Worked Example: Bootstrap Interval for an ARMA(1,1) Forecast

Suppose a hypothetical series is fitted with a stationary, invertible Gaussian ARMA(1,1) model. The model is refitted in each of $B=2{,}000$ parametric bootstrap replications, and a future observation is simulated after each refit. Assume the simulated 2.5th and 97.5th percentiles of the forecast errors are $-5.10$ and $5.20$ outcome units. The fitted plug-in forecast is $20.0$.

The bootstrap interval is formed by adding the error quantiles to the fitted forecast: $[20.0-5.10,\;20.0+5.20]=[14.90,25.20]$. It is slightly asymmetric around the forecast, which can occur when the refitting and forecasting distribution is not well represented by a symmetric normal approximation.

For comparison, suppose the average plug-in future-innovation variance across the replications is $5.50$ squared units, while the variance across the refitted forecast centers is $1.44$ squared units. Their sum is $6.94$, with standard deviation $\sqrt{6.94}=2.634$ units. This variance summary is consistent with a substantial additional parameter contribution, but the percentile interval is obtained from the simulated forecast-error distribution itself, not by assuming that these two variance summaries determine its shape. The stated percentiles are hypothetical bootstrap output; an actual analysis would calculate them from its simulated replications.

Assumptions, Interpretation, and Common Pitfalls

Both approaches require a model-based interpretation. In the delta-method calculation, the estimator’s covariance estimate must be suitable for the fitted ARMA model, the forecast gradient must be evaluated with respect to the same parameterization, and the local linear approximation must be adequate. Near nonstationarity, near a noninvertibility boundary, or with a long horizon and strong sensitivity to persistence, a first-order approximation may be unreliable. Bootstrap results also depend on the fitted model being a useful data-generating approximation and on the fitting procedure being reproduced faithfully.

A normal interval based on the combined variance additionally assumes that a normal approximation to forecast error is reasonable. Gaussian innovations support conditional normal forecast errors when parameters are known, but estimation and refitting can still make the full distribution nonnormal in finite samples. In such settings, bootstrap quantiles can be preferable, provided the bootstrap model is credible and the number of replications is adequate.

Mistake: “The usual forecast variance already includes estimated-parameter uncertainty.” The innovation-weight variance used for a plug-in ARMA interval is calculated with fitted parameters held fixed. Add a forecast-center contribution or use a refitting bootstrap to represent parameter uncertainty.
Mistake: adding the standard error of a parameter directly to the forecast standard error. The relevant first-order contribution is the quadratic form $g_h'\widehat{\operatorname{Var}}(\hat{\theta})g_h$, which accounts for forecast sensitivity, parameter variances, and covariances.
Mistake: interpreting a bootstrap as protection against every source of uncertainty. A parametric bootstrap under one fitted ARMA specification represents sampling and future-shock variation under that model; it does not automatically represent order uncertainty or structural breaks.

Key takeaway: A plug-in ARMA interval accounts for future innovations conditional on fitted parameters. To reflect estimated-parameter uncertainty, approximate the sensitivity of the forecast center using the parameter covariance matrix, or simulate repeated samples, refit the model, and generate future outcomes in a parametric bootstrap.

Check Your Understanding

Answer each question using the distinction between parameter uncertainty and future-innovation uncertainty.

In the first worked example, why does the two-step forecast have a parameter contribution even though the innovation variance is known?
What role does the gradient $g_h(\hat{\theta})$ play in the delta-method variance approximation?
Why can the covariance between two parameter estimates affect forecast uncertainty?
Which step in a parametric bootstrap would be omitted if one wanted to hold the fitted parameters fixed?
Name two limitations of a parametric bootstrap based on a single fitted ARMA model.