Expanding and Rolling Windows for Forecast Validation

Why the Validation Window Matters

A time-series forecast is made using information available at a particular time. Validation therefore requires more than selecting a set of observations to hold out: it requires a rule for which past observations are used whenever a forecast is made. An expanding window retains the full available history, while a fixed-width rolling window repeatedly removes the oldest observations. Those choices can produce different forecasts, especially when the process changes over time.

A validation comparison is informative only when its design matches the forecasting task. If a model will be updated as new observations arrive, validation should represent that process. The same evaluation dates and forecast horizons should generally be used when comparing window strategies or candidate models.

The observations after an origin are not part of that origin’s training set. For example, when forecasting \(y_{t+1}\), the value of \(y_{t+1}\) is unknown at time \(t\); it becomes an evaluation observation after the forecast is recorded. This ordering is essential. A procedure that uses information from after the origin is not a valid simulation of forecasting at that time.

Constructing the Two Window Schemes

Suppose the initial training period ends at \(m\), and validation uses one-step-ahead forecasts through time \(T\). The origins are \(m,m+1,\ldots,T-1\). The expanding training set grows by one observation at each successive origin. A rolling window with fixed width \(w\) keeps the number of training observations constant, dropping the oldest observation when a new one becomes available.

For a general fixed horizon \(h\), the target at origin \(t\) is \(y_{t+h}\). To keep that target within the observed series, origins must satisfy \(t+h\leq T\). The window definition concerns the observations used to fit the forecast procedure; the horizon specifies how far ahead the forecast reaches. These are separate design choices.

How Window Choice Changes Forecasts

An expanding window can be useful when older observations remain relevant and more data improve estimation. It avoids discarding information and may give more stable estimates when the underlying process is reasonably consistent. Its limitation is that a long history can dilute evidence of a recent change. If the level or dynamics have shifted, observations from an earlier regime may pull the fitted model away from current behavior.

A rolling window emphasizes recent observations. It can respond more quickly to changing conditions because older data no longer affect the fit once they fall outside the window. The cost is that fewer observations are available, which can make estimates less stable. A narrow window may also omit useful seasonal patterns or other longer-term structure.

Neither strategy is universally superior. The decision depends on the forecasting setting: whether older data are still informative, how quickly the process may change, how much data the model needs, and how frequently forecasts will be updated. Window width should be selected as part of the validation design, rather than chosen after looking for whichever arrangement gives a preferred result.

A Short Derivation: Updating an Expanding Mean

The expanding-window mean illustrates why each new observation changes an expanding estimate without removing any earlier data. At origin \(t\), let \(M_t\) be the arithmetic mean of \(y_1,\ldots,y_t\). By definition, the sum of those \(t\) values is \(tM_t\). When \(y_{t+1}\) arrives, the new sum is \(tM_t+y_{t+1}\), and there are \(t+1\) observations. Therefore,

This identity shows that the new mean moves from \(M_t\) toward the new observation. The adjustment is the difference \(y_{t+1}-M_t\), divided by \(t+1\), so an individual new value has a smaller effect when the expanding history is long. For a rolling mean of width \(w\), the corresponding update removes the oldest value and adds the newest:

Here \(R_t\) is the mean of \(y_{t-w+1},\ldots,y_t\). The rolling update follows because the new window’s sum equals the old window’s sum minus \(y_{t-w+1}\) plus \(y_{t+1}\), with the number of observations remaining \(w\). Unlike the expanding update, this change depends on both the incoming and departing observations. This calculation explains how window choice affects a simple estimate; fitted models with more parameters can respond differently, but the same inclusion and exclusion rule applies to their training data.

Comparing Forecast Procedures Fairly

To attribute a difference in validation results to window choice, keep the rest of the comparison controlled. In particular, use common forecast origins, common target observations, the same forecast horizon, and a clearly stated refitting rule. At each origin, fit or update the procedure using only the permitted training observations, then produce the forecast. A different initial training length may change both the first evaluation date and the number of forecasts, so those differences should be made explicit.

A rolling window need not mean that the model is estimated only once. The window specifies which observations are eligible for each fit. The model can be refitted at every origin, as in a rolling-origin evaluation, or an implementation may update its estimates as data arrive. Whatever computational method is used, it must produce the same estimates that the stated training set and fitting rule imply.

For a forecast horizon greater than one, the origin-target pairing must remain clear. For example, a three-step forecast at origin \(t\) is judged against \(y_{t+3}\), not against the next observation simply because it is the next value in the data table. If forecasts are made at consecutive origins, their target dates may overlap across horizons. That does not change the window definition, but it matters when interpreting the resulting comparison.

Common Mistakes and Practical Choices

• Calling any repeated refit a rolling window. Refitting at each origin does not by itself make a window rolling. If the training data continue to include the entire history, the scheme is expanding.
• Changing the evaluation dates unnoticed. A rolling scheme may require a larger minimum history than an expanding scheme. If this changes the first origin, it may also change the targets being compared. Use common origins when the goal is a direct comparison, or explain why the periods differ.
• Assuming more recent data must be better. A rolling window can adapt to a real change, but it also discards valid information. Its shorter history may increase estimation variability.
• Assuming an expanding window is always safer. Retaining all history is not automatically beneficial when the older observations describe a different process.
• Using a future value to make an earlier forecast. At origin \(t\), the fitting procedure must not use observations after \(t\). In a multi-step forecast, values between \(t\) and the target are also unavailable unless the forecasting setup explicitly provides them.