Why Parameter Growth Matters
A VAR can represent dynamic relationships among several variables, but each added variable and lag creates more coefficients to estimate. In a short sample, a model may therefore fit the observed data closely while estimating its coefficients imprecisely. That imprecision matters twice: it makes estimated dynamics less reliable, and it can make forecasts worse even when the model’s in-sample residuals are small.
The central issue is not that a large VAR is automatically unsuitable. It is that the information available in the sample must support the number of estimated parameters. This tutorial examines parameter growth, finite-sample bias and variance, forecast uncertainty, and three ways to control an overfitted specification.
Parameter Counts and the Sample
Consider a \(K\)-variable VAR with \(p\) lags, an intercept, and no other deterministic terms:
Here \(y_t\) is a \(K \times 1\) vector, \(c\) is a \(K \times 1\) intercept vector, each \(A_\ell\) is a \(K \times K\) coefficient matrix, and \(u_t\) is the vector of innovations. Each equation has \(Kp\) lag coefficients and one intercept, so it estimates \(q=1+Kp\) coefficients. The VAR estimates \(Kq=K(1+Kp)\) coefficients in total. Adding \(r\) deterministic regressors to each equation changes these counts to \(q=1+Kp+r\) and \(Kq\).
If there are \(T\) observations and the model uses \(p\) lags, the usable number of regression observations is \(N=T-p\). In an unrestricted equation-by-equation least-squares fit, the residual degrees of freedom are \(N-q\), provided the design matrix has full column rank. This count is a useful warning signal, not a complete measure of whether estimation will be reliable. The conditioning of the regressors, the strength of the dynamic relationships, and the location of the system’s roots also matter.
Worked Example: Counting Parameters in a Short-Sample VAR
Suppose a hypothetical quarterly sample contains \(T=76\) observations on \(K=8\) variables. An unrestricted VAR uses \(p=4\) lags and an intercept. The usable sample is \(N=76-4=72\).
Each equation estimates \(q=1+Kp=1+(8)(4)=33\) coefficients: one intercept and \(8 \times 4=32\) lag coefficients. The system therefore estimates \(Kq=(8)(33)=264\) coefficients across its eight equations. Each equation has \(72-33=39\) residual degrees of freedom if its regressors have full column rank.
The total of 264 does not mean that each equation needs 264 observations: the equations are estimated with 33 regressors apiece. But 39 residual degrees of freedom per equation leaves limited room for precise estimation, particularly if the lagged variables are highly correlated. The count flags a potential small-sample problem; it does not by itself prove that this particular VAR is overfit.
Bias, Variance, and Forecast Error
In a finite sample, VAR coefficient estimates can be biased as well as variable. Lagged dependent variables are regressors, and their finite-sample relationships with the innovations can make the usual least-squares estimates biased. The size and direction of bias depend on the data-generating process, sample length, persistence, and specification. In persistent systems, estimated own-lag effects may, for example, be pulled downward in some settings; there is no universal direction for all VAR coefficients. Bias typically becomes more consequential when the sample is short relative to the dynamic complexity.
Uncertainty also grows when there are many coefficients to estimate. For a regression equation with design matrix \(X\), under the usual homoskedastic, serially uncorrelated error conditions and a full-rank \(X\), the conditional covariance matrix of the least-squares coefficient estimate is proportional to \((X'X)^{-1}\). Adding regressors can reduce the information available to estimate individual coefficients, and collinearity among lagged variables can make \((X'X)^{-1}\) large in some directions. The resulting estimates may be sensitive to small changes in the sample.
For a one-step-ahead forecast, separate the future innovation from uncertainty about the estimated coefficients. Let \(x_T\) denote the regressor vector for one equation at forecast origin \(T\), including the intercept, and let \(\hat{\beta}\) be its estimated coefficient vector. If the innovation has variance \(\sigma^2\) and is independent of the estimation error, the forecast-error variance has the form
The first term is uncertainty from the new innovation; the second is uncertainty from estimating the coefficients. This is a one-equation, one-step expression under the stated conditions. A multi-step VAR forecast also propagates uncertainty through successive estimated dynamics and future innovations, so its full forecast uncertainty requires additional terms.
Worked Example: Separating Innovation and Parameter Uncertainty
Suppose a hypothetical one-step forecast for a variable measured in percentage points has an estimated innovation variance of \(0.64\) percentage-points-squared. For the forecast regressor vector, suppose \(x_T'\widehat{\operatorname{Var}}(\hat{\beta})x_T=0.18\) percentage-points-squared. Treat the future innovation and coefficient estimation error as uncorrelated for this calculation.
The estimated forecast-error variance is
The corresponding forecast-error standard deviation is \(\sqrt{0.82}=0.9055\) percentage points. The innovation alone would have standard deviation \(\sqrt{0.64}=0.8000\) percentage points, so ignoring parameter uncertainty would understate the estimated one-step uncertainty in this example. This calculation does not show that parameter uncertainty is always large; it shows why it should not automatically be omitted.
Recognizing Overfitting
An unrestricted VAR usually has at least as small an in-sample residual sum of squares as a nested version estimated on the same data, because the larger model can reproduce the restricted fit while also adjusting its additional coefficients. That mechanical in-sample improvement is not evidence that the extra coefficients capture stable patterns. If they instead fit sample-specific noise, the larger model can have less accurate forecasts on observations not used for estimation.
Several signs deserve attention together: large coefficient standard errors, estimates that change substantially across nearby sample periods, implausible or unstable dynamics, and forecast performance that fails to improve when assessed out of sample. None is conclusive alone. For example, imprecise coefficients may be expected for a weakly identified pattern without necessarily making every forecast unusable. Likewise, a good fit does not establish that the model’s dynamic specification is appropriate.
Worked Example: An In-Sample Gain That Does Not Settle the Forecast Choice
Suppose two hypothetical models are estimated using the same training sample for one equation. The smaller VAR has a residual sum of squares of \(52\); a larger VAR, which adds several lag coefficients, has a residual sum of squares of \(46\). The larger model has therefore reduced the in-sample residual sum of squares by \(52-46=6\), or \(6/52=0.1154\), about 11.54%.
That reduction alone does not establish a forecast improvement: extra coefficients can only help the unrestricted in-sample fit, but may add estimation noise. Suppose a held-out forecast comparison, evaluated with mean squared forecast error in squared units of the outcome, gives \(1.20\) for the smaller model and \(1.35\) for the larger one. The larger model’s held-out error is \(1.35-1.20=0.15\) higher, or \(0.15/1.20=0.1250\), 12.50% higher than the smaller model’s.
For this hypothetical evaluation period, the smaller VAR forecasts better by the stated measure, despite the larger VAR’s lower training residual sum of squares. The result supports preferring the smaller model for that forecasting task and evaluation design; it does not prove that it will forecast better in every future period.
Remedies: Reduce, Restrict, or Shrink
Three common approaches control the effective complexity of a VAR. They make different assumptions about which dynamic patterns should be retained, so none is automatically best.
| Approach | What changes | Main benefit | Main risk |
|---|---|---|---|
| Lag reduction | Estimate fewer lag matrices | Fewer coefficients and less estimation noise | Omitted dynamics can create misspecification bias |
| Restrictions | Set selected coefficients to zero or impose other specified relationships | Uses a smaller parameter set when restrictions are appropriate | Incorrect restrictions can distort estimates and forecasts |
| Shrinkage | Penalize or otherwise pull estimates toward specified target values | Can reduce estimation variance without discarding all included terms | Introduces bias, and results depend on the target and tuning strength |
Lag reduction is most defensible when the shorter lag structure is supported by the modeling purpose and evidence, rather than chosen solely to force a smaller count. Restrictions can express substantive knowledge or a carefully justified structure. A restriction that is false, however, may remove a real dynamic relationship. Shrinkage methods, including ridge-type penalties, trade some bias for reduced variance. Their performance depends on how strongly coefficients are shrunk, which coefficients are penalized, and how well the chosen target reflects the process.
Worked Example: How Shrinkage Trades Bias for Variance
Consider a hypothetical single coefficient whose true value in a repeated-sampling illustration is \(0.50\). Suppose its unshrunk estimate is unbiased, with variance \(0.16\). A shrinkage rule multiplies the estimate by \(0.70\), pulling it toward zero. This illustration isolates one coefficient; an actual VAR penalty would need a specified rule for selecting and applying its penalty.
The shrunk estimator has expected value \(0.70(0.50)=0.35\), so its bias is \(0.35-0.50=-0.15\). Its variance is \(0.70^2(0.16)=0.0784\). The unshrunk estimator’s mean squared error for this coefficient is its variance, \(0.16\), because it is unbiased. The shrunk estimator’s mean squared error is
In this hypothetical repeated-sampling calculation, shrinkage lowers the coefficient’s mean squared error from \(0.16\) to \(0.1009\), despite introducing bias. A different true value, variance, or shrinkage factor could reverse the comparison. Shrinkage is therefore a bias–variance trade-off, not a guarantee of improvement.
Common Pitfalls
- Calling every large VAR overfit. A high parameter count signals pressure on the sample, but the count must be considered alongside usable observations, regressor information, stability, and the intended task.
- Treating a smaller residual sum of squares as a forecast victory. An unrestricted model can improve its in-sample fit mechanically. Forecast claims require an appropriate out-of-sample comparison.
- Assuming restrictions or shrinkage only help. Restrictions can omit real dynamics, while shrinkage can pull important coefficients too far toward their targets. Each may exchange lower variance for greater bias.
- Ignoring uncertainty in estimated coefficients. Forecast errors include more than future innovations. When coefficient uncertainty is material, omitting it can make uncertainty appear too small.
The practical goal is not to minimize the number of VAR coefficients at any cost. It is to retain enough dynamic structure for the question while avoiding complexity that the sample cannot estimate reliably. Lag selection, restrictions, or shrinkage should be judged against that goal, with forecast evaluation used when forecasting is the purpose.
Check Your Understanding
Answer each question using the definitions and distinctions in this tutorial.
- For an intercept-only-in-the-deterministic-terms VAR with \(K=4\) variables and \(p=3\) lags, how many coefficients are estimated per equation and across the system?
- Why does a lower in-sample residual sum of squares for an unrestricted VAR not establish that it forecasts better?
- In a one-step forecast, what do the innovation-variance term and parameter-uncertainty term represent?
- What is the main trade-off when imposing a restriction that sets a coefficient to zero?
- In what sense can shrinkage improve estimation even though it introduces bias?