OLS Estimation of VAR Systems

Why OLS Works Naturally for a Reduced-Form VAR

A reduced-form vector autoregression models each variable using the same set of lagged variables. This shared regressor structure is important: it means the equations can be estimated separately by ordinary least squares (OLS), even when their innovations are contemporaneously correlated. The resulting coefficient estimates also agree with system generalized least squares (GLS), under the conditions described below.

That equivalence does not make the system’s covariance irrelevant. It determines the uncertainty of the estimated coefficients, records contemporaneous dependence among innovations, and enters joint tests that involve coefficients from more than one equation.

The standard assumptions for the estimation and inference described here are that innovations have conditional mean zero given past information, have a finite positive-definite covariance matrix \(\Sigma\), and are serially uncorrelated. Thus \(E(u_t\mid\mathcal F_{t-1})=0\) and \(E(u_tu_t'\mid\mathcal F_{t-1})=\Sigma\). The regressors must have full column rank for OLS to be uniquely defined. Large-sample inference additionally requires regularity conditions such as a nonsingular limiting regressor second moment and suitable stationarity, ergodicity, and moment conditions. Contemporaneous correlation between components of \(u_t\) is allowed.

Equation-by-Equation OLS

Let \(y_i\) denote column \(i\) of \(Y\), and let \(b_i\) be column \(i\) of \(B\). Equation \(i\) is the regression \(y_i=Xb_i+u_i\). Its OLS estimator minimizes the sum of squared residuals in that equation. Stacking the equations gives the equivalent matrix criterion: choose \(B\) to minimize the sum of squared entries of \(Y-XB\).

This follows by differentiating \(\operatorname{tr}\{(Y-XB)'(Y-XB)\}\) with respect to \(B\). The first-order condition is \(X'(Y-XB)=0\), or \(X'XB=X'Y\). Full column rank makes \(X'X\) invertible, giving the estimator above. Each column of \(\widehat B\) is exactly the OLS estimate from its corresponding equation. The residuals satisfy the normal equations \(X'\widehat U=0\).

Equation-by-equation estimation is straightforward, but it should not be confused with assuming independent equations. The estimates from different equations can be correlated because their innovations are correlated. That dependence is captured by \(\Sigma\) and matters when quantifying uncertainty or testing restrictions jointly.

When OLS Matches System Estimation

In system GLS, the covariance across equations is used to weight residuals. If every equation has the same regressor matrix \(X\), that weighting does not change the coefficient estimates. To see why, suppose \(\Sigma\) is positive definite and consider the GLS objective \(\operatorname{tr}\{\Sigma^{-1}(Y-XB)'(Y-XB)\}\). Its first-order condition is \(X'(Y-XB)\Sigma^{-1}=0\). Since \(\Sigma^{-1}\) is invertible, this is equivalent to \(X'(Y-XB)=0\), the OLS normal equation. Hence system GLS and equation-by-equation OLS give the same \(\widehat B\).

The same result holds when GLS uses a consistent estimate of \(\Sigma\): with common regressors, the weighting still cancels from the coefficient normal equations. Under conditionally Gaussian innovations, the conditional Gaussian likelihood also has its coefficient maximum at \(\widehat B\). One way to verify this is the residual cross-product identity below.

The cross terms vanish because \(X'\widehat U=0\). The second term on the right is positive semidefinite. Therefore, for every candidate \(B\), the residual cross-product matrix is at least as large as \(\widehat U'\widehat U\) in the positive-semidefinite ordering. When the residual covariance is positive definite, this also means its determinant cannot be smaller. The Gaussian likelihood, maximized over the innovation covariance, is consequently maximized at \(\widehat B\).

The shared-regressor condition is the key. In a general system where different equations use different regressors, cross-equation covariance can affect the GLS coefficient estimates. In an unrestricted reduced-form VAR, each equation uses the same lagged-variable vector, which is why equation-by-equation OLS has this special system-estimation equivalence.

Estimating the Innovation Covariance

After estimating the coefficients, collect the fitted innovations in \(\widehat U=Y-X\widehat B\). The residual cross-product \(\widehat U'\widehat U\) summarizes both residual variation within each equation and residual covariation across equations. With \(T\) usable observations, the conditional Gaussian maximum-likelihood covariance estimator divides this matrix by \(T\).

A commonly reported degrees-of-freedom-adjusted version divides by \(T-k\), since \(k\) coefficients were estimated in each equation:

Both estimates are \(m\times m\) matrices. The diagonal entry \(\widehat\Sigma_{ii}\) estimates the innovation variance in equation \(i\); an off-diagonal entry estimates the covariance between two equations’ innovations. The adjustment is a conventional finite-sample correction. It should not be treated as an exact unbiasedness guarantee for every dynamic regression, since lagged regressors are random rather than fixed.

Joint Tests of VAR Coefficients

A joint test asks whether several coefficients satisfy restrictions simultaneously. Write \(b=\operatorname{vec}(B)\), stacking the columns of \(B\) in order. A set of \(q\) linear restrictions can be written \(H_0:Rb=r\), where \(R\) is a specified \(q\times km\) matrix and \(r\) is a specified \(q\times1\) vector. The alternative is that at least one restriction fails.

Under the regularity conditions for large-sample VAR inference, a plug-in estimate of the covariance matrix of \(b\) is \(\widehat V=\widehat\Sigma\otimes(X'X)^{-1}\), using a chosen residual covariance estimate. This expression reflects both ingredients: \((X'X)^{-1}\) describes regressor information, and \(\widehat\Sigma\) includes covariance between equations. The Wald statistic is

If the restrictions are linearly independent and the null is true, \(W\) has an asymptotic chi-square distribution with \(q\) degrees of freedom. The p-value is the upper-tail probability \(P(\chi_q^2\ge W)\). This is a large-sample result; replacing \(W\) by \(W/q\) and comparing with an \(F\) reference distribution is sometimes used as a finite-sample approximation, but should not automatically be described as exact for a VAR with random lagged regressors.

Assumptions, Interpretation, and a Common Pitfall

OLS coefficient estimates describe conditional linear relationships between the modeled variables and their lags. They are not, by themselves, causal effects. For standard large-sample coefficient tests, the innovation mean-zero condition prevents systematic prediction of current innovations from past information; the covariance and serial-dependence assumptions support the stated covariance formula; and the regressor rank condition makes the estimates identifiable. If innovations are serially correlated or their conditional covariance is not constant, the usual covariance and Wald calculations may not be valid without an appropriate alternative method.

A frequent mistake is to infer that correlated innovations require system GLS to obtain different, more efficient VAR coefficients. That reasoning overlooks the common regressor matrix: GLS weights residuals across equations, but its coefficient normal equations reduce to the same equations as OLS. Correlated innovations still matter for the estimated covariance matrix and for cross-equation joint tests. Another pitfall is to call \(\widehat U'\widehat U/T\) the degrees-of-freedom-adjusted estimate; the divisor is \(T\) for the conditional Gaussian maximum-likelihood estimate and \(T-k\) for the conventional adjustment.