Tutorials › Econometrics (Graduate) › GLS Under Covariance Misspecification

Home  /  Tutorials  /  Generalized Least Squares

Generalized Least Squares · Tutorial 157 of 952

GLS Under Covariance Misspecification

Learn when GLS remains consistent under covariance misspecification, how to derive its probability limit, and why incorrect weights can reduce efficiency without changing the target.

Intermediate 11 min read

What You'll Learn

  • Write GLS using a working precision matrix and distinguish it from the true error covariance.
  • Derive the probability limit under conditional mean-zero errors and regularity conditions.
  • Explain why exogeneity, rather than correct covariance weights, protects the coefficient target.
  • Calculate how working weights change a realized estimate and its sampling variance.
  • Compare correct and incorrect weights using a linear unbiased estimator example.
  • Identify conditions under which covariance misspecification can also threaten consistency.

Why Covariance Misspecification Matters

Generalized least squares (GLS) chooses weights to account for the covariance of regression errors. When the working covariance is correct, those weights can improve precision. In practice, however, a covariance model may be only an approximation. The central question is then whether the regression coefficient is still estimated consistently, and what is lost if it is.

The key distinction is between the conditional mean of the errors and their covariance. If errors have conditional mean zero given the regressors, weighting them differently does not, by itself, change the coefficient that the regression targets. But incorrect weights generally fail to exploit the true covariance efficiently. This conclusion relies on the weights being appropriately specified as functions of the regressors, or otherwise satisfying the regularity conditions below; it is not a claim that any data-dependent weighting is harmless.

Definition: Let \(y=X\beta+u\), where \(y\) is an \(n\)-vector of outcomes, \(X\) is an \(n\times k\) regressor matrix of full column rank, \(\beta\) is the \(k\)-vector of regression coefficients, and \(u\) is the error vector. Let \(\Omega\) be the true conditional covariance matrix of \(u\) given \(X\). A GLS estimator using a working covariance matrix \(\Omega_0\) is $$ \hat{\beta}_W=(X'\Omega_0^{-1}X)^{-1}X'\Omega_0^{-1}y. $$ Here \(\Omega_0\) is positive definite and is the covariance model used to construct the weights; it need not equal \(\Omega\). The subscript \(W\) denotes the working GLS estimator.

It is often convenient to write \(W=\Omega_0^{-1}\), the working precision matrix. The true covariance \(\Omega\) describes how the errors actually vary and co-vary; the working precision \(W\) describes how GLS weights observations. Correct weighting means \(W=\Omega^{-1}\), up to an irrelevant common positive scale. Covariance misspecification means the working covariance, and hence its inverse, does not represent the true covariance.

For the consistency result, assume \(E[u\mid X]=0\), \(W\) is determined by \(X\) (or is sufficiently well behaved to have the same asymptotic properties), and the normalized weighted design matrix has a finite positive-definite limit. Also require a law of large numbers for the weighted error term. These conditions rule out singular limiting designs and weights that create uncontrolled influence from a small number of observations.

Probability Limit Under Working Weights

The estimator identity follows by substituting \(y=X\beta+u\) into the GLS formula. It separates the coefficient target from the contribution of the errors:

$$ \hat{\beta}_W =\beta+(X'WX)^{-1}X'Wu. $$

Define \(Q_W=\operatorname{plim} n^{-1}X'WX\), and suppose \(Q_W\) is nonsingular. If \(\operatorname{plim} n^{-1}X'Wu=0\), multiplying the estimator identity by the normalized matrices gives the probability limit directly. Conditional mean-zero errors and weights determined by \(X\), together with an appropriate law of large numbers, are standard sufficient conditions for the weighted error term to converge to zero.

Theorem: Under the stated regularity conditions, $$ \operatorname{plim}\hat{\beta}_W =\beta+Q_W^{-1}\operatorname{plim}(n^{-1}X'Wu) =\beta. $$ Thus GLS remains consistent under covariance misspecification when the conditional mean specification and exogeneity conditions hold. Correct covariance specification is not required for this probability limit.

The result does not say that every GLS estimate equals the true coefficient in a particular sample. The weighted error term is generally nonzero in finite samples, so changing the weights can change the realized estimate. Nor does the result apply if the weights are related to the errors in a way that makes \(n^{-1}X'Wu\) converge to a nonzero value. For example, conditioning only on \(X\) is not enough if a weight is chosen using the outcome or residuals in a way that induces such a relationship.

Worked Example: A Misspecified Weighting Pattern and the Coefficient Target

Worked Example: A Misspecified Weighting Pattern and the Coefficient Target

Suppose the conditional mean is \(E[y_i\mid x_i]=x_i'\beta\), with \(\beta=(2,3)'\), and a hypothetical design repeats three regressor vectors in equal proportions: \(x_1=(1,0)'\), \(x_2=(1,1)'\), and \(x_3=(1,2)'\). The true conditional mean errors are zero. Use working weights \(1,4,1\), which need not be proportional to the inverse of the true error variances.

The three conditional means are \(x_1'\beta=2\), \(x_2'\beta=5\), and \(x_3'\beta=8\). The limiting weighted cross-product is $$ Q_W=\frac{1}{3}\left[ \begin{pmatrix}1&0\\0&0\end{pmatrix} +4\begin{pmatrix}1&1\\1&1\end{pmatrix} +\begin{pmatrix}1&2\\2&4\end{pmatrix} \right] =\begin{pmatrix}2&2\\2&8/3\end{pmatrix}. $$ Its determinant is \(2(8/3)-2(2)=4/3\), so it is nonsingular. The weighted cross-product with the conditional mean is $$ \frac{1}{3}\left[ 1\begin{pmatrix}1\\0\end{pmatrix}2 +4\begin{pmatrix}1\\1\end{pmatrix}5 +1\begin{pmatrix}1\\2\end{pmatrix}8 \right] =\begin{pmatrix}10\\12\end{pmatrix}. $$

The population weighted normal equation therefore has solution \(b\) satisfying $$ Q_W b=\begin{pmatrix}10\\12\end{pmatrix} =Q_W\begin{pmatrix}2\\3\end{pmatrix}. $$ Because \(Q_W\) is nonsingular, its unique solution is \(b=(2,3)'\). The weights alter the relative influence of the three regressor patterns, but under the stated conditional mean they do not alter the population coefficient.

What Misspecification Changes: Sampling Variance and Efficiency

Although the probability limit can remain \(\beta\), the sampling variance depends on both the working weights and the true covariance. Conditional on \(X\), if \(E[u\mid X]=0\) and \(\operatorname{Var}(u\mid X)=\Omega\), direct variance algebra gives

$$ \operatorname{Var}(\hat{\beta}_W\mid X) =(X'WX)^{-1}X'W\Omega WX(X'WX)^{-1}, $$

when \(W\) is symmetric. This expression uses the true \(\Omega\), not the working covariance \(\Omega_0\). In large samples, under corresponding limits, the covariance of \(\sqrt{n}(\hat{\beta}_W-\beta)\) is \(Q_W^{-1}R_WQ_W^{-1}\), where \(R_W=\operatorname{plim} n^{-1}X'W\Omega WX\). A working covariance model generally changes both \(Q_W\) and \(R_W\), even when the probability limit of the coefficient is unchanged.

The efficiency comparison has a clear benchmark. With the correct covariance, GLS uses \(W=\Omega^{-1}\). Under the linear model with conditionally mean-zero errors and known covariance, this is the best linear unbiased estimator in the Gauss–Markov sense: among linear estimators unbiased conditional on \(X\), it has minimum conditional covariance. A different positive-definite working weight can still produce a linear unbiased estimator, but in general it does not attain that minimum. The loss is a precision loss, not automatically a change in the coefficient target.

Worked Example: Correct and Incorrect Weights for a Common Mean

Consider two hypothetical independent observations of a common mean \(\mu\): \(y_i=\mu+u_i\). Suppose \(E[u_i\mid X]=0\) and the true error variances are \(1\) and \(9\), respectively. The working model incorrectly assumes equal variances, so it assigns equal weights. For illustration, take observed values \(y_1=10\) and \(y_2=4\).

With equal weights, GLS is the sample average: $$ \hat{\mu}_W=\frac{y_1+y_2}{2} =\frac{10+4}{2}=7. $$ Since each observation has conditional mean \(\mu\), its expectation is \(E[\hat{\mu}_W\mid X]=(\mu+\mu)/2=\mu\). Thus the wrong weights do not introduce conditional bias in this common-mean model.

Its conditional variance is $$ \operatorname{Var}(\hat{\mu}_W\mid X) =\frac{1}{4}(1)+\frac{1}{4}(9)=2.5. $$ The correct inverse-variance weights are \(1\) and \(1/9\). After normalizing them to sum to one, the estimator is $$ \hat{\mu}_{GLS}=\frac{9}{10}y_1+\frac{1}{10}y_2 =\frac{9}{10}(10)+\frac{1}{10}(4)=9.4. $$ It is also conditionally unbiased, and its variance is $$ \operatorname{Var}(\hat{\mu}_{GLS}\mid X) =\left(\frac{9}{10}\right)^2(1) +\left(\frac{1}{10}\right)^2(9) =0.81+0.09=0.90. $$ The two observed estimates differ because they weight the realized errors differently. Both target \(\mu\), but in this example correct inverse-variance weighting has the smaller variance.

A Short Derivation of the Efficiency Benchmark

For fixed \(X\), consider any linear estimator \(Ay\) that is conditionally unbiased for \(\beta\). Unbiasedness for every \(\beta\) requires \(AX=I_k\). Let \(A_0=(X'\Omega^{-1}X)^{-1}X'\Omega^{-1}\) be the true GLS matrix, and write \(A=A_0+D\). Since \(AX=A_0X=I_k\), \(DX=0\). The covariance difference between the estimator \(Ay\) and true GLS is

$$ A\Omega A'-A_0\Omega A' _0 =D\Omega D' +A_0\Omega D' +D\Omega A_0'. $$

The cross terms vanish. Indeed, \(A_0\Omega=(X'\Omega^{-1}X)^{-1}X'\), so \(A_0\Omega D'=(X'\Omega^{-1}X)^{-1}(DX)'=0\), and its transpose is also zero. Hence

$$ A\Omega A'-A_0\Omega A_0'=D\Omega D'\succeq 0, $$

because \(\Omega\) is positive definite. This establishes the minimum-covariance property among linear conditionally unbiased estimators. A working-weight estimator generally has \(D\ne0\), so it can have larger covariance. This comparison concerns efficiency under the stated linear model; it does not establish that the working estimator is biased.

When Exogeneity Is Not Enough

The phrase “wrong covariance does not affect consistency” needs its conditions. For a working weight matrix \(W\), the exact term to examine is \(n^{-1}X'Wu\). If that term does not converge to zero, the probability limit is instead \(\beta+Q_W^{-1}r_W\), where \(r_W=\operatorname{plim}n^{-1}X'Wu\). In that case, covariance-related weighting can affect the limiting coefficient—not because a covariance model is inaccurate in isolation, but because the weighted errors fail the required orthogonality condition.

This issue can arise if weights are chosen using outcomes or errors, or if the conditional mean is misspecified. It can also arise if the weighted design becomes singular in the limit, in which case the displayed probability-limit formula is unavailable. Therefore, check the weighted exogeneity condition and the limiting design, not merely whether the working covariance resembles the true one.

Common Pitfalls

  • Claiming that incorrect covariance weights automatically cause inconsistency. Under a correctly specified conditional mean, conditional mean-zero errors, and suitable weights, the weighted error term converges to zero and GLS remains consistent.
  • Claiming that consistency means weights do not matter. Weights can change a finite-sample estimate and its variance. In general, incorrect weights sacrifice efficiency even when the probability limit is unchanged.
  • Using the working covariance to describe the estimator’s actual uncertainty. The true conditional covariance \(\Omega\) determines the variance of \(\hat{\beta}_W\); the working covariance only determines the weights used to compute it.
  • Ignoring how weights were chosen. The consistency argument needs the weighted errors to average to zero. Weights that depend on outcomes or errors may violate that condition.
Key takeaway: Under conditional mean-zero errors and regularity conditions, GLS with a misspecified but suitably exogenous working covariance can still be consistent for \(\beta\). What incorrect weighting generally changes is precision: the estimator need not achieve the efficiency of GLS based on the true covariance.

Check Your Understanding

Answer each question by identifying the condition or calculation that supports your conclusion.

  1. In the estimator identity \(\hat{\beta}_W=\beta+(X'WX)^{-1}X'Wu\), which term must converge to zero, after normalization, for consistency?
  2. Why can two working covariance matrices produce different finite-sample estimates while both estimators remain conditionally unbiased?
  3. For the design and weights in the first worked example, verify the determinant of \(Q_W\) and compute \(Q_W(2,3)'\).
  4. In the common-mean example, explain why the equal-weight estimator is unbiased but has a larger variance than the correctly weighted estimator.
  5. Give one way the consistency conclusion could fail even if the weighted design has a nonsingular limit.

Previous

Next

Back to Tutorials
Clinical Biostats