Why Error Variance Matters
OLS coefficients describe how the fitted outcome changes with the regressors, but inference also depends on how those coefficients vary across repeated samples. That sampling variation depends on the conditional variance of the regression errors. If this variance changes with the regressors, OLS can still estimate the mean relationship without bias under exogeneity, while its conventional standard errors use the wrong variance formula.
The distinction is important: a coefficient estimate, its sampling variance, and an estimated standard error are related but not interchangeable. Conditional heteroskedasticity directly changes the sampling variance. It does not, by itself, imply that the conditional mean is misspecified or that OLS coefficients are biased.
Model and Conditional Heteroskedasticity
Consider a linear regression with \(n\) observations and \(k\) regressors, including any intercept. Write the model as \(y=X\beta+u\), where \(y\) is the \(n\)-vector of outcomes, \(X\) is the \(n\)-by-\(k\) regressor matrix, \(\beta\) is the coefficient vector, and \(u\) is the error vector. Assume \(X\) has full column rank, so \(X'X\) is invertible. The OLS estimator is \(\hat{\beta}=(X'X)^{-1}X'y\).
For the finite-sample results below, condition on \(X\), assume \(E(u\mid X)=0\), and assume errors from different observations are uncorrelated conditional on \(X\). Let the conditional error variance for observation \(i\) be \(\sigma_i^2=\operatorname{Var}(u_i\mid X)\). The variances may depend on the regressors and need not be equal.
Under the stated lack of conditional correlation, the conditional covariance matrix of \(u\) is \(\Omega=\operatorname{diag}(\sigma_1^2,\ldots,\sigma_n^2)\). Conditional heteroskedasticity means that this matrix is not generally a scalar multiple of the identity matrix. If errors are conditionally correlated across observations, off-diagonal entries are also needed; the formulas here focus on heteroskedasticity with conditional uncorrelatedness.
How Heteroskedasticity Changes OLS Sampling Variance
Substitute \(y=X\beta+u\) into the OLS estimator. This gives \(\hat{\beta}=\beta+(X'X)^{-1}X'u\). Conditional on \(X\), the regressors are fixed, so the zero conditional mean assumption implies \(E(\hat{\beta}\mid X)=\beta\). The conditional covariance follows by applying the covariance rule for a linear transformation:
This is the finite-sample conditional sampling variance under the model just stated. It depends on the individual variances through \(\Omega\). By contrast, if the errors are conditionally homoskedastic, then \(\Omega=\sigma^2 I\), and the expression simplifies to \(\sigma^2(X'X)^{-1}\). The familiar conventional covariance formula relies on this simplification.
In practice, the conventional estimated covariance matrix is \(s^2(X'X)^{-1}\), where \(s^2=\mathrm{RSS}/(n-k)\) and \(\mathrm{RSS}\) is the residual sum of squares. It estimates the homoskedastic expression by replacing the common error variance with a residual-based estimate. When conditional variances differ, there is no single common \(\sigma^2\) for that formula to estimate. The resulting conventional standard errors are not generally valid for inference under heteroskedasticity, although a particular standard error can coincide by chance with an appropriate one.
Worked Examples
Worked Example: Comparing OLS Precision With Unequal Error Variances
Suppose a hypothetical regression through the origin has two fixed regressor values, \(x_1=1\) and \(x_2=2\). Assume \(y_i=x_i\beta+u_i\), \(E(u_i\mid X)=0\), the errors are conditionally uncorrelated, and their conditional variances are \(\sigma_1^2=1\) and \(\sigma_2^2=9\). The conditions for the variance calculation hold, but the errors are heteroskedastic.
The OLS slope estimate is \(\hat{\beta}=(x_1y_1+x_2y_2)/(x_1^2+x_2^2)\). Its conditional variance is
For comparison, using the average of the two error variances, \((1+9)/2=5\), as though it were a common variance would give the homoskedastic expression \(5/(1^2+2^2)=1.000\). This is not a calculation of the actual conventional residual-based standard error for a particular sample; it illustrates that even a natural common-variance substitute need not reproduce the true sampling variance.
If the two variances were known, a weighted least-squares estimate would be \(\tilde{\beta}=(x_1y_1/\sigma_1^2+x_2y_2/\sigma_2^2)/(x_1^2/\sigma_1^2+x_2^2/\sigma_2^2)\). Its variance is the reciprocal of the denominator:
In this example, the lower-variance observation receives more weight relative to its regressor value, and the resulting linear unbiased estimate is more precise than OLS. This illustrates why the homoskedastic Gauss–Markov efficiency result cannot generally be carried over unchanged when the conditional variances differ. It does not imply that OLS is always inefficient by a particular amount, or that the variances are known in applied work.
Worked Example: Heteroskedasticity Without Coefficient Bias
Consider a hypothetical regression through the origin with fixed regressors \(x_1=1\) and \(x_2=3\), and true coefficient \(\beta=2\). Suppose \(E(u_i\mid X)=0\), the errors are conditionally uncorrelated, and their conditional variances are \(4\) and \(25\). These assumptions allow the error variability to differ while keeping the conditional mean zero.
The estimate is \(\hat{\beta}=(x_1y_1+x_2y_2)/(x_1^2+x_2^2)\). Since \(y_i=x_i\beta+u_i\),
The variance, however, is not the homoskedastic formula:
For one possible realization, let the errors be \(u_1=2\) and \(u_2=-5\). Then \(y_1=4\), \(y_2=1\), and \(\hat{\beta}=(1(4)+3(1))/10=0.7000\). This realized estimate is far from 2, but a single deviation is not evidence of bias. Under the stipulated zero conditional means, averaging over repeated outcomes at the same \(X\) gives an expected estimate of 2. Unequal conditional variances affect how spread out those estimates are, not their conditional expectation.
Worked Example: Conventional Variance in a Two-Group Regression
Suppose a hypothetical regression contains an intercept and a binary regressor \(D\): \(y_i=\beta_0+\beta_1D_i+u_i\). There are three observations with \(D=0\) and two with \(D=1\). Assume zero conditional error means and uncorrelated errors, with conditional variance \(1\) in the \(D=0\) group and \(9\) in the \(D=1\) group. These assumptions permit heteroskedasticity across the groups.
The OLS slope is the difference between the two group means. Because the errors are uncorrelated, its conditional variance is the sum of the group-mean error variances:
The conventional covariance formula uses \(s^2\) multiplied by the slope entry of \((X'X)^{-1}\). For this regression, that entry is \(1/3+1/2=5/6\). To see how the conventional variance estimate behaves on average, note that the residual sum of squares within each group is the sum of squared deviations from that group’s sample mean. Its expectation is \((n_g-1)\sigma_g^2\) for group \(g\). Therefore,
Consequently, the expected conventional estimated variance for the slope is
This differs from the true variance \(29/6=4.833\), so in this example the conventional variance estimate is low on average. The calculation shows how a pooled residual variance can fail to reflect which group’s errors contribute to the uncertainty in the group difference. Other regressor configurations and variance patterns can produce different discrepancies.
Why the Distinction Matters
A standard error is used to quantify coefficient uncertainty and underlies many confidence intervals and tests. If the covariance formula is wrong, the resulting uncertainty measure can be too small or too large. In turn, intervals may have coverage different from their intended level, and tests may reject too often or too rarely. These consequences concern inference; they do not follow from a change in the OLS coefficient definition.
Conditional heteroskedasticity also removes the usual homoskedastic Gauss–Markov guarantee that OLS is the best linear unbiased estimator. OLS remains conditionally unbiased under the assumptions above, but it need not have the smallest variance among linear unbiased estimators. When the variance pattern is known, estimators that account for it can be more precise, as the first example illustrates. In many applications, the variance pattern is unknown, so an inference procedure must avoid relying on a common error variance.
Common Mistake
The assumptions should be checked separately in any argument: the zero conditional mean supports the conditional expectation result; the conditional covariance structure determines the sampling variance; and the conventional formula additionally requires conditional homoskedasticity. A failure of one assumption does not automatically establish a failure of the others.
Check Your Understanding
Use the definitions and formulas in this tutorial to answer the following questions.
- In a model with \(E(u\mid X)=0\), what does conditional heteroskedasticity say about the errors, and what does it not say about their conditional mean?
- State the conditional covariance formula for \(\hat{\beta}\) when errors are uncorrelated across observations but their variances may differ.
- Why does \(\operatorname{Var}(\hat{\beta}\mid X)=\sigma^2(X'X)^{-1}\) follow under conditional homoskedasticity?
- Can OLS be conditionally unbiased when its conventional standard errors are invalid? Explain the role of the two assumptions involved.
- In the two-group example, why is the true slope variance \(1/3+9/2\), rather than a common variance multiplied by \(5/6\)?