Why the Wald Statistic Is Asymptotically Chi-Square

Why a Wald Statistic Has a Chi-Square Limit

A Wald test measures how far the estimated parameter is from satisfying a null restriction, relative to the estimator’s uncertainty. Its large-sample calibration comes from two ingredients: the maximum likelihood estimator is approximately normal, and a suitable quadratic form of a normal vector has a chi-square distribution. The key is to apply the normal approximation to the restriction itself, not just to the full parameter vector.

We derive the result for \(q\) smooth restrictions on a \(p\)-dimensional parameter. The argument explains why the statistic has \(q\) degrees of freedom, why the covariance matrix must be estimated consistently, and which conditions are needed for the approximation. This is a null-distribution result: it does not assert that the statistic has an exact chi-square distribution at a finite sample size.

Setup and Conditions

Let \(\theta\) be a parameter in an open subset of \(\mathbb{R}^p\), and let \(\theta_0\) be its true value under the null hypothesis. Write the null as \(g(\theta)=0\), where \(g\) maps a neighborhood of \(\theta_0\) into \(\mathbb{R}^q\). Let \(G(\theta)\) be the \(q\)-by-\(p\) Jacobian matrix of \(g\). The rank condition below means that the restrictions are locally distinct rather than redundant.

For example, if \(g(\theta)=A\theta-b\) is linear, then \(G(\theta)=A\), and the restriction covariance is estimated by \(A\hat{\Sigma}_n A^{\mathsf T}/n\). In a regular likelihood problem, the required ingredients are typically the consistency and asymptotic normality of the maximum likelihood estimator, together with a consistent estimate of its asymptotic covariance. We take these as assumptions here, rather than deriving them from likelihood regularity conditions.

From the Estimator to the Restriction

The first step is to find the large-sample distribution of \(g(\hat{\theta})\). A first-order Taylor expansion around the null value gives

The remainder statement follows from differentiability at \(\theta_0\) and consistency of \(\hat{\theta}\); if \(\hat{\theta}=\theta_0\), the remainder can be taken as zero. Since \(\sqrt n(\hat{\theta}-\theta_0)\) is bounded in probability by its convergence in distribution, multiplying the remainder by \(\sqrt n\) gives a term converging to zero in probability. Since \(g(\theta_0)=0\),

The covariance matrix \(\Omega_0\) is positive definite. For any nonzero \(a\in\mathbb{R}^q\), the full row rank of \(G_0\) implies \(G_0^{\mathsf T}a\ne 0\). Therefore,

because \(\Sigma_0\) is positive definite.

Why the Quadratic Form Is Chi-Square

Let \(\Omega_0^{-1/2}\) denote the symmetric inverse square root of \(\Omega_0\). If \(Y\sim N_q(0,\Omega_0)\), then \(U=\Omega_0^{-1/2}Y\sim N_q(0,I_q)\). The components of \(U\) are independent standard normal random variables. By the definition of the chi-square distribution, their squared sum has a \(\chi^2_q\) distribution:

This also verifies the degrees of freedom: the quadratic form is built from \(q\) independent standardized components, one for each independent restriction.

Proof of the Wald Chi-Square Limit

We now combine the restriction’s asymptotic normality with the normal quadratic-form result. By continuity of \(G\), consistency of \(\hat{\theta}\), and consistency of \(\hat{\Sigma}_n\),

Since \(\Omega_0\) is positive definite, matrix inversion is continuous at \(\Omega_0\), so the inverses converge in probability as well. Rewriting the statistic on the \(\sqrt n\) scale gives

The first and last vectors converge in distribution to \(N_q(0,\Omega_0)\), and the middle matrix converges in probability to \(\Omega_0^{-1}\). Slutsky’s theorem and continuity of the quadratic-form map therefore give

This proves the stated limit. The estimated covariance need not be independent of \(\hat{\theta}\): the probability and distribution limits used here do not require that independence. \(\square\)

Worked Examples

Interpretation and a Common Pitfall

The chi-square reference distribution makes it possible to assess how surprising the observed distance from the null restriction would be under the null, after accounting for estimation uncertainty. It is a large-sample calibration under the stated assumptions, not a guarantee for every sample size or every model. In particular, the argument depends on a regular asymptotic normal approximation, a consistent covariance estimate, and restrictions whose Jacobian has full row rank at the null value.

Common mistake: forgetting the sample-size scaling. The asymptotic covariance \(\Sigma_0\) describes \(\sqrt n(\hat{\theta}-\theta_0)\). The covariance of \(\hat{\theta}\) is approximately \(\Sigma_0/n\), and the covariance of \(g(\hat{\theta})\) is approximately \(G_0\Sigma_0G_0^{\mathsf T}/n\). Using \(G_0\Sigma_0G_0^{\mathsf T}\) directly as the covariance of \(g(\hat{\theta})\) omits the factor \(1/n\). The resulting statistic is too small by a factor of \(n\). A related error is to use the full parameter covariance without transforming it through the Jacobian of the restriction.

The central logic is short but consequential: the estimator’s asymptotic normality yields a normal approximation for the restrictions; consistent covariance estimation standardizes that approximation; and its squared length converges to a chi-square variable with one degree of freedom per independent restriction.