Wald Tests for Nonlinear Parameter Restrictions

Why Nonlinear Restrictions Need a Jacobian

A hypothesis about model parameters need not be linear. It may assert that a product equals a constant, that two parameters have a specified ratio, or that a parameter vector lies on a curved surface. The Wald test handles such hypotheses by evaluating the restriction at the unrestricted estimate and measuring that discrepancy relative to its estimated uncertainty.

For nonlinear restrictions, uncertainty in the parameter estimate does not transfer unchanged to the restriction. The derivative of the restriction describes how small changes in the parameters affect its value. Applying the delta method gives the covariance needed for the test.

Definition: Let $\theta\in\mathbb{R}^p$ be a parameter vector and let $g:\mathbb{R}^p\to\mathbb{R}^q$ be a differentiable function. The null hypothesis of interest is $H_0:g(\theta)=0$, where $0$ is the $q$-vector of zeros. The restriction Jacobian is the $q\times p$ matrix $$ G(\theta)=\frac{\partial g(\theta)}{\partial\theta^\mathsf{T}}. $$ Its $j$th row is the gradient of the $j$th restriction.

Suppose an estimator $\hat\theta$ is consistent and satisfies $\sqrt{n}(\hat\theta-\theta)\xrightarrow{d}N_p(0,V)$, where $V$ is positive definite. Let $\hat V$ consistently estimate $V$, and let $G(\hat\theta)$ be the Jacobian evaluated at the unrestricted estimate. If $g$ is continuously differentiable near the true parameter and, under the null, $G(\theta)$ has full row rank $q$, the Wald statistic is

W_n = n\,g(\hat\theta)^\mathsf{T} \left\{G(\hat\theta)\hat V G(\hat\theta)^\mathsf{T}\right\}^{-1} g(\hat\theta).

The full-row-rank condition ensures that the covariance matrix for the estimated restrictions is invertible. Equivalently, one can write the statistic using an estimate of the covariance of $\hat\theta$ itself. If $\widehat{\operatorname{Cov}}(\hat\theta)$ estimates $V/n$, the same statistic is

W_n = g(\hat\theta)^\mathsf{T} \left\{ G(\hat\theta)\widehat{\operatorname{Cov}}(\hat\theta)G(\hat\theta)^\mathsf{T} \right\}^{-1} g(\hat\theta).

These forms differ only in where the factor $n$ appears. Mixing a covariance estimate for $\hat\theta$ with the first formula would count the sample-size scaling twice.

Derivation and Null Calibration

The key step is to approximate the estimated restrictions by their first-order change around the true parameter. Under the null, the restrictions at the true parameter are zero, so this approximation directly describes the sampling variation of $g(\hat\theta)$.

Delta-method result: If $\sqrt{n}(\hat\theta-\theta)\xrightarrow{d}N_p(0,V)$, $g$ is continuously differentiable near $\theta$, and $G(\theta)$ has full row rank, then $$ \sqrt{n}\{g(\hat\theta)-g(\theta)\} \xrightarrow{d} N_q\left(0,G(\theta)VG(\theta)^\mathsf{T}\right). $$

To see this, differentiability gives the expansion

g(\hat\theta)-g(\theta) = G(\theta)(\hat\theta-\theta)+r_n, \qquad \frac{\|r_n\|}{\|\hat\theta-\theta\|}\xrightarrow{p}0.

Asymptotic normality implies $\sqrt{n}(\hat\theta-\theta)=O_p(1)$, so $\hat\theta-\theta=O_p(n^{-1/2})$. The remainder therefore satisfies $\sqrt{n}r_n=o_p(1)$. Multiplying the expansion by $\sqrt{n}$, applying the assumed asymptotic normality and then Slutsky’s theorem gives the stated limit.

Theorem: Under $H_0:g(\theta)=0$, suppose the delta-method conditions above hold, $\hat\theta\xrightarrow{p}\theta$, and $\hat V\xrightarrow{p}V$. Then $$ W_n\xrightarrow{d}\chi_q^2. $$

Under the null, the delta-method result says that $\sqrt{n}g(\hat\theta)$ is asymptotically normal with covariance $G(\theta)VG(\theta)^\mathsf{T}$. Continuity of $G$, consistency of $\hat\theta$ and $\hat V$, and full row rank imply

G(\hat\theta)\hat V G(\hat\theta)^\mathsf{T} \xrightarrow{p} G(\theta)VG(\theta)^\mathsf{T}.

The limiting covariance is positive definite: for any nonzero $a\in\mathbb{R}^q$, full row rank gives $G(\theta)^\mathsf{T}a\ne 0$, and positive definiteness of $V$ then gives $a^\mathsf{T}G(\theta)VG(\theta)^\mathsf{T}a>0$. Thus the inverse in the statistic converges to the inverse of this covariance. Standardizing the limiting normal vector gives $q$ independent standard normal coordinates; the sum of their squares has a $\chi_q^2$ distribution. Slutsky’s theorem yields the claimed limit. $\square$

Worked Calculations

Worked Example: Testing a Product of Two Normal Means

Let $X_1,\ldots,X_n$ be independent bivariate normal observations with mean $\theta=(a,b)^\mathsf{T}$ and known covariance $V=\begin{pmatrix}4&0.5\\0.5&1\end{pmatrix}$. The estimator is the sample mean vector $\hat\theta=(\bar X_1,\bar X_2)^\mathsf{T}$, which has covariance $V/n$. Test $H_0:ab=1$. For $n=100$, suppose the observed sample means are $(1.2,0.9)$.

Write the restriction as $g(a,b)=ab-1$. Its Jacobian is the row vector $G(a,b)=(b,a)$, so at the estimate $g(\hat\theta)=1.2(0.9)-1=0.08$ and $G(\hat\theta)=(0.9,1.2)$.

The estimated asymptotic variance of $\sqrt{n}g(\hat\theta)$ is, here, exactly

G(\hat\theta)V G(\hat\theta)^\mathsf{T} = (0.9,1.2) \begin{pmatrix}4&0.5\\0.5&1\end{pmatrix} \begin{pmatrix}0.9\\1.2\end{pmatrix} =3.24+1.08+1.44=5.76.

Therefore $W_{100}=100(0.08)^2/5.76=0.64/5.76\approx 0.111$. The test compares this statistic with a $\chi_1^2$ reference distribution. The calculation uses the slope of the product restriction at the estimated parameter, not the covariance of either sample mean alone.

Worked Example: Two Nonlinear Restrictions on Normal Means

Let the observations be independent trivariate normal vectors with mean $\theta=(a,b,c)^\mathsf{T}$ and known covariance $I_3$, the $3\times3$ identity matrix. The sample mean has asymptotic covariance $V=I_3$ for $\sqrt{n}(\hat\theta-\theta)$. Test the two restrictions $ab=1$ and $b/c=1$. Suppose $n=200$ and $\hat\theta=(1.1,0.95,1.0)^\mathsf{T}$.

Define $g(a,b,c)=(ab-1,\ b/c-1)^\mathsf{T}$. At the estimate, $g(\hat\theta)=(0.045,-0.05)^\mathsf{T}$. Differentiating each component gives

G(a,b,c)= \begin{pmatrix} b&a&0\\ 0&1/c&-b/c^2 \end{pmatrix}, \qquad G(\hat\theta)= \begin{pmatrix} 0.95&1.1&0\\ 0&1&-0.95 \end{pmatrix}.

Since $V=I_3$, the restriction covariance estimate is $G(\hat\theta)G(\hat\theta)^\mathsf{T}$:

S= \begin{pmatrix} 2.1125&1.1\\ 1.1&1.9025 \end{pmatrix}, \qquad \det(S)=2.1125(1.9025)-1.1^2=2.80903125.

The quadratic form is

g(\hat\theta)^\mathsf{T}S^{-1}g(\hat\theta) = \frac{ 1.9025(0.045)^2-2(1.1)(0.045)(-0.05)+2.1125(-0.05)^2 }{2.80903125} \approx 0.005014.

Thus $W_{200}\approx200(0.005014)=1.003$, calibrated against $\chi_2^2$. The off-diagonal entries of $S$ matter: the two estimated restrictions are correlated because both depend on $b$.

Worked Example: Testing Equality of Two Poisson Rates

For each of two independent groups, observe $n$ independent Poisson counts with rates $\lambda_1$ and $\lambda_2$. The sample means $\hat\lambda_1$ and $\hat\lambda_2$ estimate the rates, and the asymptotic covariance of $\sqrt{n}(\hat\lambda_1-\lambda_1,\hat\lambda_2-\lambda_2)^\mathsf{T}$ is $\operatorname{diag}(\lambda_1,\lambda_2)$. Test equality of rates using $g(\lambda_1,\lambda_2)=\lambda_1/\lambda_2-1$. For $n=100$, suppose the group totals are 180 and 120, so the sample means are $1.8$ and $1.2$.

The restriction estimate is $g(\hat\theta)=1.8/1.2-1=0.5$, and its Jacobian is $G(\lambda_1,\lambda_2)=(1/\lambda_2,-\lambda_1/\lambda_2^2)$. At the estimates this is $(0.8333\ldots,-1.25)$. Plugging the estimated rates into the asymptotic covariance gives

G(\hat\theta) \begin{pmatrix}1.8&0\\0&1.2\end{pmatrix} G(\hat\theta)^\mathsf{T} = (0.8333\ldots)^2(1.8)+(1.25)^2(1.2) =1.25+1.875=3.125.

The Wald statistic is $W_{100}=100(0.5)^2/3.125=25/3.125=8$, with a $\chi_1^2$ reference distribution. The null is equality of the rates, but the calculation evaluates the ratio restriction and its derivative at the unrestricted estimates.

Practical Checks and a Common Pitfall

A nonlinear Wald calculation is easiest to audit in a fixed order: write the restrictions as a vector, differentiate them, evaluate both the restrictions and Jacobian at the unrestricted estimate, form the restriction covariance, and then compute the quadratic form. For multiple restrictions, the covariance matrix must retain cross-covariances; treating each restriction as independent generally changes the statistic.

Pitfall: substituting the null value before calculating the discrepancy. For the product restriction $ab=1$, setting $b=1/a$ inside $g$ makes $g$ identically zero. That evaluates the restriction on the null surface rather than at the unrestricted estimate, eliminating the discrepancy the test is supposed to measure. Evaluate $g(\hat\theta)$ at the unrestricted estimate.
Pitfall: using the restriction values as their own standard errors. A nonlinear transformation changes variance according to its derivatives. Use $G(\hat\theta)\hat V G(\hat\theta)^\mathsf{T}$, not just the parameter covariance or a separate variance for each parameter.
Pitfall: omitting the sample-size factor. The formula with $\hat V$ for the asymptotic covariance of $\sqrt n\hat\theta$ includes $n$. If using an estimate of $\operatorname{Cov}(\hat\theta)$, use the equivalent formula without that extra factor.

The Wald statistic is useful because it requires an unrestricted estimate, its covariance estimate, and the derivative of the restrictions. Its standard chi-square calibration is a large-sample result under the stated regularity conditions; the calculation alone does not guarantee that those conditions are adequate in a particular sample.

Check Your Understanding

Use the restriction vector, its Jacobian, and the appropriate covariance convention to answer these questions.

For $g(a,b)=a^2+b-4$, write the Jacobian and evaluate it at $(a,b)=(2,1)$.
In the Wald statistic, what covariance does $\hat V$ estimate, and where does the factor $n$ appear?
Why does a vector of restrictions generally require the off-diagonal entries of its estimated covariance matrix?
State the null hypothesis and Jacobian for testing whether $\lambda_1/\lambda_2=2$.
Under what local condition on the restriction Jacobian is the covariance of the estimated restrictions invertible, given positive definite $V$?

Wald Tests for Nonlinear Parameter Restrictions

What You'll Learn