The Scalar Wald Test

Why Use a Scalar Wald Test?

A scalar Wald test asks whether a single unknown parameter is consistent with a specified value, using how far its estimate lies from that value relative to its estimated sampling uncertainty. For example, a treatment effect estimate may be positive, but the relevant question is whether it is large compared with its standard error. The Wald statistic makes that comparison explicit.

The test is useful when an estimator and its standard error are available, even if fitting the model under the null hypothesis would be inconvenient. Its calibration is large-sample: under suitable conditions, the standardized estimation error is approximately standard normal when the null is true. This tutorial develops the resulting two-sided and one-sided tests and the conditions supporting them.

Definition and Rejection Rules

Definition: Let $\theta$ be a scalar parameter and let $\hat{\theta}$ estimate it from a sample of size $n$. To test $H_0:\theta=\theta_0$, let $\widehat{\operatorname{se}}(\hat{\theta})$ be an estimated standard error. The scalar Wald statistic is $$ Z_W=\frac{\hat{\theta}-\theta_0}{\widehat{\operatorname{se}}(\hat{\theta})}. $$ Under regularity conditions and the null hypothesis, $Z_W$ is asymptotically standard normal.

Write $\Phi$ for the standard normal cumulative distribution function and $z_q=\Phi^{-1}(q)$ for its $q$-quantile. At significance level $\alpha$, the two-sided alternative $H_1:\theta\ne\theta_0$ rejects when

|Z_W|>z_{1-\alpha/2}.

For a prespecified upper-sided alternative $H_1:\theta>\theta_0$, reject when $Z_W>z_{1-\alpha}$. For a lower-sided alternative $H_1:\theta<\theta_0$, reject when $Z_W<-z_{1-\alpha}$. These rules are different because the alternative determines where evidence against the null lies. The direction must be chosen before examining the observed statistic.

The corresponding approximate p-values are $2\{1-\Phi(|Z_W|)\}$ for a two-sided test, $1-\Phi(Z_W)$ for an upper-sided test, and $\Phi(Z_W)$ for a lower-sided test. A p-value measures extremeness under the null calibration; it is not the probability that the null hypothesis is true.

Why the Normal Calibration Works

The standard large-sample setup has two parts. First, under $H_0$, the estimator has an asymptotically normal distribution centered at $\theta_0$. Second, the estimated standard error consistently estimates the corresponding scale. More explicitly, suppose that under $\theta=\theta_0$,

\sqrt{n}(\hat{\theta}-\theta_0)\xrightarrow{d}N(0,V_0), \qquad \hat V\xrightarrow{p}V_0, \qquad 0<V_0<\infty,

where $\widehat{\operatorname{se}}(\hat{\theta})=\sqrt{\hat V/n}$. Here $\hat V$ estimates the asymptotic variance of $\sqrt n(\hat{\theta}-\theta_0)$, not the variance of $\hat{\theta}$ itself. The estimated standard error divides its square root by $\sqrt n$.

Theorem: Under the three conditions above, $Z_W\xrightarrow{d}N(0,1)$ under $H_0$. Consequently, the two-sided and one-sided Wald rules have asymptotic rejection probability $\alpha$ when their stated normal cutoffs are used.

Proof. Rewrite the statistic by multiplying numerator and denominator by $\sqrt n$:

Z_W = \frac{\sqrt n(\hat{\theta}-\theta_0)}{\sqrt{\hat V}}.

The numerator converges in distribution to $N(0,V_0)$. Since $\hat V\xrightarrow{p}V_0$ and $V_0>0$, the continuous mapping theorem gives $\sqrt{\hat V}\xrightarrow{p}\sqrt{V_0}$. Slutsky’s theorem then gives

Z_W\xrightarrow{d}\frac{N(0,V_0)}{\sqrt{V_0}}=N(0,1).

The standard normal distribution is continuous at each finite cutoff. Therefore, for the two-sided rule, the limiting rejection probability is $P(|N(0,1)|>z_{1-\alpha/2})=\alpha$. For the upper-sided rule it is $P(N(0,1)>z_{1-\alpha})=\alpha$, and for the lower-sided rule it is $P(N(0,1)<-z_{1-\alpha})=\alpha$. This proves the stated calibration. $\square$

The result is asymptotic, not an assertion of exact finite-sample size. In a particular model the normal approximation may be poor, especially with a small sample or when the parameter is near a boundary. Consistency of the estimated standard error is part of the justification; merely calculating a number called a standard error is not enough.

Worked Tests

Worked Example: Testing a Normal Mean

Suppose $X_1,\ldots,X_{36}$ are independent observations from $N(\theta,9)$, with known variance $9$. The observed sample mean is $\bar X=52.1$. Test $H_0:\theta=50$ against $H_1:\theta\ne50$ at level $0.05$.

The estimate is $\hat{\theta}=\bar X$, and its standard error is $\sqrt{9/36}=3/6=0.5$. Thus

Z_W=\frac{52.1-50}{0.5}=\frac{2.1}{0.5}=4.2.

The two-sided $0.05$ cutoff is $z_{0.975}\approx1.96$. Since $|4.2|>1.96$, reject $H_0$. The approximate p-value is $2\{1-\Phi(4.2)\}\approx0.000027$. The estimate is more than four estimated standard errors above the null value, so the data provide strong evidence against $\theta=50$.

Worked Example: The Direction Changes the Decision

Let $X_1,\ldots,X_{200}$ be independent Bernoulli observations with success probability $p$. There are 112 successes, so $\hat p=112/200=0.56$. Test $H_0:p=0.50$ using the plug-in standard error, and compare the two-sided alternative with the prespecified upper-sided alternative.

The estimated standard error is

\widehat{\operatorname{se}}(\hat p) =\sqrt{\frac{\hat p(1-\hat p)}{n}} =\sqrt{\frac{0.56(0.44)}{200}} =\sqrt{0.001232} \approx0.03510.

The statistic is $Z_W=(0.56-0.50)/0.03510\approx1.71$. For the two-sided test at level $0.05$, the cutoff is approximately $1.96$, so the test does not reject. Its approximate p-value is $2\{1-\Phi(1.71)\}\approx0.087$.

For the upper-sided test $H_1:p>0.50$, the cutoff is $z_{0.95}\approx1.645$. Since $1.71>1.645$, this test rejects at level $0.05$; its p-value is $1-\Phi(1.71)\approx0.044$. The different decisions are not contradictory: the one-sided test allocates all of its rejection probability to the direction specified in its alternative. That direction should not be selected after seeing that the estimate is above $0.50$.

Worked Example: An Exponential Rate Estimate

Suppose $X_1,\ldots,X_{100}$ are independent exponential observations with rate parameter $\theta>0$, so $E[X_i]=1/\theta$ and $\operatorname{Var}(X_i)=1/\theta^2$. The maximum likelihood estimator is $\hat{\theta}=1/\bar X$. If $\bar X=0.80$, test $H_0:\theta=1$ against $H_1:\theta\ne1$ at level $0.05$.

To obtain the large-sample standard error, the central limit theorem gives $\sqrt n(\bar X-1/\theta)\xrightarrow{d}N(0,1/\theta^2)$. Apply the delta method to $g(m)=1/m$, whose derivative is $g'(m)=-1/m^2$. At $m=1/\theta$, the derivative is $-\theta^2$, so the asymptotic variance of $\sqrt n(\hat{\theta}-\theta)$ is $(\theta^2)^2(1/\theta^2)=\theta^2$. A plug-in estimate of the standard error is therefore $\hat{\theta}/\sqrt n$.

Here $\hat{\theta}=1/0.80=1.25$, and the estimated standard error is $1.25/\sqrt{100}=0.125$. Hence

Z_W=\frac{1.25-1}{0.125}=2.00.

Because $2.00>1.96$, the two-sided test rejects at level $0.05$. The approximate p-value is $2\{1-\Phi(2.00)\}\approx0.0455$. The calculation is a large-sample test: the cutoff and p-value rely on the normal approximation, not on an assertion that this statistic has an exact standard normal distribution for $n=100$.

Connection to Wald Confidence Intervals

The two-sided test can be read as checking whether the null value lies in the corresponding large-sample Wald confidence interval. This is a direct algebraic relationship, not a separate testing procedure. Define

C_n= \left[ \hat{\theta}-z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\theta}), \; \hat{\theta}+z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\theta}) \right].

Since the estimated standard error is positive, $\theta_0\in C_n$ exactly when

\left|\frac{\hat{\theta}-\theta_0} {\widehat{\operatorname{se}}(\hat{\theta})}\right| \le z_{1-\alpha/2}.

Thus the two-sided Wald test rejects precisely when $\theta_0\notin C_n$, apart from the immaterial convention for equality at the cutoff. The theorem above also establishes the interval’s asymptotic coverage: standardization gives a statistic converging to $N(0,1)$, so the probability that it lies between $-z_{1-\alpha/2}$ and $z_{1-\alpha/2}$ converges to $1-\alpha$. The interval is therefore an equivalent way to report the same large-sample assessment.

Common Pitfalls and Practical Interpretation

Using the wrong tail. For $H_1:\theta>\theta_0$, large positive values count against the null; for $H_1:\theta<\theta_0$, large negative values do. The alternative determines the rejection region.
Using the wrong standard error scale. If $\hat V$ estimates the asymptotic variance of $\sqrt n(\hat{\theta}-\theta)$, the standard error is $\sqrt{\hat V/n}$, not $\sqrt{\hat V}$.
Calling the result exact. The theorem establishes a limiting null distribution. At a finite sample size, the actual rejection probability can differ from $\alpha$.
Choosing a one-sided test after seeing the sign. A directional alternative is justified by the scientific question, not by which direction the observed estimate happens to take.

In practice, report the estimate, its estimated standard error, the null value, the direction of the alternative, and the cutoff or p-value. That information makes the scale of the evidence visible and lets readers distinguish a small estimated effect from an estimate that is precise enough to be inconsistent with the null.

Check Your Understanding

Use the definitions and large-sample rules above to answer these questions.

Under $H_0:\theta=\theta_0$, suppose $\sqrt n(\hat{\theta}-\theta_0)\xrightarrow{d}N(0,V_0)$ and $\hat V\xrightarrow{p}V_0>0$. What estimated standard error makes the Wald statistic converge to a standard normal distribution?
For a two-sided test at level $0.01$, which normal quantile determines the rejection cutoff?
A statistic is $Z_W=-2.1$. Give the approximate p-value formula for a lower-sided test and for a two-sided test.
Why can the upper-sided and two-sided tests in the Bernoulli example lead to different decisions at the same nominal level?
State the algebraic relationship between the two-sided Wald test and its Wald confidence interval.

What You'll Learn