From Asymptotic Normality to Wald Intervals

From an Approximation to an Interval

An estimator’s asymptotic normal distribution provides a practical way to quantify its uncertainty. If the estimator is centered at the true scalar parameter and its error, after scaling, converges to a normal distribution, a normal critical value can be used to form a two-sided confidence interval. The resulting interval is called a Wald interval.

The construction depends on three quantities: the estimator $\hat{\theta}$, the limiting variance of its scaled error, and a standard error that puts the estimator and its uncertainty on the same scale. The confidence level determines the normal critical value. The word “asymptotic” matters: the claimed coverage is generally an approximation for finite samples, not an exact guarantee.

Definition: Let $\theta$ be a scalar estimand and $\hat{\theta}$ an estimator based on a sample of size $n$. Suppose, under the model and at the parameter value under consideration, $$ \sqrt{n}(\hat{\theta}-\theta)\xrightarrow{d}N(0,V(\theta)), $$ where $0<V(\theta)<\infty$. The Wald interval with nominal confidence level $1-\alpha$, for $0<\alpha<1$, is $$ \left[\hat{\theta}-z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\theta}),\; \hat{\theta}+z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\theta})\right]. $$ Here $z_{1-\alpha/2}$ is the $1-\alpha/2$ quantile of $N(0,1)$, and $\widehat{\operatorname{se}}(\hat{\theta})$ estimates the standard deviation of $\hat{\theta}$. Under the stated limiting distribution, its asymptotic standard error is $\sqrt{V(\theta)/n}$.

If $V(\theta)$ is known, the asymptotic standard error is $\sqrt{V(\theta)/n}$. If it depends on unknown parameters, one common construction estimates it by $\sqrt{\hat V/n}$, where $\hat V$ estimates $V(\theta)$. For this plug-in standard error to be justified by the usual large-sample argument, require $\hat V\xrightarrow{p}V(\theta)$ and $V(\theta)>0$. The precise behavior of such studentized intervals is developed separately; the essential point here is that the standard error must estimate the uncertainty on the scale of $\hat{\theta}$, not the limiting variance of $\sqrt n(\hat{\theta}-\theta)$ without its factor of $1/n$.

Why the Formula Has This Form

Start with the limiting distribution. Dividing the scaled error by its limiting standard deviation gives the standardized statistic $$ \frac{\sqrt n(\hat{\theta}-\theta)}{\sqrt{V(\theta)}}= \frac{\hat{\theta}-\theta}{\sqrt{V(\theta)/n}}. $$ It converges in distribution to $N(0,1)$. Thus the normal approximation places probability $1-\alpha$ between $-z_{1-\alpha/2}$ and $z_{1-\alpha/2}$. Using a standard-error estimate in place of the asymptotic standard error gives the Wald construction.

Result: If $\hat V\xrightarrow{p}V(\theta)$ with $0<V(\theta)<\infty$, then $$ \frac{\hat{\theta}-\theta}{\sqrt{\hat V/n}}\xrightarrow{d}N(0,1). $$ Consequently, the two-sided Wald interval uses the estimate $\sqrt{\hat V/n}$ for the standard error.

Justification. By continuity of $v\mapsto 1/\sqrt v$ at $V(\theta)>0$, $\hat V\xrightarrow{p}V(\theta)$ implies $1/\sqrt{\hat V}\xrightarrow{p}1/\sqrt{V(\theta)}$. The statistic can be written as $$ \frac{\hat{\theta}-\theta}{\sqrt{\hat V/n}} = \sqrt n(\hat{\theta}-\theta)\frac{1}{\sqrt{\hat V}}. $$ The first factor converges in distribution to $N(0,V(\theta))$, and the second converges in probability to the constant $1/\sqrt{V(\theta)}$. Slutsky’s theorem therefore gives the stated $N(0,1)$ limit. $\square$

The endpoint formula also follows directly by inverting the standardized inequality. For a fixed positive standard error $s$, $$ -z_{1-\alpha/2}\leq\frac{\hat{\theta}-\theta}{s}\leq z_{1-\alpha/2} $$ is equivalent, by multiplying by $s$ and rearranging for $\theta$, to $$ \hat{\theta}-z_{1-\alpha/2}s\leq\theta\leq\hat{\theta}+z_{1-\alpha/2}s. $$ For a Wald interval, $s$ is the estimated standard error. This algebra explains the symmetric endpoints around $\hat{\theta}$; it does not make the interval’s finite-sample coverage exact.

Worked Constructions

Worked Example: A Normal Mean With Known Variance

Suppose $X_1,\ldots,X_{25}$ are independent observations from $N(\theta,4)$, and their sample mean is $\bar X=10.4$. The estimator is $\hat{\theta}=\bar X$. Its variance is $4/25$, so $\sqrt n(\bar X-\theta)$ has distribution $N(0,4)$. Thus the asymptotic standard error is $\sqrt{4/25}=0.4$.

For a 95% interval, $z_{0.975}\approx1.959964$. The margin of error is $$ 1.959964(0.4)=0.7839856. $$ The interval is therefore $$ [10.4-0.7839856,\;10.4+0.7839856]=[9.6160144,\;11.1839856]. $$

The model here is normal with known variance, so the sample mean is exactly normal at this sample size. Using the exact quantile $z_{0.975}$ gives exact 95% coverage in this model. Replacing it by the rounded value $1.96$ gives approximately 95% coverage, not exactly 95%.

Worked Example: A Bernoulli Success Probability

Suppose $n=100$ independent Bernoulli observations have $60$ successes. The estimand is the success probability $p$, and the estimator is $\hat p=60/100=0.60$. The asymptotic variance of $\sqrt n(\hat p-p)$ is $p(1-p)$. Estimate it by $\hat V=\hat p(1-\hat p)=0.60(0.40)=0.24$.

The estimated standard error is $$ \sqrt{\frac{\hat V}{n}}=\sqrt{\frac{0.24}{100}}=\sqrt{0.0024}\approx0.048990. $$ Using $z_{0.975}\approx1.96$, the margin is $1.96(0.048990)\approx0.096020$. Hence the 95% Wald interval is $$ [0.60-0.096020,\;0.60+0.096020]\approx[0.503980,\;0.696020]. $$

This is a large-sample interval: its formula does not ensure good coverage for every sample size or every value of $p$. In particular, a symmetric interval can extend below 0 or above 1 in other samples, even though a probability must lie between 0 and 1.

Worked Example: A Poisson Rate

Suppose $X_1,\ldots,X_{64}$ are independent observations from a Poisson distribution with rate $\lambda$, and the observed total is $96$. The estimator is the sample mean, $\hat\lambda=96/64=1.5$. A Poisson variable has variance $\lambda$, so the limiting variance of $\sqrt n(\hat\lambda-\lambda)$ is $\lambda$. Estimate it with $\hat V=\hat\lambda=1.5$.

The estimated standard error is $$ \sqrt{\frac{1.5}{64}}=\sqrt{0.0234375}\approx0.153093. $$ For a 95% Wald interval, the margin of error is $1.96(0.153093)\approx0.300062$. The endpoints are $$ [1.5-0.300062,\;1.5+0.300062]\approx[1.199938,\;1.800062]. $$

The estimator, limiting variance, standard-error estimate, and critical value are all explicit. The interval is symmetric about the observed rate estimate, as the Wald formula requires.

Reading and Reporting a Wald Interval

A reported interval should make clear which estimand is being estimated, which estimator was used, and how its standard error was obtained. It should also state the nominal confidence level. For example, “the 95% Wald interval for the rate is 1.20 to 1.80, using the sample mean and estimated standard error $\sqrt{\bar X/n}$” conveys the construction rather than presenting endpoints without context.

The term nominal confidence level refers to the target level used to choose the critical value. It does not assert that this particular computed interval has a 95% probability of containing a fixed parameter. Nor does asymptotic normality by itself guarantee a close approximation at a particular finite sample size. The quality of the approximation depends on the model, the parameter value, and the amount of information in the data.

Common Mistakes and Limitations

Forgetting the factor $1/\sqrt n$. If the limiting variance is $V(\theta)$ for $\sqrt n(\hat{\theta}-\theta)$, the estimator’s asymptotic variance is $V(\theta)/n$, and its standard error is $\sqrt{V(\theta)/n}$.
Using a variance as though it were a standard error. The interval’s margin uses a standard error, which is on the same scale as the estimator. Take the square root of the variance before multiplying by the critical value.
Calling a rounded critical value exact. A rounded value such as $1.96$ is a numerical approximation to $z_{0.975}$. Calculations using it should be described as approximate.
Assuming symmetry is always appropriate. Wald endpoints are symmetric around the estimate, but the parameter space or sampling distribution may be strongly asymmetric. An interval can even leave the parameter space.
Treating an asymptotic approximation as a finite-sample guarantee. The normal limit is a statement about behavior as $n$ grows. It does not alone establish accurate coverage for a small sample.

Key takeaway: Begin with the limit law for $\sqrt n(\hat{\theta}-\theta)$. Its limiting variance determines the asymptotic standard error after division by $\sqrt n$; a consistent estimate can be used when that variance is unknown. Multiply the resulting standard error by the normal critical value and add and subtract that margin from $\hat{\theta}$.

Check Your Understanding

Check that you can identify each part of the construction before calculating endpoints.

If $\sqrt n(\hat{\theta}-\theta)\xrightarrow{d}N(0,9)$, what is the asymptotic standard error of $\hat{\theta}$ in terms of $n$?
An estimator has limiting variance $V(\theta)$, estimated by $\hat V$. State a consistency condition that justifies using $\sqrt{\hat V/n}$ as its standard error.
Why does using $1.96$ for a 95% interval generally make the resulting normal-reference coverage approximate rather than exactly 95%?
For a two-sided 90% Wald interval, which standard-normal quantile is used?
Give one reason a Wald interval for a probability may be questionable even when its arithmetic is correct.