Wald Confidence Intervals on a Stabilized Scale

Why Construct the Interval on a Stabilized Scale?

An estimator can have an asymptotic variance that depends strongly on the parameter being estimated. A transformation can sometimes make that variance constant, or at least easier to estimate. Once the transformed estimator has an approximately normal distribution with a well-behaved standard error, a two-sided Wald interval follows by taking the estimate plus and minus a normal critical value times that standard error.

The interval developed here is for the transformed target itself. Let \(g\) be the chosen transformation, let \(\psi=g(\theta)\), and let \(\hat{\psi}=g(\hat{\theta})\). The aim is to quantify uncertainty about \(\psi\) using a normal approximation on this scale. This is an asymptotic procedure: its nominal coverage is justified as \(n\) grows, not guaranteed for every finite sample.

Definition: A Wald confidence interval on a stabilized scale is an interval centered at \(\hat{\psi}=g(\hat{\theta})\), with half-width equal to a normal critical value times an estimated standard error for \(\hat{\psi}\). In the setting below, that standard error is \(\hat{\tau}/\sqrt{n}\), where \(\hat{\tau}\) estimates the limiting standard deviation of \(\sqrt{n}(\hat{\psi}-\psi)\).

Assumptions and the Interval Formula

Suppose \(\theta\) is an interior parameter value, \(g\) is defined at \(\theta\), and the estimator satisfies

\sqrt{n}\bigl(g(\hat{\theta})-g(\theta)\bigr) \xrightarrow{d} N(0,\tau^2), \qquad 0<\tau<\infty.

Suppose also that \(\hat{\tau}\xrightarrow{p}\tau\). Because an estimated standard deviation is used as a divisor and as a half-width, it must be positive and finite wherever the usual interval is formed. It is enough to require \(P(0<\hat{\tau}<\infty)\to 1\). A convenient stronger condition, often met by standard scale estimates, is that \(\hat{\tau}\) is positive and finite with probability one for every sample size under consideration.

For \(0<\alpha<1\), write \(z_{1-\alpha/2}\) for the \(1-\alpha/2\) quantile of the standard normal distribution. On outcomes for which \(0<\hat{\tau}<\infty\), the interval is

\left[ \hat{\psi}-z_{1-\alpha/2}\frac{\hat{\tau}}{\sqrt{n}}, \quad \hat{\psi}+z_{1-\alpha/2}\frac{\hat{\tau}}{\sqrt{n}} \right].

If the scale estimate fails to be positive and finite on some outcomes, one may define any interval on those outcomes to complete the procedure. Since their probability tends to zero under the stated condition, this choice does not affect the limiting coverage. In applications, it is preferable to use a scale estimator that is positive and finite on the relevant sample space.

Theorem: If the transformed-estimator limit and scale-estimator conditions above hold, the interval has asymptotic coverage \(1-\alpha\) for \(\psi=g(\theta)\).

Proof. Let \(A_n=\{0<\hat{\tau}<\infty\}\), so \(P(A_n)\to1\). On \(A_n\), define

T_n=\frac{\sqrt{n}\{\hat{\psi}-\psi\}}{\hat{\tau}}.

The numerator converges in distribution to \(N(0,\tau^2)\), and \(\hat{\tau}\xrightarrow{p}\tau>0\). Slutsky’s theorem therefore gives \(T_n\xrightarrow{d}N(0,1)\) on events whose probability tends to one. More formally, define \(T_n\) arbitrarily outside \(A_n\); changing a statistic on events with probability tending to zero does not change its limiting distribution.

On \(A_n\), positivity of \(\hat{\tau}\) makes the interval endpoints correctly ordered, and its coverage event is exactly

\left\{\psi\text{ belongs to the interval}\right\} = \left\{|T_n|\leq z_{1-\alpha/2}\right\}.

The standard normal distribution assigns probability \(1-\alpha\) to \([-z_{1-\alpha/2},z_{1-\alpha/2}]\), and has no point masses at the two endpoints. Convergence in distribution thus implies

P\left(|T_n|\leq z_{1-\alpha/2}\right)\longrightarrow 1-\alpha.

Any discrepancy between this event and coverage on \(A_n\) is confined to \(A_n^c\), whose probability tends to zero. The interval consequently has limiting coverage \(1-\alpha\). \(\square\)

Obtaining the Stabilized Standard Error

Often the asymptotic variance of \(\hat{\theta}\) is known or estimable, and the transformation is chosen to simplify the variance of \(g(\hat{\theta})\). For example, if \(\sqrt{n}(\hat{\theta}-\theta)\xrightarrow{d}N(0,V(\theta))\) and \(g\) is differentiable at \(\theta\), the scalar delta method gives the transformed limiting variance

\tau^2(\theta)=\{g'(\theta)\}^2V(\theta).

When \(g\) is selected so this expression is a constant, \(\tau\) is the stabilized asymptotic standard deviation. If it is not constant, a consistent estimate can be obtained by substituting consistent estimates of the relevant quantities, provided the resulting estimate converges in probability to a positive finite value. The interval formula still applies; stabilization is useful, but a constant variance is not required for the Wald construction.

For instance, with an estimated variance function \(\hat V\), a natural plug-in scale is \(|g'(\hat{\theta})|\sqrt{\hat V(\hat{\theta})}\), when it is defined and consistent for \(|g'(\theta)|\sqrt{V(\theta)}\). The absolute value matters because a standard deviation cannot be negative. The asymptotic coverage theorem depends on consistency for a positive scale, not on the particular method used to estimate it.

Result: Under the theorem’s conditions, the interval’s length on the transformed scale is \(2z_{1-\alpha/2}\hat{\tau}/\sqrt{n}\). In particular, \(\sqrt{n}\) times that length converges in probability to \(2z_{1-\alpha/2}\tau\).

Derivation. On \(A_n\), subtract the lower endpoint from the upper endpoint; the result is \(2z_{1-\alpha/2}\hat{\tau}/\sqrt{n}\). Multiplying by \(\sqrt{n}\) gives \(2z_{1-\alpha/2}\hat{\tau}\), which converges in probability to \(2z_{1-\alpha/2}\tau\) by consistency. Since \(P(A_n)\to1\), the conclusion is unaffected by how the length is defined on \(A_n^c\). Thus the interval’s scale is \(n^{-1/2}\), with its leading width determined by the limiting standard deviation and the chosen confidence level. \(\square\)

Worked Applications

The examples below calculate intervals on their stabilized scales. Their endpoints refer to the transformed target; they are not intervals on the original parameter scale.

Worked Example: A Log-Transformed Parameter in a Normal Model

Let \(Y_1,\ldots,Y_n\) be independent \(N(\mu,\sigma^2)\) observations, with known \(\sigma=0.6\). The target is \(\theta=e^\mu\), and the estimator is \(\hat{\theta}=e^{\bar Y}\). Use \(g(\theta)=\log\theta\), so \(\psi=\mu\) and \(\hat{\psi}=\bar Y\). The sampling distribution is exact:

\sqrt{n}(\hat{\psi}-\psi)=\sqrt{n}(\bar Y-\mu)\sim N(0,\sigma^2).

Thus \(\tau=\sigma=0.6\). Suppose \(n=36\) and the observed mean is \(\bar y=1.10\). For a 95% interval, \(z_{0.975}=1.96\), and the estimated standard error on the transformed scale is \(0.6/\sqrt{36}=0.1\). The half-width is \(1.96(0.1)=0.196\), so the interval for \(\psi=\log\theta\) is

[1.10-0.196,\ 1.10+0.196]=[0.904,\ 1.296].

The calculation uses the normal critical value and the standard error of \(\bar Y\). In this model the normal approximation happens to be exact, but the general theorem only promises asymptotic coverage.

Worked Example: A Bounded Parameter With an Estimated Scale

Let \(X_1,\ldots,X_n\) be independent \(N(\mu,\sigma^2)\) observations, with unknown \(\sigma\). Define the parameter \(\theta=\tanh(\mu)\), which lies in \((-1,1)\), and use \(g(\theta)=\operatorname{atanh}(\theta)\). Then \(g(\theta)=\mu\). Since \(\hat{\mu}=\bar X\) estimates \(\mu\), set \(\hat{\theta}=\tanh(\bar X)\); consequently \(g(\hat{\theta})=\bar X\). The central limit theorem gives the transformed limit with \(\tau=\sigma\), and the sample standard deviation \(S\) is consistent for \(\sigma\). Here \(S>0\) with probability one for a nondegenerate continuous normal sample when \(n\geq2\).

Suppose \(n=25\), \(\bar x=0.40\), and \(s=1.00\). The plug-in standard error is \(s/\sqrt{n}=1/5=0.20\). Using \(z_{0.975}=1.96\), the half-width is \(1.96(0.20)=0.392\). The 95% Wald interval for the transformed target \(\psi=\operatorname{atanh}(\theta)=\mu\) is

[0.40-0.392,\ 0.40+0.392]=[0.008,\ 0.792].

The interval is centered at the transformed estimate \(\bar x\), and its scale estimate is positive and finite. It uses the large-sample normal critical value, not a finite-sample \(t\) critical value.

Worked Example: A Lognormal Mean With an Estimated Stabilized Scale

Suppose \(X_1,\ldots,X_n\) are independent lognormal observations: \(\log X_i\sim N(\mu,\sigma^2)\), with \(\sigma^2>0\). The mean is \(\theta=E[X_i]=e^{\mu+\sigma^2/2}>0\); estimate it by \(\hat{\theta}=\bar X\), and take \(g(\theta)=\log\theta\). The ordinary central limit theorem gives \(\sqrt{n}(\bar X-\theta)\xrightarrow{d}N(0,\operatorname{Var}(X_i))\). Applying the delta method to the logarithm gives

\sqrt{n}\{\log(\bar X)-\log(\theta)\} \xrightarrow{d} N\left(0,\frac{\operatorname{Var}(X_i)}{\theta^2}\right) = N\left(0,e^{\sigma^2}-1\right).

The variance identity follows from \(E[X_i^2]=e^{2\mu+2\sigma^2}\) and \(\theta^2=e^{2\mu+\sigma^2}\), so \(E[X_i^2]/\theta^2=e^{\sigma^2}\). A consistent scale estimate is \(\hat{\tau}=S/\bar X\), where \(S^2\) is the sample variance with denominator \(n-1\). Indeed, \(S^2\xrightarrow{p}\operatorname{Var}(X_i)\) and \(\bar X\xrightarrow{p}\theta>0\), so \(S/\bar X\xrightarrow{p}\sqrt{e^{\sigma^2}-1}>0\). For positive lognormal observations, \(\bar X>0\), and \(S>0\) almost surely for a nondegenerate sample with \(n\geq2\).

For an observed sample with \(n=100\), \(\bar x=2\), and \(s^2=4\), the estimated scale is \(s/\bar x=2/2=1\). The standard error for \(\log(\theta)\) is therefore \(1/\sqrt{100}=0.1\). The 95% half-width is \(1.96(0.1)=0.196\). Since \(\log(2)\approx0.6931\), the interval for the log mean is

[0.6931-0.196,\ 0.6931+0.196]=[0.4971,\ 0.8891].

This is an asymptotic interval: the lognormal sample mean need not be normally distributed at the observed sample size. Its justification is the central limit theorem, the delta method, and consistency of the estimated scale.

Common Pitfall: Treating a Consistent Scale as Always Positive

A frequent error is to argue that \(\hat{\tau}\xrightarrow{p}\tau>0\), then use the exact event identity \(\{\psi\text{ is in the interval}\}=\{|T_n|\leq z_{1-\alpha/2}\}\) without checking whether \(\hat{\tau}\) is positive for the observed sample. Consistency alone does not imply finite-sample positivity. An estimator can be negative with small probability for every \(n\), even though that probability tends to zero. When \(\hat{\tau}<0\), the displayed endpoints are reversed, and when \(\hat{\tau}=0\), the standardized statistic is undefined.

The correction is to state the event identity only on \(A_n=\{0<\hat{\tau}<\infty\}\), and to use \(P(A_n)\to1\) to establish the asymptotic conclusion. On \(A_n\), the interval is correctly ordered and the identity holds exactly. Outside \(A_n\), its effect on coverage is bounded by \(P(A_n^c)\), which vanishes. Alternatively, require \(\hat{\tau}\) to be positive and finite for every sample outcome. In either case, distinguish an exact finite-sample identity from an asymptotic coverage statement.

A second important distinction is between the target and the estimate: the interval is centered at \(g(\hat{\theta})\) and covers \(g(\theta)\). The standard error must describe the variability of \(g(\hat{\theta})\), not simply the standard error of \(\hat{\theta}\) reused unchanged.

Check Your Understanding

Answer each question using the conditions and constructions developed above.

State the limiting distribution and scale-estimator conditions that justify a Wald interval for \(g(\theta)\).
Why does positivity of \(\hat{\tau}\) matter for the exact coverage-event equivalence?
If \(\sqrt{n}(\hat{\theta}-\theta)\xrightarrow{d}N(0,V(\theta))\), what is the delta-method expression for the transformed limiting variance?
In the lognormal-mean example, why is \(S/\bar X\) a consistent estimate of the stabilized scale?
What is the 95% half-width when \(n=64\) and the estimated stabilized scale is \(1.5\)?