Common Errors in Delta-Method Interval Construction

Why Delta-Method Intervals Need an Audit

A delta-method interval is a sequence of linked approximations. First, an estimator is approximately normal on its original scale. A differentiable transformation then changes its first-order variance, and an estimated standard error is used to form an interval. If any link is mishandled, the final limits can be numerically plausible but statistically unjustified.

The checks are practical: include the derivative, use the variance on the correct scale, justify every plug-in quantity, and take the inverse image of the entire transformed interval. Even when all these steps are correct, the usual claim is asymptotic coverage, not exact finite-sample coverage.

Definition: Suppose $\hat{\theta}$ estimates a scalar parameter $\theta$, and for a positive function $V(\theta)$, $$ \sqrt{n}(\hat{\theta}-\theta)\xrightarrow{d}N(0,V(\theta)). $$ For a differentiable function $g$, a first-order delta-method interval is formed using the approximate variance $g'(\theta)^2V(\theta)/n$ of $g(\hat{\theta})$, usually with unknown quantities replaced by consistent estimates. The resulting interval is an approximation unless a separate finite-sample argument establishes exact coverage.

The Derivative and the Variance Scale

The derivative is not an optional adjustment. A small change in $\hat{\theta}$ produces, to first order, a change of approximately $g'(\theta)(\hat{\theta}-\theta)$ in $g(\hat{\theta})$. Squaring that multiplier gives the transformed variance. The factor $1/n$ also matters: $V(\theta)$ is the limiting variance of $\sqrt n(\hat{\theta}-\theta)$, not the variance of $\hat{\theta}$ itself.

Theorem: If $\sqrt n(\hat{\theta}-\theta)\xrightarrow{d}N(0,V(\theta))$, where $0<V(\theta)<\infty$, and $g$ is differentiable at $\theta$, then $$ \sqrt n\{g(\hat{\theta})-g(\theta)\} \xrightarrow{d}N\!\left(0,g'(\theta)^2V(\theta)\right). $$ If $g'(\theta)\ne 0$, this limit is nondegenerate.

Proof. Differentiability at $\theta$ means that, for $u$ near zero, $g(\theta+u)-g(\theta)=g'(\theta)u+r(u)$, where $r(u)/u\to0$ as $u\to0$. The assumed convergence in distribution implies $\hat{\theta}-\theta=O_p(n^{-1/2})$, so $\hat{\theta}\xrightarrow{p}\theta$. Consequently $r(\hat{\theta}-\theta)/(\hat{\theta}-\theta)\xrightarrow{p}0$ whenever the ratio is defined, with the remainder taken as zero when its argument is zero. Also, $\sqrt n(\hat{\theta}-\theta)=O_p(1)$. Their product gives $\sqrt n r(\hat{\theta}-\theta)=o_p(1)$. Thus $$ \sqrt n\{g(\hat{\theta})-g(\theta)\} =g'(\theta)\sqrt n(\hat{\theta}-\theta)+o_p(1). $$ The result follows from the assumed limit and Slutsky’s theorem. $\square$

Worked Example: A Missing Derivative on the Log Scale

Suppose $X_1,\ldots,X_n$ are independent exponential observations with mean $\theta>0$, and estimate $\theta$ by $\bar X$. Its first-order variance is $\theta^2/n$, so $V(\theta)=\theta^2$ for the root-$n$ convention above. Consider $g(\theta)=\log\theta$. For $n=25$ and observed $\bar X=2.4$, construct the usual 95% first-order interval on the log scale and back-transform it.

The derivative is $g'(\theta)=1/\theta$. The estimated standard error on the log scale is therefore $$ |g'(\bar X)|\sqrt{\frac{\bar X^2}{n}} =\frac{1}{2.4}\sqrt{\frac{2.4^2}{25}} =\frac{1}{5}=0.2. $$ The interval on the log scale is $\log(2.4)\pm1.96(0.2)$. Since $\log(2.4)\approx0.87547$, its limits are $0.48347$ and $1.26747$. Exponentiating gives approximately $[1.622,3.552]$.

If the derivative is omitted while the estimated original-scale standard deviation $2.4/5=0.48$ is used as though it were a log-scale standard error, the log-scale half-width becomes $1.96(0.48)=0.9408$, not $0.392$. That calculation mixes scales and produces a different, unjustified interval.

Worked Example: Using the Root-n Variance at the Right Scale

Let $X_1,\ldots,X_{25}$ be independent $N(\theta,4)$ observations, with $\hat{\theta}=\bar X$, and suppose $\bar X=0.6$. Estimate $g(\theta)=e^\theta$ using a first-order 95% interval on the $g$-scale.

Here $\operatorname{Var}(\bar X)=4/25=0.16$, while the root-$n$ asymptotic variance coefficient is $V(\theta)=4$. Since $g'(\theta)=e^\theta$, the estimated standard error for $g(\hat{\theta})$ is $$ e^{0.6}\sqrt{\frac{4}{25}} =1.82212(0.4)\approx0.72885. $$ Thus the interval on the transformed scale is $$ 1.82212\pm1.96(0.72885) \approx[0.39358,3.25066]. $$

Using $4$ as if it were $\operatorname{Var}(\bar X)$ would instead give the standard error $e^{0.6}\sqrt{4}\approx3.64424$, too large by a factor of five. Conversely, if one starts with the already scaled variance $0.16$, dividing it by $n$ again gives a standard error that is too small. Always establish whether the quantity being plugged in estimates $V(\theta)$ or $\operatorname{Var}(\hat{\theta})$.

The range of $g(\theta)=e^\theta$ is $(0,\infty)$. Both limits here are in that range, so applying the logarithm gives the corresponding interval for $\theta$: approximately $[\log(0.39358),\log(3.25066)]=[-0.932,1.179]$.

Plug-in Standard Errors and Studentization

In practice, $\theta$ and $V(\theta)$ are usually unknown. A common construction uses $\hat V$ with $\hat V\xrightarrow{p}V(\theta)$, and evaluates the derivative at $\hat{\theta}$. This step requires more than differentiability of $g$ at the true parameter: to ensure the estimated derivative behaves properly, a sufficient condition is that $g'$ exists in a neighborhood of $\theta$ and is continuous at $\theta$. It also requires $\hat{\theta}$ to fall in that neighborhood with probability tending to one. If $V(\theta)>0$ and $g'(\theta)\ne0$, these conditions support the studentized statistic below.

Theorem: Under the preceding conditions, if $\hat V\xrightarrow{p}V(\theta)>0$ and $g'$ exists in a neighborhood of $\theta$ and is continuous at $\theta$, then $$ \frac{\sqrt n\{g(\hat{\theta})-g(\theta)\}} {|g'(\hat{\theta})|\sqrt{\hat V}} \xrightarrow{d}N(0,1), $$ provided $g'(\theta)\ne0$. Hence, for $0<\alpha<1$, the interval $$ \left[ g(\hat{\theta})-z_{1-\alpha/2}|g'(\hat{\theta})|\sqrt{\frac{\hat V}{n}}, \quad g(\hat{\theta})+z_{1-\alpha/2}|g'(\hat{\theta})|\sqrt{\frac{\hat V}{n}} \right] $$ has asymptotic coverage $1-\alpha$ for $g(\theta)$. Here $z_{1-\alpha/2}$ is the $1-\alpha/2$ quantile of $N(0,1)$.

Proof. Continuity of $g'$ at $\theta$, together with $\hat{\theta}\xrightarrow{p}\theta$, gives $g'(\hat{\theta})\xrightarrow{p}g'(\theta)$, and the derivative is defined with probability tending to one. Consistency of $\hat V$ then gives $|g'(\hat{\theta})|\sqrt{\hat V}\xrightarrow{p}|g'(\theta)|\sqrt{V(\theta)}>0$. By the first theorem, the numerator converges in distribution to a normal variable with variance $g'(\theta)^2V(\theta)$. Dividing by the denominator and applying Slutsky’s theorem yields the stated standard normal limit. The interval contains $g(\theta)$ exactly when the absolute value of this statistic is at most $z_{1-\alpha/2}$. The limiting normal distribution assigns probability $1-\alpha$ to that event, since its endpoints have probability zero. $\square$

Worked Example: Plugging In for a Binomial Logit

Let $X\sim\operatorname{Binomial}(n,p)$, with $0<p<1$, and estimate $p$ by $\hat p=X/n$. For $g(p)=\log\{p/(1-p)\}$, the root-$n$ variance coefficient for $\hat p$ is $V(p)=p(1-p)$, and $g'(p)=1/\{p(1-p)\}$. Suppose $n=50$ and $X=10$, so $\hat p=0.2$.

The plug-in standard error on the logit scale is $$ |g'(\hat p)|\sqrt{\frac{\hat p(1-\hat p)}{n}} =\frac{1}{0.2(0.8)}\sqrt{\frac{0.2(0.8)}{50}} =\frac{1}{\sqrt{50(0.16)}}\approx0.35355. $$ The estimate is $g(\hat p)=\log(0.25)\approx-1.38629$. The 95% logit interval has limits $-1.38629\pm1.96(0.35355)$, approximately $[-2.07926,-0.69333]$. Applying the inverse logit gives approximately $[0.111,0.333]$.

This is a plug-in, first-order interval, not an exact binomial interval. If $X=0$ or $X=n$, the sample logit is not finite; the regular interior calculation cannot simply be used at those data values. Boundary cases require a method whose assumptions and construction address the boundary.

Back-Transformation Means Taking an Inverse Image

A transformed interval is a set of possible values for $g(\theta)$. To obtain a confidence set for $\theta$, take all parameter values whose transformed values lie in that set. If $g$ is strictly monotone on the parameter space, the set is an interval when the transformed limits are in the range of $g$; its endpoint order is reversed when $g$ is decreasing. If a transformed endpoint falls outside the range, restrict the transformed interval to the range before applying the inverse. If $g$ is not one-to-one, the inverse image can be disconnected and need not be an interval.

Worked Example: A Nonmonotone Transformation and a Range Restriction

Suppose a transformed interval for $g(\theta)=\theta^2$, with parameter space $\mathbb R$, is $[1,4]$. The inverse image is $$ \{\theta\in\mathbb R:1\le\theta^2\le4\} =[-2,-1]\cup[1,2]. $$ Reporting only $[1,2]$ silently assumes $\theta\ge0$. Without that parameter restriction, it drops possible values and changes the confidence set.

For a separate range issue, let $g(\theta)=e^\theta$, whose range is $(0,\infty)$, and suppose a transformed interval is $[-1,1]$. Its intersection with the range is $(0,1]$. The inverse image is therefore $$ \{\theta:e^\theta\in[-1,1]\} =\{\theta:e^\theta\in(0,1]\} =(-\infty,0]. $$ Taking $\log(-1)$ is undefined; applying the inverse to the entire transformed interval’s endpoints is not a valid operation.

Asymptotic Coverage Is Not Exact Coverage

The coverage proof establishes a limit as $n$ grows. It does not establish that the interval has probability exactly $1-\alpha$ at a particular finite sample size. The delta method approximates the distribution near the parameter using a first-order expansion; finite-sample skewness, curvature, and boundary behavior can all affect coverage. An exact claim needs a separate derivation from the finite-sample distribution.

Worked Example: A Delta Interval Need Not Have Exact Coverage

Let $X_1,\ldots,X_n$ be independent $N(\theta,1)$ observations, and consider the 95% delta interval for $e^\theta$: $$ \left[ e^{\bar X}-1.96\frac{e^{\bar X}}{\sqrt n}, \quad e^{\bar X}+1.96\frac{e^{\bar X}}{\sqrt n} \right]. $$ Take $n=4$ and $\theta=0$, so the true transformed parameter is $e^\theta=1$, and $\bar X\sim N(0,1/4)$.

Here $1.96/\sqrt n=0.98$, so the interval endpoints are $0.02e^{\bar X}$ and $1.98e^{\bar X}$. It covers 1 exactly when $0.02e^{\bar X}\le1\le1.98e^{\bar X}$, equivalently $-\log(1.98)\le\bar X\le\log(50)$. Standardizing by the standard deviation $0.5$, the coverage probability is $$ \Phi\!\left(\frac{\log(50)}{0.5}\right) -\Phi\!\left(\frac{-\log(1.98)}{0.5}\right) \approx \Phi(7.824)-\Phi(-1.366) \approx0.914. $$ This finite-sample coverage is not 0.95. The calculation does not contradict the asymptotic theorem: that theorem concerns the limit as $n\to\infty$, not exact coverage at $n=4$.

A Practical Error Check

Before reporting a delta-method interval, check each link in the calculation:

Derivative: Write $g'(\hat{\theta})$ explicitly and square the derivative when calculating variance. A derivative can also be zero, in which case this first-order nondegenerate approximation does not apply.
Scaling: Identify whether the variance quantity is $V(\theta)$, the variance coefficient for $\sqrt n(\hat{\theta}-\theta)$, or the variance of $\hat{\theta}$, which is approximately $V(\theta)/n$.
Plug-in: Check consistency and that the derivative is defined and behaves continuously near the parameter. A formula evaluated at a boundary estimate may not meet those requirements.
Inverse image: Check the range and monotonicity of $g$. Use the inverse image of the full transformed interval, not an inverse applied mechanically to endpoints.
Coverage language: Say “asymptotic 95% coverage” when that is what the limiting argument proves. Do not call the interval exact without a finite-sample proof.

These checks separate errors in algebra from errors in interpretation. In particular, a narrow interval is not evidence that a variance was scaled correctly, and neatly ordered back-transformed limits do not establish that the inverse was defined or that the resulting set has the claimed coverage.

Check Your Understanding

For each question, identify the assumptions or calculation that justify the conclusion.

If $\sqrt n(\hat{\theta}-\theta)$ has limiting variance $V(\theta)$, what is the first-order variance of $g(\hat{\theta})$, and where does the derivative enter?
A variance estimate is for $\hat{\theta}$, not for $\sqrt n(\hat{\theta}-\theta)$. What scaling should be checked before using it in an interval?
Why is differentiability of $g$ at $\theta$ alone insufficient to justify substituting $g'(\hat{\theta})$ into a studentized statistic?
For $g(\theta)=\theta^2$ on $\mathbb R$, what is the inverse image of the transformed interval $[1,9]$?
What additional argument is needed to call a delta-method interval’s coverage exactly $1-\alpha$ at a finite sample size?