Why Wald Intervals Depend on Parameterization

Two Ways to Put an Interval on a Transformed Scale

A Wald interval is built from an estimate and an estimated standard error. But if the parameter is re-expressed, there are two reasonable procedures: transform the endpoints of the original interval, or construct a new Wald interval for the transformed parameter. For a nonlinear transformation, these procedures generally give different intervals. This is not a contradiction: the transformed endpoints retain the original interval's geometry, whereas the new Wald interval uses a local linear approximation at the estimate.

Let $\hat{\theta}$ estimate a scalar parameter $\theta$, and let $\hat{s}$ estimate its standard error. For a fixed positive critical value $c$, the Wald interval on the original scale is $[\hat{\theta}-c\hat{s},\hat{\theta}+c\hat{s}]$. If $g$ is differentiable, the delta method gives the transformed estimate $g(\hat{\theta})$ an estimated standard error $|g'(\hat{\theta})|\hat{s}$. The absolute value is needed because a standard error is nonnegative, including when $g$ is decreasing.

Definition: The transformed Wald interval is the interval between the images of the original Wald interval's two endpoints, with those images put in increasing order; when $g$ is monotone on the original interval, this equals the image of that interval under $g$. The new Wald interval on the transformed scale is $$ [g(\hat{\theta})-c|g'(\hat{\theta})|\hat{s},\;g(\hat{\theta})+c|g'(\hat{\theta})|\hat{s}]. $$ The first procedure transforms the interval; the second constructs an interval using the transformed estimate and its delta-method standard error.

A Nonlinear Example: The Log Scale

Worked Example: Transforming a Wald Interval for a Positive Parameter

Suppose $\hat{\theta}=4$, $\hat{s}=1$, and $c=1.96$. Use $g(\theta)=\log\theta$, which is increasing for positive $\theta$. The original Wald interval is $$ [4-1.96(1),\;4+1.96(1)]=[2.04,\;5.96]. $$

Transforming its endpoints gives $$ [\log(2.04),\;\log(5.96)]\approx[0.713,\;1.785]. $$

For a new Wald interval on the log scale, the transformed estimate is $\log(4)\approx1.386$. Since $g'(\theta)=1/\theta$, the delta-method standard error is $|g'(4)|\hat{s}=1/4=0.25$. The new interval is therefore $$ [1.386-1.96(0.25),\;1.386+1.96(0.25)]\approx[0.896,\;1.876]. $$

The intervals differ: their lower endpoints are about $0.713$ and $0.896$, and their upper endpoints are about $1.785$ and $1.876$. The transformed interval keeps the endpoint distances produced by the original scale; the new Wald interval uses the slope of the log function at $4$ to make a symmetric, local approximation.

Why the Intervals Agree to First Order

The difference in the example does not mean that the methods are unrelated. Under standard local conditions, their endpoints differ by an amount of smaller order than the interval's width. The essential condition is that the transformation is monotone throughout the original interval with probability tending to one—not merely that its derivative has a particular sign at the center.

Theorem: Suppose $\hat{\theta}\xrightarrow{p}\theta_0$, $\hat{s}=O_p(n^{-1/2})$, and $c>0$ is fixed. Let $g$ be twice continuously differentiable on an open neighborhood of $\theta_0$, and suppose $g'(\theta_0)\ne 0$. Then, with probability tending to one, $g$ is monotone on an interval containing $[\hat{\theta}-c\hat{s},\hat{\theta}+c\hat{s}]$. If the transformed Wald interval is written with its endpoints ordered, each of its endpoints differs from the corresponding endpoint of the new Wald interval by $O_p(n^{-1})$.

Here is a proof that accounts for endpoint ordering. By continuity of $g'$ and $g'(\theta_0)\ne0$, there is a neighborhood $J$ of $\theta_0$ and a constant $a>0$ such that $g'$ has constant sign on $J$ and $|g'(x)|\ge a$ there. Consistency puts $\hat{\theta}$ in a smaller neighborhood of $\theta_0$ with probability tending to one. Also, $\hat{s}=O_p(n^{-1/2})$ implies $\hat{s}\xrightarrow{p}0$, so the entire interval with endpoints $\hat{\theta}\pm c\hat{s}$ lies in $J$ with probability tending to one. Thus $g$ is monotone throughout that interval on this event, and its endpoint images can be ordered according to the sign of $g'$ on $J$.

Because $g''$ is continuous, it is bounded in absolute value by some finite $M$ on a sufficiently small closed neighborhood inside $J$. Taylor's theorem, applied at $\hat{\theta}$, gives for each $q\in\{-1,1\}$ $$ g(\hat{\theta}+qc\hat{s}) =g(\hat{\theta})+qc\hat{s}g'(\hat{\theta})+R_q, \qquad |R_q|\le \frac{M}{2}c^2\hat{s}^{\,2}. $$

If $g'$ is positive on $J$, the lower and upper transformed endpoints correspond respectively to $q=-1$ and $q=1$. If $g'$ is negative on $J$, they correspond respectively to $q=1$ and $q=-1$. In either case, the linear terms at the ordered endpoints are $g(\hat{\theta})-c|g'(\hat{\theta})|\hat{s}$ and $g(\hat{\theta})+c|g'(\hat{\theta})|\hat{s}$. The Taylor bound shows that each endpoint discrepancy is at most $Mc^2\hat{s}^{\,2}/2$. Since $\hat{s}=O_p(n^{-1/2})$, $\hat{s}^{\,2}=O_p(n^{-1})$, proving the assertion. $\square$

This result describes first-order agreement, not equality. Curvature contributes a remainder, and the two endpoint constructions need not match exactly at a finite sample size. The comparison also relies on local monotonicity: it makes the image of the original interval an interval whose endpoints are the images of its two endpoints.

Affine Transformations Are Different

For an affine transformation $g(\theta)=a+b\theta$, with $b\ne0$, the two procedures agree exactly. The original interval has endpoints $\hat{\theta}\pm c\hat{s}$. Their images are $a+b\hat{\theta}\pm bc\hat{s}$, which, after ordering, have endpoints $a+b\hat{\theta}\pm c|b|\hat{s}$. Since $|g'(\hat{\theta})|=|b|$, these are precisely the endpoints of the new Wald interval. Nonlinearity, rather than transformation itself, is what allows the procedures to differ.

Worked Example: Exact Agreement Under a Linear Rescaling

Let $\hat{\theta}=5$, $\hat{s}=0.4$, $c=1.96$, and $g(\theta)=3\theta+2$. The original interval is $$ [5-1.96(0.4),\;5+1.96(0.4)]=[4.216,\;5.784]. $$

Transforming the endpoints gives $[3(4.216)+2,\;3(5.784)+2]=[14.648,\;19.352]$. The transformed estimate is $17$, and its standard error is $|3|(0.4)=1.2$. The new Wald interval is $[17-1.96(1.2),\;17+1.96(1.2)]=[14.648,\;19.352]$, exactly the same interval.

Curvature in a Probability Example

Worked Example: A Logit Wald Interval

Suppose $\hat{p}=0.400$, $\hat{s}=0.050$, and $c=1.96$, and consider $g(p)=\log\{p/(1-p)\}$. The original interval is $[0.400-1.96(0.050),\;0.400+1.96(0.050)]=[0.302,\;0.498]$. It lies inside $(0,1)$, the domain of the logit transformation.

Transforming the endpoints gives $$ \left[\log\left(\frac{0.302}{0.698}\right),\; \log\left(\frac{0.498}{0.502}\right)\right] \approx[-0.838,\;-0.008]. $$

For the new Wald interval, $g(0.400)=\log(0.400/0.600)\approx-0.4055$, and $g'(p)=1/\{p(1-p)\}$. Thus $$ |g'(0.400)|\hat{s} =\frac{0.050}{(0.400)(0.600)} \approx0.2083. $$

The new interval is approximately $[-0.4055-1.96(0.2083),\;-0.4055+1.96(0.2083)]=[-0.814,\;0.003]$. It is close to, but not the same as, the transformed interval. The difference arises because the logit curve is not a straight line across the original interval.

Common Pitfall: Checking the Derivative Only at the Estimate

A positive derivative at $\hat{\theta}$ alone does not guarantee that $g(\hat{\theta}-c\hat{s})$ is the lower transformed endpoint. The derivative could change sign between the center and an endpoint. For example, let $g(x)=x^2-x^3$, $\hat{\theta}=n^{-3}$, $\hat{s}=n^{-1}$, and $c=1$. At the center, $$ g'(\hat{\theta})=2n^{-3}-3n^{-6}>0 $$ for all sufficiently large $n$. But the difference between the two endpoint images is $$ g(\hat{\theta}+\hat{s})-g(\hat{\theta}-\hat{s}) =2\hat{s}\{g'(\hat{\theta})-\hat{s}^{\,2}\}<0 $$ for sufficiently large $n$, because $g'(\hat{\theta})$ is of order $n^{-3}$ whereas $\hat{s}^{\,2}=n^{-2}$. The endpoint images are reversed despite the positive derivative at the center. In this sequence, $\hat{s}=O(n^{-1/2})$; what fails is the theorem's local regularity around a limiting parameter with nonzero derivative. The safe comparison either establishes monotonicity across the whole interval with probability tending to one, as above, or explicitly sorts the endpoint images before comparing them.

A related mistake is to assume the transformed interval must be symmetric about $g(\hat{\theta})$. That is true for the new Wald interval by construction, but generally false for the image of the original interval. A nonlinear function need not place the two transformed endpoints at equal distances from its value at the center.

Key takeaway: Transforming an existing Wald interval preserves its endpoint mapping; constructing a new Wald interval uses a derivative-based linearization. For a smooth transformation with nonzero derivative near the true parameter, the two are first-order close but generally not identical. They are exactly identical under affine transformations.

Check Your Understanding

Use the definitions and endpoint comparison above to answer these questions.

For a differentiable decreasing transformation, why does the delta-method standard error use $|g'(\hat{\theta})|$ rather than $g'(\hat{\theta})$?
What condition on $g$ across the original interval ensures its endpoint images can be ordered by the sign of its derivative?
For $g(\theta)=\log\theta$, explain why transforming the endpoints and constructing a new Wald interval need not give identical limits.
Why do the two procedures agree exactly for $g(\theta)=a+b\theta$ when $b\ne0$?
Why is a positive value of $g'(\hat{\theta})$ by itself insufficient to determine which endpoint image is smaller?