Delta Methods at Parameter-Space Boundaries

Why Boundaries Change the Delta Method

The ordinary delta method linearizes a transformation near the true parameter. At an interior point, small perturbations can usually approach from either side. At a boundary, the parameter and estimator may be restricted to one side, and the transformation may only be defined there. The relevant local behavior is then a one-sided expansion, not necessarily a derivative on an open interval.

A boundary can also change the estimator’s limiting distribution. For example, an estimator constrained to be nonnegative cannot have a nondegenerate normal limit on the whole real line after centering at zero. Even when its rate remains the usual square-root rate, its limit may be concentrated on the feasible side and may have a point mass at the boundary. Applying the ordinary delta method without checking these facts can give an incorrect limit or rate.

Definition: Let the parameter space contain $[\theta_0,\theta_0+\varepsilon)$ for some $\varepsilon>0$, with $\theta_0$ its lower boundary. A transformation $g$ has a finite right derivative $c$ at $\theta_0$ if $g(\theta_0)$ is defined and $$ \lim_{h\downarrow 0}\frac{g(\theta_0+h)-g(\theta_0)}{h}=c. $$ Equivalently, $g(\theta_0+h)=g(\theta_0)+ch+o(h)$ as $h\downarrow0$. This condition concerns only feasible perturbations to the right.

The corresponding definition at an upper boundary uses $h\uparrow0$, so that the perturbations approach zero from below. A one-sided derivative need not agree with the derivative of an arbitrary extension of $g$ outside the parameter space. The estimator and transformation must also be defined where they are used; a finite one-sided derivative cannot repair a transformation that has no finite value at the boundary.

The One-Sided First-Order Result

Suppose $T_n$ is an estimator that remains in the feasible half-line to the right of $\theta_0$, and $a_n$ is a positive sequence tending to infinity. Its scaled error is $W_n=a_n(T_n-\theta_0)$. If $W_n$ has a finite limiting distribution and $g$ has a finite right derivative, the one-sided expansion transfers that limit through $g$.

Theorem: Suppose $T_n\geq\theta_0$ with probability one, $a_n\to\infty$, and $$ a_n(T_n-\theta_0)\xrightarrow{d}W $$ for a finite random variable $W$. Suppose $g(\theta_0)$ is defined and $g(\theta_0+h)=g(\theta_0)+ch+o(h)$ as $h\downarrow0$, where $c$ is finite. Then $$ a_n\{g(T_n)-g(\theta_0)\}\xrightarrow{d}cW. $$

The proof must control the Taylor remainder when its argument is random. Define $r(h)=g(\theta_0+h)-g(\theta_0)-ch$ for $h>0$, and set $r(0)=0$. The one-sided expansion says $r(h)/h\to0$ as $h\downarrow0$. Since $W_n=a_n(T_n-\theta_0)$ converges in distribution, it is tight. For any fixed $M>0$, on $W_n\leq M$ and $W_n>0$, $$ a_n|r(T_n-\theta_0)| = W_n\frac{|r(T_n-\theta_0)|}{T_n-\theta_0}. $$ For large $n$, $T_n-\theta_0\leq M/a_n$ on this event, so the ratio is uniformly small by the definition of the limit $r(h)/h\to0$. Tightness lets us choose $M$ so that $P(W_n>M)$ is arbitrarily small for all sufficiently large $n$. If $W_n=0$, the remainder is zero. Thus $a_nr(T_n-\theta_0)\xrightarrow{p}0$. Consequently, $$ a_n\{g(T_n)-g(\theta_0)\}=cW_n+a_nr(T_n-\theta_0)\xrightarrow{d}cW, $$ by Slutsky’s theorem. $\square$

This result does not require $W$ to be normal. Nor does it require $c$ to be nonzero. If $c=0$, the stated scaling gives a degenerate limit; a higher-order expansion may be needed to find a nondegenerate rate. The result also relies on $T_n$ staying on the side where the one-sided expansion applies.

Worked Examples at a Lower Boundary

Worked Example: A Constrained Normal-Mean Estimator and a Finite Derivative

Let $X_1,\ldots,X_n$ be independent $N(\theta,1)$ observations, with parameter space $\theta\geq0$, and suppose the true value is $\theta_0=0$. The constrained maximum likelihood estimator is $T_n=\max(\bar X,0)$. Find the limit of $\sqrt n\log(1+T_n)$.

At the boundary, $\sqrt n\bar X$ has exactly the $N(0,1)$ distribution. Write $Z\sim N(0,1)$. Then $$ \sqrt n T_n=\max(\sqrt n\bar X,0)\xrightarrow{d}\max(Z,0). $$ The transformation $g(\theta)=\log(1+\theta)$ is defined for $\theta\geq0$, and its right derivative at zero is $$ \lim_{h\downarrow0}\frac{\log(1+h)-\log(1)}{h}=1. $$ The one-sided theorem, with $a_n=\sqrt n$, therefore gives $$ \sqrt n\log(1+T_n)\xrightarrow{d}\max(Z,0). $$ The limit has probability $1/2$ at zero, because $P(Z\leq0)=1/2$. The boundary restriction is visible in the limit; an unconstrained $N(0,1)$ approximation to $\sqrt nT_n$ would assign positive probability to impossible negative values.

Worked Example: A Square-Root Transformation Changes the Rate

Keep the same model and estimator, but transform by $g(\theta)=\sqrt{\theta}$, which is defined on the parameter space. Determine the rate and limiting distribution of $\sqrt{T_n}$ at $\theta_0=0$.

The right difference quotient at zero is $h^{-1/2}$, which diverges as $h\downarrow0$. Thus there is no finite right derivative, and the first-order theorem does not apply. Instead, the transformation has the exact identity $\sqrt{T_n}=T_n^{1/2}$. From the preceding calculation, $$ \sqrt n T_n\xrightarrow{d}\max(Z,0). $$ Taking square roots, a continuous function on $[0,\infty)$, gives $$ n^{1/4}\sqrt{T_n} =\sqrt{\sqrt n T_n} \xrightarrow{d}\sqrt{\max(Z,0)}. $$ The rate is $n^{1/4}$, not $\sqrt n$. The limit is zero when $Z\leq0$, and equals $\sqrt Z$ when $Z>0$. The exponent $1/4$ follows from the square-root transformation applied to an estimator of order $n^{-1/2}$ at the boundary.

Worked Example: A Power Transformation with a Different Fractional Rate

For the same estimator, let $g(\theta)=\theta^{1/3}$ on $[0,\infty)$. Find a rate giving a nondegenerate limit for $g(T_n)$.

Again, the right derivative at zero is infinite, so the finite-derivative theorem is not applicable. The exact power identity gives $$ n^{1/6}T_n^{1/3}=(\sqrt n T_n)^{1/3}. $$ Since $x\mapsto x^{1/3}$ is continuous on $[0,\infty)$ and $\sqrt nT_n\xrightarrow{d}\max(Z,0)$, $$ n^{1/6}T_n^{1/3}\xrightarrow{d}\{\max(Z,0)\}^{1/3}. $$ Thus the rate is $n^{1/6}$. More generally, if $T_n\geq0$, $\sqrt nT_n\xrightarrow{d}W$, and $g(\theta)=\theta^\alpha$ for $0<\alpha<1$, then continuity gives $n^{\alpha/2}T_n^\alpha\xrightarrow{d}W^\alpha$, provided $W\geq0$ almost surely. The rate depends on the local power of the transformation.

An Upper-Boundary Example

Worked Example: A Finite Left Derivative

Let $X_1,\ldots,X_n$ be independent $N(\theta,1)$ observations with parameter space $\theta\leq1$, true value $\theta_0=1$, and constrained estimator $T_n=\min(\bar X,1)$. For $g(\theta)=1/(2-\theta)$, find the limit of $\sqrt n\{g(T_n)-g(1)\}$.

Write $Z\sim N(0,1)$. Since $\sqrt n(\bar X-1)$ is exactly $N(0,1)$, $$ \sqrt n(1-T_n)=\max\{-\sqrt n(\bar X-1),0\}\xrightarrow{d}\max(-Z,0). $$ The variable $-Z$ is also standard normal, so $\max(-Z,0)$ has the same distribution as $\max(Z,0)$. At the upper boundary, put $h=1-\theta\geq0$. Then $$ g(1-h)=\frac{1}{1+h}=1-h+o(h), $$ so the left derivative with respect to $\theta$ is $1$, or equivalently the first-order change is $-h$. Therefore $$ \sqrt n\{g(T_n)-g(1)\}\xrightarrow{d}-\max(Z,0). $$ The negative sign records that $g(T_n)\leq g(1)$ for feasible $T_n\leq1$.

Common Boundary Pitfalls

Using a two-sided derivative when only one side is feasible. The relevant expansion is on the parameter space’s feasible side. An extension outside the space is not part of the statistical problem.
Assuming a constrained estimator has an ordinary normal limit. In the examples, the estimator cannot cross the boundary, and its limit has a point mass there. Use the estimator’s actual limiting distribution.
Applying the finite-derivative formula to a singular transformation. The square-root and fractional-power examples have unbounded difference quotients at zero. Determine the rate from their local power behavior instead.
Transforming a parameter value outside the domain. Before using a transformation, check that both its boundary value and its values at the estimator are defined.

Key takeaway: At a parameter-space boundary, identify the feasible direction, establish the estimator’s limit on that side, and inspect the transformation’s local expansion there. A finite one-sided derivative preserves the estimator’s rate; a singular boundary behavior can change it.

Check Your Understanding

Use the feasible side of the parameter space in each argument.

For a lower boundary at $\theta_0$, state the expansion required for the one-sided first-order theorem.
Why does the constrained estimator in the normal-mean example not have an ordinary $N(0,1)$ limit after multiplication by $\sqrt n$?
If $\sqrt nT_n\xrightarrow{d}W$ with $W\geq0$, what rate and limit follow for $T_n^{2/3}$?
At an upper boundary, which direction must the one-sided difference quotient use?
Why is a finite one-sided derivative insufficient when the transformed parameter is not defined at the boundary?