Worked Problems at Kinks and Boundaries

Why Kinks and Boundaries Need Care

The delta method transfers a limit for an estimator to a smooth transformation of that estimator. At a kink, however, there may be no single linear approximation: approaching from different directions can produce different slopes. At a parameter-space boundary, only some directions may be feasible. And when the first derivative vanishes, the usual first-order scaling may give a degenerate limit.

These cases are not exceptions to asymptotic reasoning; they are situations in which the approximation must match the local geometry. The examples below show how to choose among an ordinary first-order delta method, a directional delta method, and a higher-order expansion.

Directional Limits at a Kink

Definition: Let \(D\subseteq\mathbb R^d\), let \(\theta\in D\), and let \(C\subseteq\mathbb R^d\) be a set of allowable directions. A function \(g:D\to\mathbb R^k\) is Hadamard directionally differentiable at \(\theta\), tangentially to \(C\), with derivative \(g'_\theta:C\to\mathbb R^k\), if, whenever \(t_m>0\), \(t_m\to0\), \(h_m\to h\in C\), and \(\theta+t_mh_m\in D\), we have \[ \frac{g(\theta+t_mh_m)-g(\theta)}{t_m}\longrightarrow g'_\theta(h). \] The derivative is directional: it need not be linear in \(h\).

For an estimator \(T_n\), write \(a_n(T_n-\theta)\xrightarrow{d}Z\), where \(a_n\to\infty\). The usual differentiable delta method uses a linear derivative. At a kink, a direction-dependent derivative can replace it, provided the estimator’s scaled deviations lie in the relevant directions and the derivative behaves continuously there.

Directional delta method: Suppose \(T_n\in D\), \(a_n(T_n-\theta)\in C\) with probability tending to one, \(C\) is closed, and \(a_n(T_n-\theta)\xrightarrow{d}Z\) with \(P(Z\in C)=1\). Suppose \(g\) is Hadamard directionally differentiable at \(\theta\), tangentially to \(C\), and \(g'_\theta\) is continuous on \(C\). Then \[ a_n\{g(T_n)-g(\theta)\}\xrightarrow{d}g'_\theta(Z). \]

Here is the remainder argument behind the result. Put \(H_n=a_n(T_n-\theta)\) and \(t_n=1/a_n\). For fixed \(M\), consider feasible \(h\in C\) with \(\|h\|\le M\). The difference between the scaled increment and its directional approximation is \[ a_n\{g(\theta+h/a_n)-g(\theta)\}-g'_\theta(h). \] Its supremum in norm over these \(h\) tends to zero. Otherwise, there would be a subsequence and feasible \(h_n\) in the bounded set for which the norm stays bounded away from zero. A further subsequence has \(h_n\to h\in C\), since \(C\) is closed. Hadamard directional differentiability, applied with \(t_n\) and \(h_n\), makes the first term converge to \(g'_\theta(h)\); continuity makes \(g'_\theta(h_n)\to g'_\theta(h)\). Their difference therefore tends to zero, a contradiction. Since \(H_n\) is tight, it lies in a sufficiently large bounded set with probability arbitrarily close to one. Thus the approximation error at \(H_n\) is \(o_p(1)\). Finally, continuity of \(g'_\theta\) and the continuous mapping theorem give the stated limit. \(\square\)

The continuity condition in this version is explicit. In applications with absolute values and maxima it holds because the directional derivative is itself an absolute-value or maximum function.

Worked Example: Absolute Value of a Sample Mean

Let \(X_1,\ldots,X_n\) be independent \(N(\mu,1)\) observations with \(\mu=0\). Find the limit of \(\sqrt n|\bar X_n|\).

The central limit theorem gives \(\sqrt n\bar X_n\xrightarrow{d}Z\), where \(Z\sim N(0,1)\). For \(g(x)=|x|\), the derivative at zero is directional: \[ \frac{g(0+th)-g(0)}{t}=\frac{|th|}{t}=|h|,\qquad t>0. \] Thus \(g'_0(h)=|h|\), which is continuous but not linear. The directional delta method yields \[ \sqrt n|\bar X_n|\xrightarrow{d}|Z|. \] For \(x\ge0\), \(P(|Z|\le x)=P(-x\le Z\le x)=2\Phi(x)-1\); for \(x<0\), the probability is zero. Therefore the limit is a half-normal distribution, not a centered normal distribution. Its density for \(x>0\) is \(2\phi(x)=\sqrt{2/\pi}\exp(-x^2/2)\).

Worked Example: The Maximum of Two Estimates

For each \(n\), let \(\bar X_n\) and \(\bar Y_n\) be sample means from two independent samples, each of size \(n\) from \(N(0,1)\). Find the limit of \(\sqrt n\max(\bar X_n,\bar Y_n)\).

The vector central limit theorem gives \[ \sqrt n(\bar X_n,\bar Y_n)\xrightarrow{d}(Z_1,Z_2), \] where \(Z_1,Z_2\) are independent \(N(0,1)\) variables. For \(g(x,y)=\max(x,y)\), homogeneity gives \(g(th_1,th_2)/t=\max(h_1,h_2)\) for \(t>0\). Hence the directional derivative at \((0,0)\) is \(g'_{(0,0)}(h_1,h_2)=\max(h_1,h_2)\), and \[ \sqrt n\max(\bar X_n,\bar Y_n)\xrightarrow{d}\max(Z_1,Z_2). \] For any real \(x\), independence gives \[ P\{\max(Z_1,Z_2)\le x\}=P(Z_1\le x,\ Z_2\le x)=\Phi(x)^2. \] This limit is generally not normal. The calculation also shows why replacing the maximum by a linear expression would give the wrong limit.

Boundary Constraints and Higher-Order Limits

A boundary affects which local directions an estimator can take. For a parameter space \([0,\infty)\) and boundary point \(\theta=0\), feasible local directions are nonnegative. A transformation may have an ordinary derivative on the whole real line, but the limit still depends on the estimator’s boundary behavior. Separately, if the derivative is zero, a first-order approximation can vanish and the correct rate may be higher.

Second-order delta method: Suppose \(T_n\in\mathbb R^d\), \(\sqrt n(T_n-\theta)\xrightarrow{d}Z\), and, in a neighborhood of \(\theta\), a scalar function \(g\) has the expansion \[ g(\theta+u)=g(\theta)+\nabla g(\theta)^{\mathsf T}u+\frac12u^{\mathsf T}H(\theta)u+o(\|u\|^2), \] where \(H(\theta)\) is the Hessian matrix. If \(\nabla g(\theta)=0\), then \[ n\{g(T_n)-g(\theta)\}\xrightarrow{d}\frac12 Z^{\mathsf T}H(\theta)Z. \]

To verify the scaling, write \(U_n=T_n-\theta\). The assumed convergence makes \(\sqrt nU_n=O_p(1)\), so \(U_n=O_p(n^{-1/2})\). The Taylor remainder satisfies \(n\,o(\|U_n\|^2)=o_p(1)\): for any \(\varepsilon>0\), the remainder’s absolute value is at most \(\varepsilon\|U_n\|^2\) whenever \(U_n\) is sufficiently small, and \(n\|U_n\|^2=O_p(1)\). The linear term vanishes by the zero-gradient condition. The quadratic term becomes \[ \frac12(\sqrt nU_n)^{\mathsf T}H(\theta)(\sqrt nU_n), \] which converges in distribution to \(\frac12Z^{\mathsf T}H(\theta)Z\) by continuity of the quadratic map. Adding the \(o_p(1)\) remainder proves the result. \(\square\)

This statement uses a local expansion on an open neighborhood, or another setting where the displayed expansion is valid for all the estimator’s possible deviations. If the estimator is constrained and the transformation is only defined on the parameter space, check the expansion along feasible deviations rather than silently extending the function.

Worked Example: Squaring a Boundary-Constrained Estimate

Suppose \(X_1,\ldots,X_n\) are independent \(N(\theta,1)\) observations with parameter space \(\theta\ge0\), and the true value is \(\theta=0\). The constrained maximum likelihood estimator is \(\hat\theta_n=\max(0,\bar X_n)\). Find the limit of \(n\hat\theta_n^2\).

Because \(\sqrt n\bar X_n\sim N(0,1)\) for every \(n\), the boundary projection gives \[ \sqrt n\hat\theta_n=\max(0,\sqrt n\bar X_n)\xrightarrow{d}\max(0,Z),\qquad Z\sim N(0,1). \] The transformation is \(g(\theta)=\theta^2\), with \(g'(0)=0\) and \(g''(0)=2\). Squaring the preceding limit, or applying the second-order expansion on the feasible half-line, gives \[ n\{\hat\theta_n^2-0\}\xrightarrow{d}\{\max(0,Z)\}^2. \] For \(x<0\) the limit distribution function is zero. For \(x\ge0\), \[ P\{\max(0,Z)^2\le x\}=P(Z\le\sqrt{x})=\Phi(\sqrt{x}). \] In particular, the limit has probability \(1/2\) at zero and has a positive continuous part above zero. The atom is a direct consequence of the boundary constraint: whenever the unconstrained mean is negative, the estimator is set to zero.

How to Choose the Approximation

A reliable solution begins by identifying the local behavior of the transformation and the estimator, rather than choosing a formula by habit. The following sequence keeps the rate and limiting distribution aligned.

Find the estimator’s rate and limit.
Write \(a_n(T_n-\theta)\xrightarrow{d}Z\), including any dependence between coordinates and any support restrictions on \(Z\).

Inspect the transformation locally.
If \(g\) is differentiable with a nonzero derivative in the relevant directions, use the ordinary first-order approximation. At a kink, calculate the directional derivative. If the first derivative vanishes, examine the next nonzero Taylor term.

Check the domain and scale.
At a boundary, verify that the local deviations are feasible and that the derivative or expansion applies to them. A quadratic leading term typically changes the rate from \(a_n\) to \(a_n^2\) when \(a_n(T_n-\theta)\) has a nondegenerate limit.

Transform the limit and describe its law.
Apply the appropriate continuous map to \(Z\), then check for atoms, support restrictions, or a nonnormal shape before naming the resulting distribution.

Common Mistakes

Using a single slope at a kink. The absolute-value function has different slopes on either side of zero. Its directional derivative at zero is \(|h|\), not \(ch\) for one fixed constant \(c\).
Assuming every transformed limit is normal. A continuous transformation of a Gaussian limit need not be Gaussian. The absolute value and maximum examples produce nonnormal distributions.
Ignoring a boundary atom. A constrained estimator can equal the boundary with positive asymptotic probability. In the squaring example this produces an atom of size \(1/2\) at zero; describing only a continuous density misses part of the limit.
Keeping the first-order rate when its coefficient is zero. If \(\nabla g(\theta)=0\), the first-order limit may be identically zero. Check higher-order terms and rescale accordingly.
Applying a derivative outside its domain. At a boundary, establish that the estimator’s deviations are feasible and that the approximation is valid on those deviations.

These distinctions matter in likelihood problems, constrained estimation, and inference for nonsmooth summaries. A correct limiting law determines which approximations are justified; confusing a kink with a smooth point or overlooking a boundary can change the limit’s shape, support, and point masses.

Check Your Understanding

For each problem, state the relevant local method before calculating or interpreting the limit.

If \(\sqrt nT_n\xrightarrow{d}Z\sim N(0,1)\), what is the limit of \(\sqrt n|T_n|\), and why is the ordinary differentiable delta method at zero unsuitable?
For independent standard normal limits \(Z_1,Z_2\), write the distribution function of \(\max(Z_1,Z_2)\).
What feature of the parameter constraint produces the atom at zero in the boundary-constrained squaring example?
Suppose \(\sqrt n(T_n-\theta)\xrightarrow{d}Z\) and \(g'(\theta)=0\). Under the stated second-order conditions, what rate and limiting expression should be examined?
Why must continuity of the directional derivative be checked when applying the directional delta method as stated here?