Why Kinks Change the Limit
The first-order delta method gives a normal limit when a smooth transformation acts on an estimator with an asymptotically normal limit. But some important transformations are not differentiable everywhere. The absolute value has a kink at zero, and the maximum of two coordinates has a kink where the coordinates are equal. At these points, small estimation errors can arrive from either side, and the transformed error need not be a linear function of the original error.
The consequences are visible even when the estimator itself has a simple normal limit. At a kink, the scaled transformed estimator may converge to an absolute value or maximum of Gaussian variables, rather than to a normal variable. The key is to keep the transformation intact and examine how it acts on the scaled error.
Absolute Value at Zero
Suppose \(T_n\) estimates a real parameter whose value is zero, and for a positive sequence \(r_n\) tending to infinity, \(r_n T_n\xrightarrow{d} Z\). Because absolute value is positively homogeneous, scaling and transforming commute exactly: \(r_n|T_n|=|r_nT_n|\). No Taylor expansion is needed.
Proof. The map \(x\mapsto |x|\) is continuous on the real line. The continuous mapping theorem therefore gives \(|r_nT_n|\xrightarrow{d}|Z|\). Since \(r_n>0\), \(|r_nT_n|=r_n|T_n|\) for every \(n\), proving the claim. \(\square\)
When \(Z\sim N(0,\sigma^2)\) with \(\sigma>0\), the limit \(|Z|\) is a folded normal variable (in this centered case, a half-normal variable). Its distribution function is \(P(|Z|\le t)=2\Phi(t/\sigma)-1\) for \(t\ge 0\), and is zero for \(t<0\), where \(\Phi\) is the standard normal distribution function. This is not a normal distribution: it is nonnegative and has positive probability below every positive threshold, whereas a nondegenerate normal variable also takes negative values.
Worked Example: The Absolute Value of a Sample Mean at Zero
Let \(X_1,\ldots,X_n\) be independent \(N(0,4)\) observations, and let \(\bar X_n=n^{-1}\sum_{i=1}^n X_i\). Find the limit of \(\sqrt n|\bar X_n|\).
The sample mean is exactly normal, with \(\bar X_n\sim N(0,4/n)\). Therefore \(\sqrt n\,\bar X_n\sim N(0,4)\) for every \(n\). Taking absolute values gives \(\sqrt n|\bar X_n|\sim |N(0,4)|\) for every \(n\), not just asymptotically. Equivalently, the theorem applies with \(r_n=\sqrt n\) and \(Z\sim N(0,4)\), so the limiting distribution is \(|Z|\).
For example, at \(t=2\), the limiting probability of being at most \(2\) is \(P(|Z|\le 2)=P(-2\le Z\le 2)=\Phi(1)-\Phi(-1)=2\Phi(1)-1\), approximately \(0.683\). The limit has no negative values. Applying a normal approximation directly to the transformed error would miss this feature.
A Maximum at a Tie
The maximum has a parallel exact identity. Suppose two estimators \(T_{n1}\) and \(T_{n2}\) both estimate the same value \(\theta\), and their scaled errors converge jointly: \[ \sqrt n \begin{pmatrix} T_{n1}-\theta\\ T_{n2}-\theta \end{pmatrix} \xrightarrow{d} \begin{pmatrix} Z_1\\ Z_2 \end{pmatrix}. \] Subtracting the common center and using the positive homogeneity of the maximum gives \(\sqrt n\{\max(T_{n1},T_{n2})-\theta\}=\max\{\sqrt n(T_{n1}-\theta),\sqrt n(T_{n2}-\theta)\}\). The maximum map is continuous, so the joint limit determines the transformed limit.
Proof. Define \(M(x,y)=\max(x,y)\). For any two points \((x,y)\) and \((u,v)\), \(|M(x,y)-M(u,v)|\le \max(|x-u|,|y-v|)\), so \(M\) is continuous. By the continuous mapping theorem, applying \(M\) to the jointly convergent vector of scaled errors gives convergence in distribution to \(M(Z_1,Z_2)\). The exact identity above identifies the transformed statistic with that map for every \(n\). Hence its limit is \(\max(Z_1,Z_2)\). \(\square\)
Worked Example: The Maximum of Two Tied Sample Means
Let \(X_1,\ldots,X_n\) and \(Y_1,\ldots,Y_n\) be two independent samples, each consisting of independent \(N(\theta,1)\) observations. Write their sample means as \(\bar X_n\) and \(\bar Y_n\). Find the limit of \(\sqrt n\{\max(\bar X_n,\bar Y_n)-\theta\}\).
For every \(n\), \(\sqrt n(\bar X_n-\theta)\) and \(\sqrt n(\bar Y_n-\theta)\) are independent \(N(0,1)\) variables. Their joint distribution is therefore that of independent \(Z_1,Z_2\sim N(0,1)\). The theorem gives the limit \(\max(Z_1,Z_2)\).
For any real \(x\), independence yields \(P\{\max(Z_1,Z_2)\le x\}=P(Z_1\le x,Z_2\le x)=\Phi(x)^2\). In particular, the probability that the limit is at most zero is \(\Phi(0)^2=(1/2)^2=1/4\). The limit is not normal: it has this asymmetric distribution function rather than a normal distribution function.
If the Gaussian limits are correlated, the same theorem still applies; only the distribution calculation changes. For a jointly normal pair with standard normal marginal distributions and correlation \(\rho\), the maximum has distribution function \(P(Z_1\le x,Z_2\le x)\), the bivariate normal probability at \((x,x)\) with correlation \(\rho\). Independence is not needed for the convergence result, but it is needed for the product \(\Phi(x)^2\) in the example.
When the Maximum Is Unique
A tie is essential to the nonsmooth limit above. If one population value is strictly larger than the other, then sufficiently small estimation errors are unlikely to change which coordinate is larger. The maximum then behaves asymptotically like the estimator of the uniquely larger value.
Worked Example: A Strictly Larger Population Value
Let \(\bar X_n\) and \(\bar Y_n\) be independent sample means from samples of \(N(2,1)\) and \(N(1,1)\) observations, respectively, with sample size \(n\) in each. Determine the limit of \(\sqrt n\{\max(\bar X_n,\bar Y_n)-2\}\).
The difference \(\bar Y_n-\bar X_n\) is normal with mean \(-1\) and variance \(2/n\). Thus the probability that the second estimate exceeds the first is \[ P(\bar Y_n>\bar X_n) =P\left(N(0,1)>\sqrt{n/2}\right) =1-\Phi\left(\sqrt{n/2}\right)\longrightarrow 0. \] With probability tending to one, the maximum is \(\bar X_n\). On that event, \(\sqrt n\{\max(\bar X_n,\bar Y_n)-2\}=\sqrt n(\bar X_n-2)\), which has distribution \(N(0,1)\). The two scaled quantities are equal with probability tending to one, so they have the same distributional limit. Consequently, \(\sqrt n\{\max(\bar X_n,\bar Y_n)-2\}\xrightarrow{d}N(0,1)\).
This differs from the tied case: with a strict gap, the limiting error is just the Gaussian error of the winning coordinate. At a tie, either coordinate can determine the maximum, producing a maximum of Gaussian errors instead.
Absolute Value Away from Zero
The same contrast appears for absolute value. If the target value \(\theta\) is nonzero and \(\sqrt n(T_n-\theta)\xrightarrow{d}Z\), then the probability that \(T_n\) has the same sign as \(\theta\) tends to one. Indeed, \(T_n-\theta\xrightarrow{p}0\), so \(T_n\xrightarrow{p}\theta\); because \(\theta\ne0\), a neighborhood of \(\theta\) has a constant sign. On that neighborhood, absolute value is linear: it equals \(x\) if \(\theta>0\), and \(-x\) if \(\theta<0\). It follows that \[ \sqrt n\{|T_n|-|\theta|\}\xrightarrow{d} \begin{cases} Z,&\theta>0,\\ -Z,&\theta<0. \end{cases} \] Thus it is the location of the target—not simply the presence of an absolute value in the formula—that determines whether the kink matters.
Worked Example: Absolute Value at a Nonzero Target
Suppose \(T_n\sim N(3,4/n)\). Then \(\sqrt n(T_n-3)\sim N(0,4)\). Since the target \(3\) is positive, the absolute-value transformation is locally the identity. In fact, \(P(T_n<0)=\Phi(-3\sqrt n/2)\to0\). On the event \(T_n\ge0\), \(\sqrt n(|T_n|-3)=\sqrt n(T_n-3)\). Therefore \(\sqrt n(|T_n|-3)\xrightarrow{d}N(0,4)\), rather than a half-normal limit.
Common Mistake and Practical Implications
A common mistake is to apply the ordinary delta method at a kink by assigning it a convenient derivative. At zero, one might choose derivative \(1\) for \(g(x)=|x|\), and predict the limit \(Z\); choosing \(-1\) predicts \(-Z\), while choosing \(0\) predicts a degenerate limit. None is justified: the derivative does not exist there. The actual limit is \(|Z|\), as the exact scaling identity shows. Similarly, at a tie, selecting either coordinate's gradient for \(h(x,y)=\max(x,y)\) would predict \(Z_1\) or \(Z_2\), but the limit is \(\max(Z_1,Z_2)\).
These distinctions matter for asymptotic inference. A normal approximation that ignores the kink can assign probability to impossible negative values for an absolute-value estimator, or use the wrong tail probabilities for a maximum at a tie. Before using a differentiability-based result, check whether the target point lies at a kink. At a kink, first identify the joint or scalar limit of the scaled estimation error, then apply the original transformation to that limit when an exact scaling identity and continuity justify doing so.
Check Your Understanding
Use the stated assumptions in each question, and distinguish carefully between a tie and a strict gap.
- If \(r_nT_n\xrightarrow{d}Z\) and \(r_n>0\), what is the limit of \(r_n|T_n|\), and which theorem justifies the transformation?
- If \(Z\sim N(0,1)\), find the distribution function of \(|Z|\) for \(t\ge0\). Why is this not a normal distribution?
- Suppose two estimators of the same value have joint scaled limit \((Z_1,Z_2)\). What is the limit of their scaled maximum after centering at that common value?
- For independent standard normal \(Z_1,Z_2\), calculate \(P\{\max(Z_1,Z_2)\le x\}\).
- Why does a strictly larger population value lead to a different maximum limit from a tie?
- What is wrong with using a gradient of the maximum function at a point where its two arguments are equal?