Transforming Asymptotic Normal Limits

Why the Transformation Changes the Limit

An estimator may have a simple asymptotic normal limit, while the quantity of scientific interest is a logarithm, power, exponential, or another smooth function of that estimator. The delta method turns the estimator's local fluctuations into the transformed quantity's fluctuations. In practice, the main calculation is often only one derivative, but a correct conclusion also depends on using the right center, rate, and variance.

Suppose $T_n$ estimates a fixed scalar $\theta$, and its fluctuations are of order $n^{-1/2}$. Near $\theta$, a differentiable function $g$ is approximately linear: $g(T_n)-g(\theta)$ is approximately $g'(\theta)(T_n-\theta)$. Thus the transformation multiplies the limiting fluctuation by its local slope. The slope can be negative, so it changes the direction of the fluctuation; the variance, however, involves the square of the slope.

First-order transformation rule: Let $T_n$ be real-valued, let $\theta$ be an interior point of an interval on which $g$ is defined, and suppose $$ \sqrt n(T_n-\theta)\xrightarrow{d}N(\mu,\tau^2), $$ where $\mu$ is finite and $\tau^2\geq 0$. If $g$ is differentiable at $\theta$, then $$ \sqrt n\{g(T_n)-g(\theta)\}\xrightarrow{d}N\bigl(g'(\theta)\mu,\,[g'(\theta)]^2\tau^2\bigr). $$ In particular, when $\mu=0$, the transformed limit is centered at zero.

The rule concerns a local approximation around $\theta$. It requires $T_n$ to be near $\theta$ with probability tending to one; this follows from the stated root-$n$ convergence. If $g$ is defined only near $\theta$, the transformed statistic must be defined on that neighborhood, and any definition assigned outside it has no effect on the limit when the probability of leaving the neighborhood tends to zero.

Why the First-Order Rule Holds

The proof is a Taylor approximation with a remainder small relative to the $n^{-1/2}$ fluctuation. The convergence in distribution of $\sqrt n(T_n-\theta)$ implies that this sequence is bounded in probability. Consequently, $T_n-\theta=O_p(n^{-1/2})$, and in particular $T_n\xrightarrow{p}\theta$.

Derivation: Differentiability at $\theta$ gives $$ g(\theta+h)-g(\theta)=g'(\theta)h+r(h), \qquad \frac{r(h)}{h}\longrightarrow 0 \quad (h\longrightarrow 0), $$ with $r(0)=0$. Substituting $h=T_n-\theta$ yields $$ \sqrt n\{g(T_n)-g(\theta)\} =g'(\theta)\sqrt n(T_n-\theta)+\sqrt n\,r(T_n-\theta). $$ The last term converges to zero in probability. To see this, fix $\varepsilon>0$. Differentiability gives a $\delta>0$ such that $|r(h)|\leq\varepsilon |h|$ when $0<|h|<\delta$. Since $T_n\xrightarrow{p}\theta$, the probability that $|T_n-\theta|\geq\delta$ tends to zero. On the complementary event, $$ \sqrt n\,|r(T_n-\theta)|\leq\varepsilon\sqrt n\,|T_n-\theta|. $$ The sequence on the right, apart from the fixed factor $\varepsilon$, is bounded in probability. Taking $\varepsilon$ arbitrarily small shows that $\sqrt n\,r(T_n-\theta)\xrightarrow{p}0$. Slutsky's theorem now gives the stated normal limit, since multiplication by $g'(\theta)$ changes the mean to $g'(\theta)\mu$ and the variance to $[g'(\theta)]^2\tau^2$. $\square$

This derivation makes two details explicit. The original limit need not be centered at zero, and the transformed variance is the original variance multiplied by the squared derivative. A decreasing transformation reverses the sign of a nonzero limiting mean, but it does not create a negative variance.

Common Smooth Transformations

For $T_n$ fluctuating around a fixed $\theta$, the derivative is evaluated at $\theta$, not at the observed value $T_n$. The following formulas assume $\sqrt n(T_n-\theta)\xrightarrow{d}N(0,\tau^2)$ and that the indicated transformation is differentiable at $\theta$.

Transformation	Derivative at $\theta$	Limit for $\sqrt n\{g(T_n)-g(\theta)\}$	Condition near $\theta$
$g(x)=\log x$	$1/\theta$	$N(0,\tau^2/\theta^2)$	$\theta>0$
$g(x)=x^r$, fixed real $r$	$r\theta^{r-1}$	$N(0,r^2\theta^{2r-2}\tau^2)$	$\theta>0$ for general real $r$
$g(x)=e^x$	$e^\theta$	$N(0,e^{2\theta}\tau^2)$	No restriction on real $\theta$
$g(x)=\sqrt{x}$	$1/(2\sqrt\theta)$	$N(0,\tau^2/(4\theta))$	$\theta>0$

For an integer power, $x^r$ can be defined even at some nonpositive values, but the table's positive-$\theta$ condition gives one unambiguous neighborhood for general real powers, logarithms, and square roots. At $\theta=0$, several derivatives in the table do not exist or are not finite, so this first-order rule does not apply as stated.

Worked Example: The Logarithm of an Exponential Sample Mean

Let $X_1,\ldots,X_n$ be independent exponential random variables with mean $2$, and write $\bar X_n=n^{-1}\sum_{i=1}^n X_i$. Find the first-order limit for $\log\bar X_n$, centered at $\log 2$.

An exponential variable with mean $2$ has variance $2^2=4$. The central limit theorem gives $$ \sqrt n(\bar X_n-2)\xrightarrow{d}N(0,4). $$ Here $g(x)=\log x$, so $g(2)=\log 2$ and $g'(2)=1/2$. The transformed variance is the input variance times the squared derivative: $$ \left(\frac12\right)^2 4=1. $$ Therefore $$ \sqrt n\{\log\bar X_n-\log 2\}\xrightarrow{d}N(0,1). $$ The logarithm is defined because $\bar X_n>0$ with probability one. The result is on the same $\sqrt n$ scale as the sample mean; only the local variance changes.

Worked Example: Squaring a Sample Proportion

Let $Y_1,\ldots,Y_n$ be independent Bernoulli random variables with success probability $p=0.4$, and let $\hat p_n=n^{-1}\sum_{i=1}^nY_i$. Find the limiting distribution of $\sqrt n(\hat p_n^2-p^2)$.

The Bernoulli variance is $p(1-p)=0.4(0.6)=0.24$, so $$ \sqrt n(\hat p_n-0.4)\xrightarrow{d}N(0,0.24). $$ Take $g(x)=x^2$. Its derivative at $p=0.4$ is $g'(0.4)=2(0.4)=0.8$, and $g(0.4)=0.16$. The transformed variance is $$ (0.8)^2(0.24)=0.64(0.24)=0.1536. $$ Hence $$ \sqrt n(\hat p_n^2-0.16)\xrightarrow{d}N(0,0.1536). $$ The square root of the limiting variance is approximately $0.392$, so the transformed estimator's first-order standard deviation is approximately $0.392/\sqrt n$. The variance itself is $0.1536$, not $0.392$.

Worked Example: Exponentiating a Sample Mean

Suppose $X_1,\ldots,X_n$ are independent and identically distributed with mean $\theta=0.3$ and variance $4$, and assume the central limit theorem applies. Find the first-order limit for $e^{\bar X_n}$, centered at $e^{0.3}$.

The central limit theorem supplies $$ \sqrt n(\bar X_n-0.3)\xrightarrow{d}N(0,4). $$ For $g(x)=e^x$, the derivative is $g'(x)=e^x$, giving $g'(0.3)=e^{0.3}$. The transformed variance is $$ (e^{0.3})^2(4)=4e^{0.6}\approx 7.288. $$ Since $g(0.3)=e^{0.3}$, it follows that $$ \sqrt n\{e^{\bar X_n}-e^{0.3}\}\xrightarrow{d}N(0,4e^{0.6}). $$ The exponential function is increasing, so positive fluctuations of $\bar X_n$ produce positive first-order fluctuations of $e^{\bar X_n}$. Its slope at $0.3$, $e^{0.3}$, determines how much those fluctuations are magnified.

Centering Is Part of the Limit

The center in a normal limit is not interchangeable with another nearby quantity. The natural center for $g(T_n)$ in the first-order rule is $g(\theta)$. If instead one subtracts a sequence $c_n$, the difference between $c_n$ and $g(\theta)$ can survive on the $\sqrt n$ scale and shift the limiting mean.

Centering-shift result: Under the assumptions of the first-order transformation rule, suppose also that $c_n$ is deterministic and $$ \sqrt n\{c_n-g(\theta)\}\longrightarrow a $$ for a finite constant $a$. Then $$ \sqrt n\{g(T_n)-c_n\} \xrightarrow{d} N\bigl(g'(\theta)\mu-a,\,[g'(\theta)]^2\tau^2\bigr). $$

To verify this, split the centered quantity exactly: $$ \sqrt n\{g(T_n)-c_n\} =\sqrt n\{g(T_n)-g(\theta)\}-\sqrt n\{c_n-g(\theta)\}. $$ The first term converges in distribution to $N(g'(\theta)\mu,[g'(\theta)]^2\tau^2)$ by the transformation rule. The second term converges to the constant $a$. Slutsky's theorem therefore subtracts $a$ from the limiting mean and leaves the variance unchanged. $\square$

Worked Example: A Root-n Shift in the Center

Return to the exponential sample mean with mean $2$. Instead of centering $\log\bar X_n$ at $\log 2$, center it at $c_n=\log 2+0.5/\sqrt n$. Find the limit.

The unshifted result is $$ \sqrt n\{\log\bar X_n-\log 2\}\xrightarrow{d}N(0,1). $$ The new center satisfies $$ \sqrt n(c_n-\log 2) =\sqrt n\left(\frac{0.5}{\sqrt n}\right)=0.5. $$ Subtracting this root-$n$ shift changes the limiting mean by $-0.5$, not the variance. Thus $$ \sqrt n\{\log\bar X_n-c_n\}\xrightarrow{d}N(-0.5,1). $$ A center that differs by $o(n^{-1/2})$ would give the same limit as $\log 2$. A difference of order $n^{-1/2}$ can change the limiting mean, as it does here.

Common Mistakes and Practical Checks

Before reporting a transformed limit, write down the untransformed limit and identify the proposed center for the transformed statistic. Then evaluate the derivative at the fixed limiting value and square it when computing the variance.

Using the derivative at the random estimate. The first-order limit uses $g'(\theta)$, where $\theta$ is fixed. Substituting $g'(T_n)$ into the limiting variance confuses a random finite-sample quantity with the local slope that determines the limit.
Forgetting to square the derivative. The fluctuation is multiplied by $g'(\theta)$, so its variance is multiplied by $[g'(\theta)]^2$. A negative derivative cannot produce a negative variance.
Changing the center without checking its scale. Replacing $g(\theta)$ by $c_n$ is harmless for this limit only if $\sqrt n\{c_n-g(\theta)\}\to 0$. A nonzero limit shifts the asymptotic mean.
Applying the rule where the derivative fails. The logarithm at $\theta=0$ and the square root at $\theta=0$ do not have finite derivatives there. The first-order formula cannot be justified by simply inserting those values into the derivative formulas.
Confusing the limiting variance with the standard deviation. If the limit is $N(0,v)$, its standard deviation is $\sqrt v$. Keep the second argument of $N$ as a variance.

Key takeaway: For a smooth scalar transformation, the first-order normal limit is determined by three explicit choices: the input center $\theta$, the transformed center $g(\theta)$, and the local slope $g'(\theta)$. The limiting variance is the input variance multiplied by the squared slope, and any center shifted on the $n^{-1/2}$ scale must be accounted for separately.

Check Your Understanding

For each question, identify the center, derivative, and variance calculation before stating a limit.

If $\sqrt n(T_n-3)\xrightarrow{d}N(0,9)$, what is the limit of $\sqrt n\{\log T_n-\log 3\}$?
If $\sqrt n(T_n-2)\xrightarrow{d}N(1,4)$, what mean and variance does the first-order rule give for $\sqrt n(e^{T_n}-e^2)$?
For a positive $\theta$, state the first-order limiting variance for $\sqrt n(T_n^r-\theta^r)$ when $\sqrt n(T_n-\theta)\xrightarrow{d}N(0,\tau^2)$.
If $\sqrt n\{c_n-g(\theta)\}\to a$, how does centering at $c_n$ instead of $g(\theta)$ affect the limiting mean?
Explain why a negative derivative can reverse the sign of a limiting mean without making the limiting variance negative.

Transformation	Derivative at \(\theta\)	Limit for \(\sqrt n\{g(T_n)-g(\theta)\}\)	Condition near \(\theta\)
\(g(x)=\log x\)	\(1/\theta\)	\(N(0,\tau^2/\theta^2)\)	\(\theta>0\)
\(g(x)=x^r\), fixed real \(r\)	\(r\theta^{r-1}\)	\(N(0,r^2\theta^{2r-2}\tau^2)\)	\(\theta>0\) for general real \(r\)
\(g(x)=e^x\)	\(e^\theta\)	\(N(0,e^{2\theta}\tau^2)\)	No restriction on real \(\theta\)
\(g(x)=\sqrt{x}\)	\(1/(2\sqrt\theta)\)	\(N(0,\tau^2/(4\theta))\)	\(\theta>0\)