Why the Center and Rate Must Match
A delta-method calculation has two choices that must fit the sequence being transformed: the center to subtract and the rate at which to normalize. If a statistic fluctuates around a sequence of centers \(\theta_n\), centering instead at its limit \(\theta\) can add a shift that remains visible after normalization. Likewise, the rate that gives a finite limit for the original statistic is usually the rate to use for its first-order transformation.
The main task here is not to expand a function mechanically. It is to identify the sequence’s actual asymptotic centering, retain its rate through the transformation, and account for any deterministic displacement introduced by changing centers.
Fixed-Center and Moving-Center Limits
The fixed-center formulation is the special case \(\theta_n=\theta\). The theorem uses the same rate \(a_n\) as the input limit. Its hypotheses ensure that both the original and transformed quantities are real-valued; the condition \(X_n\in I\) almost surely is not merely a condition that holds with high probability.
Here is the proof. Convergence in distribution of \(a_n(X_n-\theta_n)\) implies that this sequence is tight, so \(a_n(X_n-\theta_n)=O_p(1)\). As \(a_n\to\infty\), it follows that \(X_n-\theta_n=o_p(1)\). Together with \(\theta_n\to\theta\), this gives \(X_n\xrightarrow{p}\theta\).
By the mean value theorem, when \(X_n\ne\theta_n\) there is a point \(\xi_n\) between them such that \[ g(X_n)-g(\theta_n)=g'(\xi_n)(X_n-\theta_n). \] When \(X_n=\theta_n\), the same equality holds by setting the coefficient equal to \(g'(\theta_n)\). In either case, the coefficient \(Q_n\) in this equality satisfies \(Q_n\xrightarrow{p}g'(\theta)\): the point at which the derivative is evaluated lies between two quantities converging in probability to \(\theta\), and \(g'\) is continuous there. Therefore \[ a_n\{g(X_n)-g(\theta_n)\} =Q_n\,a_n(X_n-\theta_n) \xrightarrow{d}g'(\theta)Z \] by Slutsky’s theorem. This proves the result. \(\square\)
Changing from the Moving Center to the Limiting Center
A common target is not \(g(\theta_n)\), but the fixed value \(g(\theta)\). The difference between these choices is deterministic, and it must be evaluated at the same rate as the random fluctuation.
Indeed, the exact decomposition is \[ a_n\{g(X_n)-g(\theta)\} =a_n\{g(X_n)-g(\theta_n)\} +a_n\{g(\theta_n)-g(\theta)\}. \] The first term converges in distribution to \(g'(\theta)Z\), and the second converges to \(b\). Slutsky’s theorem gives the stated limit. In particular, if \(\theta_n=\theta+h/a_n\) for a fixed \(h\), differentiability at \(\theta\) gives \(a_n\{g(\theta_n)-g(\theta)\}\to g'(\theta)h\). Thus a shift in the original center of order \(1/a_n\) can produce a nonzero shift in the transformed limit.
If the scaled center difference tends to zero, the two transformed centers give the same limit. If it tends to a nonzero finite value, the limit is shifted. If it does not have a finite limit, centering at \(g(\theta)\) need not produce the same finite limiting distribution. These possibilities are why the centering sequence should be identified before simplifying the statistic.
Worked Examples
Worked Example: A Fixed Center for an Exponential Transformation
Let \(Z\sim N(0,4)\), and set \(X_n=1+Z/\sqrt n\). Find the limit of \(\sqrt n\{\exp(X_n)-\exp(1)\}\).
The input has the fixed center \(\theta=1\), and \[ \sqrt n(X_n-1)=Z \] for every \(n\). Take \(g(x)=\exp(x)\), which is differentiable on all of \(\mathbb R\), with \(g'(1)=e\). The fixed-center theorem, with \(a_n=\sqrt n\), gives \[ \sqrt n\{\exp(X_n)-e\}\xrightarrow{d}eZ. \] Since \(Z\sim N(0,4)\), scaling by \(e\) yields \(eZ\sim N(0,4e^2)\). The transformed limit is centered at \(e\), the image of the input center, and the rate remains \(\sqrt n\).
Worked Example: A Moving Center and Its Fixed-Center Shift
Let \(U\sim\operatorname{Uniform}(-1,1)\), \(\theta_n=2+1/\sqrt n\), and \(X_n=\theta_n+U/\sqrt n\). Compare the limits obtained by centering \(\log(X_n)\) at \(\log(\theta_n)\) and at \(\log(2)\).
First check the domain. Since \(U>-1\) almost surely, \[ X_n=2+\frac{1+U}{\sqrt n}>2 \] almost surely, so both logarithms are defined for every \(n\). Also, \(\theta_n\to2\) and \(\sqrt n(X_n-\theta_n)=U\). Apply the moving-center theorem to \(g(x)=\log x\), for which \(g'(2)=1/2\). It gives \[ \sqrt n\{\log(X_n)-\log(\theta_n)\} \xrightarrow{d}\frac{U}{2}. \] For the fixed center, decompose the statistic: \[ \sqrt n\{\log(X_n)-\log(2)\} =\sqrt n\{\log(X_n)-\log(\theta_n)\} +\sqrt n\log\left(1+\frac{1}{2\sqrt n}\right). \] The deterministic term tends to \(1/2\), since \(\log(1+x)/x\to1\) as \(x\to0\). Consequently, \[ \sqrt n\{\log(X_n)-\log(2)\} \xrightarrow{d}\frac{U}{2}+\frac12. \] The moving-center limit is uniform on \((-1/2,1/2)\), while the fixed-center limit is that distribution shifted by \(1/2\). The shift comes from the centers, not from a change in the random fluctuation.
Worked Example: A Faster Rate and a Nonzero Center Shift
Let \(Z\sim N(0,1)\), \(\theta_n=3+1/n\), and \(X_n=\theta_n+Z/n\). Find the limits for the exponential transformation when centered at \(\exp(\theta_n)\) and at \(\exp(3)\), using rate \(n\).
The input fluctuation is exactly \(n(X_n-\theta_n)=Z\), so the relevant rate is \(n\), not \(\sqrt n\). The function \(g(x)=\exp(x)\) is defined for every real input, including all possible values of \(X_n\); thus the transformed statistic is real-valued almost surely. Since \(\theta_n\to3\) and \(g'(3)=e^3\), the moving-center theorem gives \[ n\{\exp(X_n)-\exp(\theta_n)\}\xrightarrow{d}e^3Z \sim N(0,e^6). \] For the fixed center, the deterministic displacement is \[ n\{\exp(\theta_n)-\exp(3)\} =e^3 n\{\exp(1/n)-1\}\longrightarrow e^3, \] using \((e^x-1)/x\to1\) as \(x\to0\). Therefore \[ n\{\exp(X_n)-\exp(3)\}\xrightarrow{d}e^3Z+e^3 \sim N(e^3,e^6). \] The rate \(n\) preserves the input’s nondegenerate fluctuation, and the difference between the two limits is exactly the scaled center shift.
Choosing the Rate and Avoiding Common Errors
The rate is selected from the input limit. If \(a_n(X_n-\theta_n)\) has a nondegenerate limit and \(g'(\theta)\ne0\), the first-order transformed fluctuation is naturally normalized by \(a_n\) as well. The theorem gives the limit at that rate; it does not say that every other rate is suitable. A rate much larger than \(a_n\) can magnify the fluctuation without bound, while a much smaller rate can make it vanish. When \(g'(\theta)=0\), this first-order theorem gives a degenerate limit and does not identify a different nondegenerate rate; that case requires higher-order analysis.
- Wrong: Since \(\theta_n\to\theta\), replace \(g(\theta_n)\) by \(g(\theta)\) without further calculation. Correction: Check \(a_n\{g(\theta_n)-g(\theta)\}\); it may converge to a nonzero shift.
- Wrong: Change the normalization rate just because the transformed quantity looks numerically smaller or larger. Correction: Begin with the rate in the input convergence and use the theorem’s first-order conclusion.
- Wrong: Treat an expression as a real-valued random variable merely because it is defined with probability tending to one. For example, a logarithm may fail to be real on an event of positive probability at every finite \(n\). Correction: Ensure the argument is in the function’s domain almost surely, or explicitly define a real-valued extension on the exceptional event and analyze that extension.
The last correction matters even when the exceptional event’s probability tends to zero. A limit theorem concerns a sequence of well-defined random variables. If a statistic is undefined on an event of positive probability, it must first be given a valid definition there; only then can a negligible-probability argument justify why the extension does not change its asymptotic distribution.
Check Your Understanding
For each question, identify the center and rate before applying a transformation.
- If \(a_n(X_n-\theta)\xrightarrow{d}Z\), where \(a_n\to\infty\), what additional regularity condition on \(g\) at \(\theta\) supports the fixed-center first-order limit?
- Suppose \(a_n(X_n-\theta_n)\xrightarrow{d}Z\) and \(a_n\{g(\theta_n)-g(\theta)\}\to b\). What is the limit when the transformed statistic is centered at \(g(\theta)\)?
- If \(\theta_n=\theta+h/a_n\), what is the limiting deterministic center shift under differentiability of \(g\) at \(\theta\)?
- Why is convergence of the probability that \(\log(X_n)\) is defined to one not, by itself, enough to define \(\log(X_n)\) as a real-valued random variable for each \(n\)?
- When \(g'(\theta)=0\), what does the first-order moving-center theorem conclude, and what does it not establish about a faster rate?