Regularity Under Local Parameter Changes

Why Pointwise Behavior Is Not Enough

An estimator can have a simple limiting distribution when the parameter is held fixed, yet behave differently when the parameter moves toward that fixed value as the sample size grows. Shifts of order $n^{-1/2}$ are important because they are on the same scale as many first-order estimation errors. First-order regularity asks whether the estimator’s limiting error distribution remains stable under these local changes.

This is a distributional requirement, not merely a statement about a variance. A pointwise asymptotic variance describes behavior at a fixed parameter value. By itself, it does not say what happens under nearby parameter sequences, nor does it rule out an estimator whose apparent precision depends sharply on the exact parameter value.

Definition of First-Order Regularity

Let $X_1,\ldots,X_n$ be observations from a model indexed by a scalar parameter $\theta$ in an open set $\Theta$, and let $\hat{\theta}_n$ estimate $\theta$. Fix an interior point $\theta$. For any fixed real number $h$, define the local parameter value $\theta_{n,h}=\theta+h/\sqrt n$. Since $\theta$ is interior, $\theta_{n,h}\in\Theta$ for all sufficiently large $n$.

Definition: The estimator $\hat{\theta}_n$ is first-order regular at $\theta$ if there is a probability distribution $Q_\theta$, not depending on the fixed local shift $h$, such that, for every fixed $h$, $$ \sqrt n\bigl(\hat{\theta}_n-\theta_{n,h}\bigr) \xrightarrow{d} Q_\theta \quad\text{under }P_{\theta_{n,h}}. $$ Here convergence “under $P_{\theta_{n,h}}$” means that the observations in the $n$-sample have the model distribution at parameter $\theta_{n,h}$. The same limiting distribution must apply to every fixed $h$, including $h=0$.

The centering in this definition matters: under the local model, the target is $\theta_{n,h}$, not the base value $\theta$. The identity

\sqrt n(\hat{\theta}_n-\theta) = \sqrt n(\hat{\theta}_n-\theta_{n,h})+h

shows that regularity is equivalently expressed by convergence of the base-centered error to the distribution $Q_\theta$ shifted by $h$. Thus, under the local alternative, the base-centered limit should move by exactly the change in the target; the estimator’s centered error distribution should not acquire an additional dependence on $h$.

Worked Examples of Local Behavior

Worked Example: The Normal Sample Mean Is Regular

Suppose $X_1,\ldots,X_n$ are independent $N(\theta,\sigma^2)$ observations, where $\sigma^2>0$ is known, and take $\hat{\theta}_n=\bar X$. Under the local parameter $\theta_{n,h}=\theta+h/\sqrt n$, the sample mean has the exact distribution

\bar X\sim N\left(\theta_{n,h},\frac{\sigma^2}{n}\right).

Consequently, for every $n$ and every fixed $h$,

\sqrt n(\bar X-\theta_{n,h})\sim N(0,\sigma^2).

The centered distribution is already the same for all $h$, so the sample mean is regular at $\theta$ with $Q_\theta=N(0,\sigma^2)$. If instead the error is centered at the base value, then $\sqrt n(\bar X-\theta)\sim N(h,\sigma^2)$, consistent with the required shift by $h$.

Worked Example: A Bernoulli Sample Mean Is Regular at an Interior Parameter

Let $X_1,\ldots,X_n$ be independent Bernoulli observations with success probability $p$, where $0<p<1$, and estimate $p$ by $\hat p_n=\bar X$. For a fixed $h$, set $p_n=p+h/\sqrt n$. For all sufficiently large $n$, $p_n\in(0,1)$, and under this local model,

E_{p_n}[X_i]=p_n,\qquad \operatorname{Var}_{p_n}(X_i)=p_n(1-p_n).

The variance converges to $p(1-p)>0$. The centered summands $X_i-p_n$ are bounded in absolute value by 1, so for every $\varepsilon>0$, they satisfy the Lindeberg condition for all sufficiently large $n$: $|X_i-p_n|\leq 1<\varepsilon\sqrt n$, making the relevant tail indicator zero. The triangular-array central limit theorem therefore gives

\sqrt n(\bar X-p_n)\xrightarrow{d}N\bigl(0,p(1-p)\bigr).

The limit is the same for each fixed $h$, so $\bar X$ is regular at every interior $p$, with $Q_p=N(0,p(1-p))$. This argument is local to interior values: it does not assert regularity at a boundary such as $p=0$, where two-sided shifts of the stated form need not remain in the parameter space.

Worked Example: The Hodges Estimator Fails the Local Test

Suppose $X_i$ are independent $N(\theta,1)$ observations. Define the Hodges estimator by setting $\hat{\theta}_n=0$ when $|\bar X|\leq n^{-1/4}$, and setting $\hat{\theta}_n=\bar X$ otherwise. At the fixed value $\theta=0$, $\sqrt n\bar X\sim N(0,1)$. Hence

P_0\bigl(|\bar X|>n^{-1/4}\bigr) = P\bigl(|Z|>n^{1/4}\bigr)\longrightarrow 0, \qquad Z\sim N(0,1).

Thus the estimator equals zero with probability tending to one at $\theta=0$, and $\sqrt n\hat{\theta}_n\xrightarrow{p}0$. Its pointwise limiting distribution at zero is degenerate.

Now use the local parameter $\theta_{n,h}=h/\sqrt n$. Under this parameter, $\sqrt n\bar X$ has the $N(h,1)$ distribution. The estimator uses the sample mean only if

|\bar X|>n^{-1/4} \quad\Longleftrightarrow\quad |\sqrt n\bar X|>n^{1/4}.

Since $\sqrt n\bar X$ has the fixed $N(h,1)$ distribution, the probability of this event tends to zero for each fixed $h$. Therefore $\hat{\theta}_n=0$ with probability tending to one, and

\sqrt n(\hat{\theta}_n-\theta_{n,h}) \xrightarrow{d}\delta_{-h},

where $\delta_{-h}$ denotes a point mass at $-h$. For $h=0$ the limit is a point mass at zero; for $h\ne0$ it is a point mass at $-h$. These limits differ, so the estimator is not regular at zero. Its unusually concentrated pointwise limit does not describe its behavior uniformly across $n^{-1/2}$-scale alternatives.

A Sufficient Condition and Its Proof

A common way to establish regularity is to find an asymptotic linear representation under each local model. The following condition makes explicit a detail that cannot be omitted: the limiting variance must be the same constant for every fixed local shift.

Theorem: Fix an interior parameter value $\theta$. Suppose that for each fixed $h$, under $P_{\theta_{n,h}}$, the estimator has the representation $$ \sqrt n(\hat{\theta}_n-\theta_{n,h}) = \frac{1}{\sqrt n}\sum_{i=1}^n \psi_{\theta_{n,h}}(X_i)+r_{n,h}, \qquad r_{n,h}\xrightarrow{p}0. $$ Assume that within each row the $X_i$ are independent and identically distributed under $P_{\theta_{n,h}}$, and that $E_{\theta_{n,h}}[\psi_{\theta_{n,h}}(X_i)]=0$. Write $v_{n,h}=E_{\theta_{n,h}}[\psi_{\theta_{n,h}}(X_i)^2]$. Suppose, for every fixed $h$, that $v_{n,h}\to V$ for one and the same constant $V>0$, independent of $h$, and that the triangular-array Lindeberg condition holds: $$ \frac{1}{n v_{n,h}}\sum_{i=1}^n E_{\theta_{n,h}}\!\left[ \psi_{\theta_{n,h}}(X_i)^2 \mathbf 1\left\{ |\psi_{\theta_{n,h}}(X_i)|>\varepsilon\sqrt{n v_{n,h}} \right\} \right]\longrightarrow 0 $$ for every $\varepsilon>0$ and each fixed $h$. Then $\hat{\theta}_n$ is first-order regular at $\theta$, with $Q_\theta=N(0,V)$.

Proof. Fix any $h$. By the stated Lindeberg condition and the triangular-array central limit theorem, the normalized sum of the centered terms converges in distribution to $N(0,1)$. Since $v_{n,h}\to V>0$, multiplying that normalized sum by $\sqrt{v_{n,h}}$ gives

\frac{1}{\sqrt n}\sum_{i=1}^n\psi_{\theta_{n,h}}(X_i) \xrightarrow{d}N(0,V) \quad\text{under }P_{\theta_{n,h}}.

The remainder converges in probability to zero under the same local model. Slutsky’s theorem therefore implies

\sqrt n(\hat{\theta}_n-\theta_{n,h}) \xrightarrow{d}N(0,V) \quad\text{under }P_{\theta_{n,h}}.

The argument applies to every fixed $h$, and the assumed limit $V$ is common to all such $h$. Thus the limiting distribution is the same $N(0,V)$ for every local shift, which is exactly first-order regularity. $\square$

The common-limit requirement is essential. Having a central limit theorem separately for each $h$ is not enough if its variance is allowed to depend on $h$.

Worked Example: Pointwise Variance Can Change Under Local Shifts

Consider a model $X_i\sim N(\eta,v(\eta))$, with independent observations and $v(0)=1$, while $v(\eta)=2$ for every $\eta\ne0$. Use $\bar X$ to estimate $\eta$. At the fixed parameter $\eta=0$,

\sqrt n(\bar X-0)\sim N(0,1)

for every $n$. Thus the pointwise asymptotic variance is 1. Under a local parameter $\eta_{n,h}=h/\sqrt n$, however, $h=0$ gives the same $N(0,1)$ limit, whereas every fixed $h\ne0$ has $\eta_{n,h}\ne0$ and

\sqrt n(\bar X-\eta_{n,h})\sim N(0,2)

for every $n$. The centered limits depend on whether $h=0$. Therefore the sample mean is not regular at zero in this model, despite having a well-defined pointwise asymptotic variance there. In the sufficient condition above, the corresponding variance limit is 1 for $h=0$ and 2 for $h\ne0$; it is not one common $V$.

What Regularity Does—and Does Not—Tell You

First-order regularity is useful when comparing estimators at a fixed parameter value: it rules out conclusions based only on behavior at that exact point when the estimator changes sharply under nearby alternatives. It also supplies the local stability condition needed in first-order efficiency arguments. The definition itself does not give an efficiency bound or say which regular estimator is best; those are separate questions.

Regularity does not require a normal limiting distribution. The definition permits any probability distribution $Q_\theta$, provided the same one arises for every fixed local shift. Conversely, a normal limit at the base parameter alone does not prove regularity: the centered limit may change under local alternatives, as the variance-changing normal model demonstrates.

Common pitfall: “The estimator has asymptotic variance $V$ at $\theta$, so it is regular.” This reasoning checks only the fixed-parameter limit. To establish regularity, analyze $\sqrt n(\hat{\theta}_n-\theta_{n,h})$ under $P_{\theta_{n,h}}$ for every fixed $h$, and verify that its limiting distribution is the same for all $h$. A separate limit for each shift, with a shift-dependent variance or shape, is not sufficient.

Check Your Understanding

Use the definition and local centering carefully in each answer.

State the definition of first-order regularity at an interior scalar parameter value.
If $\sqrt n(\hat{\theta}_n-\theta_{n,h})$ has a common limit $Q_\theta$, what is the corresponding limiting distribution of $\sqrt n(\hat{\theta}_n-\theta)$ under the local model?
Why does a pointwise asymptotic variance at $h=0$ fail to establish regularity?
For the Hodges estimator in the example, what is the limiting distribution of the locally centered error when $h\ne0$, and why does this violate regularity?
In the sufficient-condition theorem, why must $v_{n,h}$ converge to the same $V$ for every fixed $h$?