Uniform Asymptotic Linear Representations

Why Uniformity Matters

An asymptotic expansion can describe an estimator accurately when the sample size is large under each fixed data-generating process. But in applications, the process generating the data may not be known, and it may change with the sample size. A statement that works eventually for every fixed process need not work reliably across a whole class of processes at once. Uniform asymptotic linearity formalizes the stronger requirement: the expansion's error must become negligible uniformly over the class.

This distinction matters whenever an approximation is used to support inference across a range of plausible distributions or parameter values. A remainder that is small for each fixed process, but large for some process in the class at every sample size, does not support a single large-sample guarantee for that class.

Definition and Notation

For each sample size $n$, suppose the observed data $Z_1,\ldots,Z_n$ are independent and identically distributed under some $P$ in a class $\mathcal P$. The estimator is $\hat\theta_n$, the target under $P$ is $\theta(P)$, and the influence function is $\psi_P(Z)$, which may depend on $P$. For vector-valued quantities, use a fixed norm, such as the Euclidean norm.

Definition: The estimator has a uniform asymptotic linear representation over $\mathcal P$ if $$ \sqrt n\{\hat\theta_n-\theta(P)\} = \frac{1}{\sqrt n}\sum_{i=1}^n \psi_P(Z_i)+r_{n,P}, $$ where $E_P[\psi_P(Z_i)]=0$ and, for every $\varepsilon>0$, $$ \sup_{P\in\mathcal P}P\bigl(\|r_{n,P}\|>\varepsilon\bigr)\longrightarrow 0. $$ The last condition is written $r_{n,P}=o_{\mathcal P}(1)$. It requires the remainder to vanish in probability uniformly over the class.

The expansion separates the scaled estimation error into a sum of mean-zero contributions and a remainder. The influence function describes the leading random variation. It is often useful to impose a uniform bound on its second moments, but that bound is not part of the definition itself. Nor does the definition alone imply that the leading sum is approximately normal.

The supremum in the remainder condition is essential. It means that for any fixed tolerance $\varepsilon$, the largest probability of an error greater than $\varepsilon$, taken over all $P\in\mathcal P$, goes to zero. Equivalently, the condition controls sequences $P_n\in\mathcal P$ that are allowed to change with $n$. This is the feature that connects uniform expansions to triangular-array reasoning.

Worked Examples

Worked Example: The Sample Mean Has an Exact Uniform Representation

Suppose $X_1,\ldots,X_n$ are independent and identically distributed under $P$, with mean $\mu(P)$, and estimate $\mu(P)$ by $\bar X$. Consider a class $\mathcal P$ of distributions with finite means. The influence function is $\psi_P(X)=X-\mu(P)$. The representation is exact: $$ \sqrt n\{\bar X-\mu(P)\} = \frac{1}{\sqrt n}\sum_{i=1}^n\{X_i-\mu(P)\}. $$ Thus $r_{n,P}=0$ for every $n$ and every $P$ in the class. In particular, the remainder condition holds uniformly without requiring a limiting argument.

For a numerical illustration, suppose a hypothetical sample of size $4$ is $8,9,11,14$, and the target mean under the hypothetical distribution is $\mu(P)=10$. Then $\bar X=(8+9+11+14)/4=10.5$, so the scaled estimation error is $$ \sqrt{4}(10.5-10)=2(0.5)=1. $$ The influence-function sum gives the same result: $$ \frac{(8-10)+(9-10)+(11-10)+(14-10)}{\sqrt{4}} = \frac{-2-1+1+4}{2} = 1. $$ The equality holds for every sample, not only this one. For a uniform normal approximation to this sum, however, additional distributional conditions would be needed.

Worked Example: A Ratio Estimator and Its Uniform Remainder

Let $Z_i=(X_i,Y_i)$ be independent and identically distributed under $P$, and suppose $\mu_X(P)=E_P[X_i]$ is positive. The target and estimator are $$ \theta(P)=\frac{\mu_Y(P)}{\mu_X(P)}, \qquad \hat\theta_n=\frac{\bar Y}{\bar X}. $$ Suppose the class satisfies $\mu_X(P)\ge c>0$, uniformly bounded second moments for $X_i,Y_i$, and a uniform bound on $|\theta(P)|$. These conditions keep the population denominator away from zero and control the fluctuations of the sample means.

Define $\psi_P(X_i,Y_i)=\{Y_i-\theta(P)X_i\}/\mu_X(P)$. It has mean zero because $\mu_Y(P)-\theta(P)\mu_X(P)=0$. The exact difference can be written $$ \hat\theta_n-\theta(P) = \frac{\bar Y-\theta(P)\bar X}{\bar X}. $$ Adding and subtracting the version with denominator $\mu_X(P)$ yields $$ \sqrt n\{\hat\theta_n-\theta(P)\} = \frac{1}{\sqrt n}\sum_{i=1}^n \frac{Y_i-\theta(P)X_i}{\mu_X(P)} + \sqrt n\{\bar Y-\theta(P)\bar X\} \left\{\frac{1}{\bar X}-\frac{1}{\mu_X(P)}\right\}. $$ The last term is the remainder.

Here is a complete numerical check using hypothetical data. Suppose $\mu_X(P)=2$, $\mu_Y(P)=5$, so $\theta(P)=2.5$, and the four observed pairs are $(1,2),(2,5),(3,4),(3,7)$. Then $\bar X=9/4=2.25$, $\bar Y=18/4=4.5$, and $\hat\theta_n=4.5/2.25=2$. The scaled estimation error is $2(2-2.5)=-1$. The leading term is $$ \frac{1}{\sqrt4}\sum_{i=1}^4 \frac{Y_i-2.5X_i}{2} = \frac{1}{2}\cdot\frac{18-2.5(9)}{2} = \frac{1}{2}\cdot\frac{-4.5}{2} = -1.125. $$ The remainder is $$ 2(4.5-2.5(2.25)) \left(\frac{1}{2.25}-\frac{1}{2}\right) = 2(-1.125)(-0.05556) = 0.125. $$ Thus the leading term plus remainder is $-1.125+0.125=-1$, exactly the scaled estimation error.

To see why the remainder is uniformly negligible under the stated conditions, write $A_n=\bar Y-\theta(P)\bar X$. Uniform moment bounds imply $A_n=O_{\mathcal P}(n^{-1/2})$ and $\bar X-\mu_X(P)=O_{\mathcal P}(n^{-1/2})$. On the event $\bar X\ge c/2$, $$ \left|\frac{1}{\bar X}-\frac{1}{\mu_X(P)}\right| = \frac{|\bar X-\mu_X(P)|}{|\bar X|\mu_X(P)} \le \frac{2|\bar X-\mu_X(P)|}{c^2}. $$ The complement of this event has probability tending to zero uniformly by the uniform concentration of $\bar X$ around $\mu_X(P)$. On the event itself, the scaled remainder is a product of $\sqrt n A_n=O_{\mathcal P}(1)$ and a reciprocal difference of order $O_{\mathcal P}(n^{-1/2})$, hence is $o_{\mathcal P}(1)$. This verifies the uniform expansion under these conditions.

Worked Example: Pointwise Control Can Fail Uniformly

Let $X_i$ be independent Bernoulli observations with success probability $p$, and let the class contain every $p\in(0,1)$. The target is $p$. Consider the estimator $$ \tilde p_n=\bar X+n^{-1/2}I(X_1=0,\ldots,X_n=0). $$ Its scaled error has an exact representation with influence function $X_i-p$: $$ \sqrt n(\tilde p_n-p) = \frac{1}{\sqrt n}\sum_{i=1}^n(X_i-p)+I(X_1=0,\ldots,X_n=0). $$ The remainder is the indicator of the all-zero event.

For any fixed $p>0$, the probability of that event is $(1-p)^n$, which tends to zero. So the remainder is negligible pointwise for every fixed distribution. But it is not negligible uniformly over $p\in(0,1)$: $$ \sup_{0<p<1}P_p(X_1=0,\ldots,X_n=0) = \sup_{0<p<1}(1-p)^n = 1. $$ The supremum equals $1$ even though no $p$ in the class equals zero: probabilities can be made arbitrarily close to $1$ by choosing $p$ sufficiently small. For example, at $p=0.01$ and $n=10$, the remainder is nonzero with probability $0.99^{10}\approx0.9044$. Restricting the class to $p\ge\varepsilon>0$ would instead bound this probability by $(1-\varepsilon)^n$, which tends to zero. This example shows why the class of processes is part of a uniform claim.

How Uniform Remainders Support Approximation

An asymptotic linear representation is often used together with a limit theorem for its leading sum. To make the role of uniformity precise, suppose that for each $P$, the leading term $S_{n,P}=n^{-1/2}\sum_i\psi_P(Z_i)$ has a proposed approximation $G_P$, and that the approximation error tends to zero uniformly in $P$. For example, $G_P$ may be a centered normal distribution with covariance equal to that of $\psi_P(Z_i)$, provided a suitable uniform central limit theorem applies.

The remainder condition ensures that replacing the estimator's scaled error by $S_{n,P}$ does not spoil such an approximation. Here is a short justification using bounded Lipschitz test functions. Let $f$ satisfy $|f|\le M$ and $|f(x)-f(y)|\le L\|x-y\|$. For any $\eta>0$, split according to whether $\|r_{n,P}\|\le\eta$: $$ \left|E_P f(S_{n,P}+r_{n,P})-E_P f(S_{n,P})\right| \le L\eta+2M P(\|r_{n,P}\|>\eta). $$ Taking the supremum over $P$ makes the second term vanish as $n$ grows, for each fixed $\eta$. Letting $\eta$ decrease to zero then makes the first term arbitrarily small. This proves that the estimator and its leading sum have uniformly close expectations for this class of test functions. If the leading sums themselves converge uniformly to $G_P$, the triangle inequality gives the same uniform approximation for the estimator.

The two ingredients have different jobs: a uniform asymptotic linear representation controls the estimator-specific remainder, while a uniform limit theorem controls the leading sum. Neither ingredient should be silently substituted for the other. In particular, linearity alone does not imply a normal limit, and a pointwise central limit theorem plus a pointwise expansion does not automatically produce a uniform normal approximation.

Common Pitfall and Practical Checks

A common mistake is to establish that the remainder tends to zero for every fixed $P$, then state that the result holds uniformly over $\mathcal P$. The Bernoulli example shows why this reasoning is invalid: the process can move closer to a problematic boundary as $n$ grows. The correction is to verify the supremum probability directly, or to prove bounds that hold with constants independent of $P$.

When checking a proposed representation, proceed in a disciplined order:

State the target $\theta(P)$, the class $\mathcal P$, and the sampling assumptions.
Specify a mean-zero influence function $\psi_P$, and verify its centering under each $P$.
Write the exact difference between the estimator and its leading sum; identify the remainder rather than discarding it.
Give conditions that bound the remainder uniformly, including any denominator or moment conditions.
Separate the remainder argument from any claim of a uniform normal or other distributional approximation.

Uniformity is especially valuable when the data-generating process is uncertain or changes across samples. A uniform result protects the approximation against every process in the stated class, including sequences of processes that approach its difficult regions. The protection is only as broad as the class and the assumptions: narrowing the class may make uniform control possible, while expanding it may make the same claim false.

Key takeaway: A uniform asymptotic linear representation requires the scaled estimation error to equal its influence-function sum plus a remainder that vanishes in probability uniformly over the stated class. Uniform control is what lets a pointwise expansion support reliable approximations across changing data-generating processes.

Check Your Understanding

Answer each question by identifying both the relevant condition and its role.

In the definition of uniform asymptotic linearity, why is the supremum over $P\in\mathcal P$ taken before the limit in $n$?
For the sample mean, identify the influence function and explain why its representation has zero remainder for every sample.
In the ratio-estimator example, what condition keeps the reciprocal of the sample denominator under control?
For the Bernoulli estimator, why does pointwise remainder convergence hold for each fixed $p>0$, while uniform convergence fails over $p\in(0,1)$?
Does a uniform asymptotic linear representation alone guarantee a uniform normal approximation? Explain what additional result is needed.