Why Some Parameters Resist Regular Inference
An influence function gives a local description of how a parameter changes when the data distribution is perturbed. When that description is a finite-variance function of one observation, it supports the usual regular root-\(n\) framework: an estimator can often be asymptotically linear, with a finite asymptotic variance. But not every parameter has such a description. A parameter may respond too sharply to small changes in the distribution, or a change in the parameter may require moving the support in a way that is not a regular local perturbation.
Two instructive cases are the value of a density at one point and an endpoint of a distribution's support. The first is sensitive to changes concentrated in an arbitrarily small neighborhood. The second can change when probability mass appears or disappears at the edge of the support. These examples clarify what failure of pathwise differentiability means—and what it does not mean: extra structural assumptions may still permit useful inference.
The finite-variance requirement is essential. By the Cauchy–Schwarz inequality, any influence function must satisfy \(|E_P[\phi s]|\leq \{E_P[\phi^2]\}^{1/2}\{E_P[s^2]\}^{1/2}\). Therefore, the derivative of the parameter must be bounded as a function of the score, when score size is measured by its \(L_2(P)\) norm. If one can find score directions with norm one but arbitrarily large parameter derivatives, no finite-variance influence function can represent those derivatives.
Pointwise Density Evaluation
Consider the parameter \(\theta(P)=f(x_0)\), the density's value at a fixed interior point \(x_0\). A density is ordinarily specified only up to changes on sets of probability zero, so this point-value parameter is not even well-defined for unrestricted density equivalence classes. To give it a meaningful interpretation, suppose instead that the model specifies a continuous density representative near \(x_0\). Even with that convention, pointwise evaluation is not pathwise differentiable in a sufficiently rich model of densities.
The mechanism is localization. A perturbation can have a large value at \(x_0\) but be confined to a very narrow neighborhood, keeping its squared integral—and hence its score norm—controlled. A second, disjoint perturbation balances the total integral so the changed density still integrates to one. The following construction uses infinitely differentiable bump functions, so the localized changes really are smooth.
Worked Example: Smooth Paths That Defeat a Density Influence Function
Take the baseline density \(f_0(x)=1\) on \([0,1]\), and evaluate it at \(x_0=0.5\). Let \(b\) be a nonnegative, infinitely differentiable function supported on \((-1,1)\), with \(b(0)>0\). For example, take \(b(u)=\exp\{-1/(1-u^2)\}\) when \(|u|<1\), and \(b(u)=0\) otherwise. Let \(q\) be a nonnegative, infinitely differentiable function supported on \((0.7,0.9)\), with \(Q=\int_0^1q(x)\,dx>0\). Write \(B=\int_{-1}^1b(u)\,du\) and \(C=\int_0^1q(x)^2\,dx\).
For \(0<\varepsilon<0.1\), define \(a_\varepsilon=\sqrt{\varepsilon}B/Q\) and \(s_\varepsilon(x)=\varepsilon^{-1/2}b((x-0.5)/\varepsilon)-a_\varepsilon q(x)\). The two terms have disjoint supports. Substitution \(u=(x-0.5)/\varepsilon\) gives \(\int_0^1s_\varepsilon(x)\,dx=\sqrt{\varepsilon}B-a_\varepsilon Q=0\). Thus \(s_\varepsilon\) is a mean-zero score direction. Its squared norm is \(\int_0^1s_\varepsilon(x)^2\,dx=\int_{-1}^1b(u)^2\,du+\varepsilon B^2C/Q^2\), which converges to a finite positive constant as \(\varepsilon\) decreases to zero.
Normalize the score by setting \(g_\varepsilon=s_\varepsilon/\|s_\varepsilon\|_2\), so \(\|g_\varepsilon\|_2=1\). For each fixed \(\varepsilon\), the path \(f_t(x)=1+t g_\varepsilon(x)\) is a density for all sufficiently small \(|t|\): its integral is one, and boundedness of \(g_\varepsilon\) ensures nonnegativity when \(|t|<1/\|g_\varepsilon\|_\infty\). This is a differentiable-in-quadratic-mean path with score \(g_\varepsilon\). Since \(q(0.5)=0\), its density-value derivative is \(\left.\frac{d}{dt}f_t(0.5)\right|_{t=0}=g_\varepsilon(0.5)=\frac{b(0)}{\sqrt{\varepsilon}\|s_\varepsilon\|_2}\). The denominator norm stays bounded away from zero, so this derivative diverges as \(\varepsilon\) tends to zero, despite every score having norm one.
If a finite-variance influence function existed, Cauchy–Schwarz would bound these derivatives by the same constant \(\|\phi\|_2\). The divergence contradicts that requirement. The construction proves failure of pathwise differentiability in this model; it does not rule out regular inference after imposing a parametric form or sufficiently strong smoothness restrictions.
Support Endpoints and Irregular Paths
A support endpoint illustrates a different difficulty. Consider the model \(X_i\stackrel{\mathrm{iid}}{\sim}\mathrm{Uniform}[\theta,1]\), with \(\theta<1\), where the target is the lower endpoint \(\theta\). Changing \(\theta\) shifts the support. Near the boundary, a small change in \(\theta\) can put probability mass where the baseline distribution assigned none. This behavior is unlike a regular score perturbation that changes a density smoothly within a common support.
Worked Example: The Minimum in a Uniform Endpoint Model
For each sample size \(n\), let \(M_n=X_{(1)}=\min_iX_i\) be the minimum of \(n\) independent observations from \(\mathrm{Uniform}[\theta,1]\). For \(\theta\leq m\leq1\), independence gives \(P_\theta(M_n>m)=\{(1-m)/(1-\theta)\}^n\). In particular, at \(\theta=0\) and \(n=100\), \(P_0(M_n>m)=(1-m)^{100}\). The minimum therefore moves toward the endpoint on a scale of order \(1/n\), not the usual \(1/\sqrt n\) scale.
More precisely, for \(z\geq0\), \(P_0(nM_n>z)=P_0(M_n>z/n)=(1-z/n)^n\) when \(0\leq z\leq n\), and this converges to \(e^{-z}\) for each fixed \(z\). Thus \(nM_n\) converges in distribution to an exponential random variable with rate one. This nonnormal, faster-than-root-\(n\) behavior is a feature of the support boundary, not ordinary regular root-\(n\) estimation.
The underlying path is also nonregular at \(\theta=0\). Under \(P_0\), the interval \([0,\theta)\) has probability \(\theta\); under \(P_\theta\), it has probability zero. The distributions consequently differ on a region of probability order \(\theta\), and their Hellinger distance is of order \(\sqrt{\theta}\), not order \(\theta\). A differentiable-in-quadratic-mean path has square-root densities with a first-order \(L_2\) expansion, which implies Hellinger distance of order at most \(|\theta|\). The endpoint-shifting path fails that condition. The usual score-and-influence-function framework therefore does not describe this endpoint path.
This example should not be read as a general theorem that every support endpoint has the same rate or limiting distribution. Those depend on the model and on how density behaves near the boundary. It demonstrates why endpoint parameters require a check of the model's local regularity before applying standard root-\(n\) theory.
A Regular Contrast: An Integrated Functional
The distinction between local evaluation and integration helps explain the density example. An integral averages changes across a region; a point evaluation can detect a narrow spike. For a simple comparison, the mean under the uniform baseline has a finite-variance influence function.
Worked Example: The Mean Under a Uniform Baseline
For \(X\sim\mathrm{Uniform}[0,1]\), the mean is \(\mu=E[X]=1/2\). Along a regular density path with mean-zero score \(s\), differentiating the mean gives \(\left.\frac{d}{dt}E_{P_t}[X]\right|_{t=0}=E_P[Xs]\). Because \(E_P[s]=0\), this equals \(E_P[(X-1/2)s]\). Hence \(\phi(X)=X-1/2\) represents the derivative. It has mean zero and variance \(E_P[\phi(X)^2]=\int_0^1(x-1/2)^2\,dx=1/12\), so it is a finite-variance influence function. The mean's derivative is bounded by \(\sqrt{1/12}\|s\|_2\), unlike pointwise density evaluation in the unrestricted model.
What Failure of Pathwise Differentiability Implies
In the standard semiparametric framework, pathwise differentiability is the condition that allows a parameter's first-order change to be represented by a square-integrable influence function across regular score directions. Under the usual conditions for regular asymptotically linear estimation, that representation yields a finite asymptotic variance and an efficiency bound. If no such influence function exists, there is no finite bound of this usual form in the unrestricted model, and the standard influence-function route to regular root-\(n\) inference is unavailable.
This is not a claim that no estimator can converge at rate \(\sqrt n\) under any circumstances. A parametric model, a smoothness class, or other restrictions may shrink the available set of perturbations and make the target regular relative to that smaller model. For density estimation, smoothing can estimate local features at rates determined by smoothness; it does not automatically turn point evaluation into a regular root-\(n\) target in the unrestricted model. For endpoint parameters, specialized boundary methods may use their own rates and limiting distributions.
Common Pitfalls
- Assuming every density value is a well-defined parameter. Densities that agree almost everywhere can differ at a single point. Specify a continuous representative or a model where point evaluation is meaningful before discussing its differentiability.
- Calling a localized perturbation smooth without checking it. A triangular or piecewise-linear bump is not infinitely differentiable at its corners. The construction above uses \(C^\infty\) bump functions and a separate compensating bump, with the normalization and score norm shown explicitly.
- Using an influence function without testing boundedness. A proposed derivative representation must hold for every score in the model's tangent space and must satisfy the Cauchy–Schwarz bound.
- Treating a support shift as an ordinary score path. Moving support can violate differentiability in quadratic mean. Check the path's regularity before invoking standard influence-function inference.
- Concluding that nonregular means impossible to estimate. Nonregularity rules out the usual regular finite-variance root-\(n\) framework in the stated model; it does not rule out estimation under stronger restrictions or with different asymptotics.
Check Your Understanding
Use the definitions and examples above to answer the following questions.
- Why does a finite-variance influence function imply a bound on the derivative along any score direction?
- In the density-value construction, why is a second, disjoint bump included in the score?
- What happens to the pointwise density derivative when the localized score is normalized to have unit \(L_2\) norm?
- Why does the uniform endpoint path fail the usual differentiability-in-quadratic-mean condition?
- Does failure of pathwise differentiability in an unrestricted model rule out useful inference under a more restrictive model? Explain.