Differentiable Parametric Submodels

Why Vary a Distribution Along a Path?

Many questions in semiparametric analysis ask how a quantity changes when the underlying probability distribution is perturbed. A useful way to make a perturbation precise is to introduce a real parameter $t$ and a family of distributions $P_t$, with $P_0$ as the distribution of interest. Moving $t$ slightly away from zero traces a one-dimensional path through a larger collection of possible distributions.

The derivative at zero describes the path’s local direction. It is not, by itself, the derivative of every statistic or parameter of interest: to obtain that derivative, we also specify a functional of the distribution and calculate how it changes along the path. This tutorial develops that distinction through three constructions: exponential tilting, location shifts, and contamination paths.

Definition: Let $P_t$, for $t$ in an open interval around zero, be probability distributions with densities $p_t$ relative to a common measure $\mu$. The family is differentiable in $L^1(\mu)$ at zero if there is an integrable function $\dot p_0$ such that $$ \int \left|p_t(x)-p_0(x)-t\dot p_0(x)\right|\,d\mu(x)=o(|t|). $$ The function $\dot p_0$ is the density derivative of the path at $P_0$. A one-dimensional path is a family indexed by the single scalar $t$, with the baseline distribution at $t=0$.

The common dominating measure makes the densities comparable along the path. The $L^1$ condition is stronger than merely requiring that $p_t(x)$ have a pointwise derivative for each $x$; it also controls the total error in the density approximation. In the examples below, the paths have explicit densities and satisfy this condition. Here, “regular path” refers to this kind of local differentiable construction; it does not, by itself, assert every stronger regularity condition used in semiparametric theory.

From a Density Derivative to an Expectation Derivative

The density derivative must integrate to zero, because every $p_t$ integrates to one. Indeed, integrating the $L^1$ expansion and using $\int p_t\,d\mu=\int p_0\,d\mu=1$ gives

0=t\int \dot p_0(x)\,d\mu(x)+o(|t|), \qquad\text{so}\qquad \int \dot p_0(x)\,d\mu(x)=0.

For a bounded measurable function $h$, define its expectation under $P_t$ by $\Psi(P_t)=\int h(x)p_t(x)\,d\mu(x)$. Multiplying the density expansion by $h$ and integrating gives the path derivative:

\left.\frac{d}{dt}\Psi(P_t)\right|_{t=0} = \int h(x)\dot p_0(x)\,d\mu(x).

This calculation follows because the absolute value of the integrated remainder is at most $\sup_x|h(x)|$ times the $L^1$ remainder, which is $o(|t|)$. For an unbounded $h$, the same formula requires additional integrability and control of the remainder; pointwise density differentiation alone is not enough. The density derivative describes the path, while the integral against $h$ gives the derivative of this particular expectation.

Exponential Tilting

Exponential tilting changes the relative weight assigned to different outcomes according to a chosen function $g$. Let $P_0$ have density $p_0$, and suppose $g$ is bounded. Define the normalizing function $A(t)$ and the tilted density $p_t$ by

A(t)=\log\left(\int e^{t g(x)}p_0(x)\,d\mu(x)\right), \qquad p_t(x)=e^{t g(x)-A(t)}p_0(x).

The normalizer ensures that $p_t$ integrates to one. Boundedness of $g$ makes the integral finite for $t$ near zero and permits differentiation. Since $A(0)=0$, differentiating the normalizing integral gives $A'(0)=E_0[g(X)]$, where $E_0$ denotes expectation under $P_0$. Differentiating the density then yields

\dot p_0(x) = p_0(x)\left(g(x)-E_0[g(X)]\right).

The centering term is essential: without it, the derivative generally would not integrate to zero. For any bounded $h$, the derivative of its expectation along this path is

\left.\frac{d}{dt}E_t[h(X)]\right|_{t=0} = E_0[h(X)g(X)]-E_0[h(X)]E_0[g(X)].

Thus, the change in an expectation depends on how $h$ and the tilting function $g$ vary together under the baseline distribution.

Worked Example: A Finite-Support Exponential Tilt

Suppose $X$ has baseline probabilities $P_0(X=0)=0.5$, $P_0(X=1)=0.3$, and $P_0(X=2)=0.2$. Tilt by $g(x)=x$. Find the density derivative at each outcome and the derivative of $P_t(X=2)$ at zero.

The normalizing function is $$ A(t)=\log\left(0.5+0.3e^t+0.2e^{2t}\right). $$ At zero, $E_0[X]=0(0.5)+1(0.3)+2(0.2)=0.7$, so $A'(0)=0.7$. The density-derivative formula gives $\dot p_0(x)=p_0(x)(x-0.7)$. In the order $x=0,1,2$, the derivatives are $$ 0.5(0-0.7)=-0.35,\qquad 0.3(1-0.7)=0.09,\qquad 0.2(2-0.7)=0.26. $$ They sum to zero, as required for a density derivative. Since the event $X=2$ has probability $p_t(2)$, its derivative is $\dot p_0(2)=0.26$. For small positive $t$, its probability increases at an initial rate of $0.26$ per unit of $t$.

Location Shifts

A location-shift path translates a distribution by $t$ measurement units without changing its shape. If $X$ has density $f$ at the baseline, the density of $X+t$ is $p_t(x)=f(x-t)$. When $f$ is differentiable, differentiating with respect to $t$ at zero gives

\dot p_0(x)=-f'(x).

For example, if the baseline density is normal with mean 10 units and variance 4 square units, then $f'(x)=-(x-10)f(x)/4$. Therefore the location-shift path has $\dot p_0(x)=(x-10)f(x)/4$. This derivative integrates to zero. For the probability of exceeding 12 units, differentiating its normal tail probability gives

P_t(X>12)=1-\Phi\left(\frac{12-(10+t)}{2}\right), \qquad \left.\frac{d}{dt}P_t(X>12)\right|_{t=0} =\frac{\phi(1)}{2} \approx 0.1210.

Here $\Phi$ and $\phi$ are the standard normal distribution and density functions. The derivative is an initial change in probability of about 0.1210 per one-unit increase in the location parameter; it is not a probability or a percentage change in the location itself.

Worked Example: The Mean Along a Normal Location Path

Let the baseline distribution be $N(10,4)$, so its mean is 10 measurement units and its standard deviation is 2 units. Consider $X_t=X_0+t$, where $X_0\sim N(10,4)$. Find the density derivative at zero and the derivative of the mean.

The shifted density is $$ p_t(x)=\frac{1}{2\sqrt{2\pi}} \exp\left(-\frac{(x-10-t)^2}{8}\right). $$ Differentiating this expression with respect to $t$ gives $$ \dot p_0(x)=\frac{x-10}{4}p_0(x). $$ Its integral is zero because $E_0[X-10]=0$. For the mean, the path has an exact expectation: $E_t[X]=E[X_0+t]=10+t$. Consequently, $$ \left.\frac{d}{dt}E_t[X]\right|_{t=0}=1. $$ The derivative is one measurement unit per one-unit shift. This mean calculation is exact and does not rely on applying the bounded-function expectation result to the unbounded function $h(x)=x$.

Contamination Paths

A contamination path combines a baseline distribution $P_0$ with another distribution $Q$:

$$ P_t=(1-t)P_0+tQ. $$

For $0\leq t\leq1$, this is a mixture: a fraction $1-t$ of the probability comes from $P_0$, and a fraction $t$ comes from $Q$. If both distributions have densities $p_0$ and $q$ relative to a common measure, then the density is $p_t=(1-t)p_0+tq$, and its derivative is $\dot p_0=q-p_0$. For any bounded $h$, linearity of expectation gives the exact identity

E_t[h(X)]=(1-t)E_0[h(X)]+tE_Q[h(X)], \qquad \left.\frac{d}{dt}E_t[h(X)]\right|_{t=0} =E_Q[h(X)]-E_0[h(X)].

The mixture is automatically a valid distribution for $t$ between zero and one. A two-sided path around zero requires that the formula remain nonnegative for small negative $t$ as well. That holds in the finite-support example below; in general it needs to be checked rather than assumed. Likewise, a common dominating measure is needed to express the density derivative. A path may be differentiable as a mixture without satisfying every stronger regularity condition used for other purposes.

Worked Example: A Finite-Support Contamination Path

Let $P_0$ assign probabilities $(0.5,0.3,0.2)$ to $X=0,1,2$, respectively, and let $Q$ assign probabilities $(0.1,0.4,0.5)$ to the same outcomes. Calculate the density derivative, check that the path is valid near zero on both sides, and find the derivative of the mean.

For each outcome, the density derivative is $q(x)-p_0(x)$. Thus, in the order $0,1,2$, it is $$ (0.1-0.5,\ 0.4-0.3,\ 0.5-0.2)=(-0.4,\ 0.1,\ 0.3). $$ These entries sum to zero. The path probabilities are $(0.5-0.4t,\ 0.3+0.1t,\ 0.2+0.3t)$. They sum to one, and all are positive for $t$ sufficiently close to zero from either side; in particular, they are positive for $-0.6<t<1$.

The baseline mean is $E_0[X]=0(0.5)+1(0.3)+2(0.2)=0.7$, while $E_Q[X]=0(0.1)+1(0.4)+2(0.5)=1.4$. Hence $$ E_t[X]=(1-t)(0.7)+t(1.4)=0.7+0.7t, \qquad \left.\frac{d}{dt}E_t[X]\right|_{t=0}=0.7. $$ The mean initially increases by 0.7 outcome units per unit increase in the contamination fraction $t$.

Common Pitfalls and Checks

A path is a mathematical perturbation of a distribution, not automatically a change in an intervention or policy. Its derivative describes local distributional change; causal interpretation requires additional assumptions and a causal estimand. Before using a path, check that the density integrates to one for nearby $t$, that the proposed derivative integrates to zero, and that the target functional can be differentiated under the conditions you have imposed.

Confusing $\dot p_0$ with a target derivative. The density derivative describes the path. The derivative of an expectation also depends on the function being averaged: it is $\int h\dot p_0\,d\mu$ when the conditions for that formula hold.
Checking only pointwise derivatives. Pointwise differentiation does not guarantee an integrated derivative. The $L^1$ expansion, or an appropriate domination argument, controls the remainder.
Forgetting normalization. Since each $p_t$ integrates to one, $\int\dot p_0\,d\mu=0$. A proposed density derivative that fails this check cannot be correct.
Assuming every mixture formula works for negative $t$. A contamination mixture is automatically a distribution for $0\leq t\leq1$. A two-sided local path needs a separate nonnegativity check.
Ignoring units. A derivative with respect to a location parameter has units of the target divided by measurement units; a derivative with respect to a contamination fraction has units of the target per unit fraction.

Key takeaway: Construct a path $P_t$ through the baseline $P_0$, identify its density derivative $\dot p_0$, and check normalization and local validity. Then differentiate the chosen functional along that path; the answer depends on both the path and the functional.

Check Your Understanding

Answer each question by identifying the path, its derivative, or the condition that must be checked.

Why must the density derivative of a normalized path integrate to zero?
For an exponential tilt by $g$, why does the density derivative contain the centering term $E_0[g(X)]$?
Under a location shift $X_t=X_0+t$, what is the derivative of the mean, and what are its units?
For a contamination path between $P_0$ and $Q$, express the derivative of $E_t[h(X)]$ using expectations under those distributions.
Why is pointwise differentiability of $p_t(x)$ alone insufficient to justify differentiating an expectation?