Regularity Conditions for Stein’s Identity

Why Regularity Conditions Matter

Stein’s identity turns an expectation involving a centered normal variable into an expectation involving a derivative. That step is integration by parts, and integration by parts has hypotheses: the relevant integrals must exist, the function must have enough regularity, and the boundary term must vanish. If these conditions are skipped, a formal derivative calculation can give an incorrect identity.

We work with \(X\sim N_p(\mu,I_p)\), where \(\mu\in\mathbb R^p\), and use the coordinatewise integration-by-parts form. The aim here is to specify conditions under which that calculation is valid and to see exactly what can go wrong when a boundary contribution remains.

Conditions on Coordinate Slices

For a vector-valued function \(g=(g_1,\ldots,g_p):\mathbb R^p\to\mathbb R^p\), write \(x_{-i}\) for all coordinates except the \(i\)th. When \(x_{-i}\) is fixed, the function \(t\mapsto g_i(x_1,\ldots,t,\ldots,x_p)\) is the \(i\)th coordinate slice. The normal density factors into one-dimensional densities, so integration by parts can be applied to these slices.

Theorem: Let \(X\sim N_p(\mu,I_p)\), and let \(g:\mathbb R^p\to\mathbb R^p\) be measurable. For each \(i\), suppose that for almost every \(x_{-i}\), the slice \(t\mapsto g_i(x_1,\ldots,t,\ldots,x_p)\) is locally absolutely continuous. Suppose also that

\(E[|g_i(X)(X_i-\mu_i)|]<\infty\) and \(E[|\partial_i g_i(X)|]<\infty\),

and, for almost every such \(x_{-i}\),

\(\lim_{t\to-\infty}g_i(x_1,\ldots,t,\ldots,x_p)\phi(t-\mu_i)= \lim_{t\to\infty}g_i(x_1,\ldots,t,\ldots,x_p)\phi(t-\mu_i)=0\),

where \(\phi(u)=(2\pi)^{-1/2}e^{-u^2/2}\), and \(\partial_i g_i\) denotes the derivative along that slice, defined almost everywhere. Then

\(E[(X_i-\mu_i)g_i(X)]=E[\partial_i g_i(X)]\)

for every \(i\). Consequently, \(E[(X-\mu)^{\mathsf T}g(X)]=E[\operatorname{div}g(X)]\), where \(\operatorname{div}g=\sum_{i=1}^p\partial_i g_i\).

The integrability assumptions are absolute, not merely conditional statements about a formally simplified integral. They ensure that both expectations are finite and that integrating over the other coordinates is legitimate. The boundary condition is imposed on almost every coordinate slice; pointwise decay in every direction is not required by this theorem.

Local absolute continuity is a convenient classical condition: on every bounded interval, a slice can be recovered by integrating its almost-everywhere derivative. In particular, a continuously differentiable slice qualifies. A function that jumps inside a slice does not qualify, even if its ordinary derivative exists and is zero away from the jump.

Proof by Coordinatewise Integration by Parts

Fix a coordinate \(i\). The joint density of \(X\) is the product of the coordinate densities \(\prod_{j=1}^p\phi(x_j-\mu_j)\). Fix a value \(x_{-i}\) for which the slice assumptions hold, and abbreviate that slice by \(h(t)\). Since \(\phi'(t-\mu_i)=-(t-\mu_i)\phi(t-\mu_i)\), integration by parts on a finite interval \([-M,M]\) gives

\int_{-M}^{M}(t-\mu_i)h(t)\phi(t-\mu_i)\,dt = -h(M)\phi(M-\mu_i)+h(-M)\phi(-M-\mu_i) +\int_{-M}^{M}h'(t)\phi(t-\mu_i)\,dt.

The boundary terms tend to zero as \(M\to\infty\) by the assumed decay. The two integrals converge absolutely for almost every fixed \(x_{-i}\): this follows from joint absolute integrability and Fubini’s theorem. Thus the limit gives the slice identity

\int_{\mathbb R}(t-\mu_i)h(t)\phi(t-\mu_i)\,dt = \int_{\mathbb R}h'(t)\phi(t-\mu_i)\,dt.

Now multiply by the density of \(X_{-i}\) and integrate over \(x_{-i}\). The absolute integrability assumptions allow Fubini’s theorem on both sides, yielding \(E[(X_i-\mu_i)g_i(X)]=E[\partial_i g_i(X)]\). Summing these finite equalities over \(i=1,\ldots,p\) proves the divergence identity. \(\square\)

A Practical Sufficient Condition

The slice conditions can be checked directly for a particular function, but a common sufficient condition is easier to use. Suppose each \(g_i\) is continuously differentiable and there are finite constants \(C\) and \(k\geq0\) such that, for all \(x\),

|g_i(x)|+|\partial_i g_i(x)| \leq C(1+\|x\|^k).

Then the theorem’s assumptions hold. To see why, all polynomial moments of a normal random vector are finite, so both required expectations are finite. For fixed \(x_{-i}\), the bound on \(g_i\) grows at most polynomially in \(|t|\), while \(\phi(t-\mu_i)\) decays exponentially in \(t^2\). Hence \(g_i(x)\phi(t-\mu_i)\to0\) at both ends of that slice. Continuity of the derivatives gives local absolute continuity. This condition is sufficient, not necessary: some functions grow faster than any polynomial and still satisfy the theorem’s integrability and boundary requirements.

Key distinction: Differentiability alone does not establish Stein’s identity. Check that the derivative and the centered-function term are integrable, and that the boundary term vanishes. These are separate obligations.

Worked Examples

Worked Example: A Polynomial Function

Let \(X\sim N(\mu,1)\) and \(g(x)=x^3\). The function and its derivative have polynomial growth, so the sufficient condition applies. The identity predicts \(E[(X-\mu)X^3]=E[3X^2]\). Verify both sides.

Write \(Z=X-\mu\), so \(Z\sim N(0,1)\) and \(X=\mu+Z\). Using \(E[Z]=E[Z^3]=0\), \(E[Z^2]=1\), and \(E[Z^4]=3\), the left side is

E[ZX^3] =E[Z(\mu+Z)^3] =3\mu^2E[Z^2]+E[Z^4] =3\mu^2+3.

The right side is \(E[g'(X)]=E[3X^2]=3(\mu^2+1)=3\mu^2+3\). The two finite expectations agree, as the theorem requires.

Worked Example: An Exponential Function

Let \(X\sim N(\mu,1)\) and \(g(x)=e^{ax}\), where \(a\) is any fixed real number. Although \(g\) need not have polynomial growth, the required integrability and boundary decay still hold.

Completing the square gives \(e^{ax}\phi(x-\mu)=e^{a\mu+a^2/2}\phi(x-\mu-a)\). The factor on the right is a finite constant times a shifted normal density. In particular, its integral is finite and it tends to zero at both endpoints. Also \(g'(x)=ae^{ax}\), so the derivative expectation is finite. For the left side, using \(y=x-\mu-a\),

\begin{aligned} E[(X-\mu)e^{aX}] &=e^{a\mu+a^2/2}\int_{\mathbb R}(x-\mu)\phi(x-\mu-a)\,dx\\ &=e^{a\mu+a^2/2}\int_{\mathbb R}(y+a)\phi(y)\,dy\\ &=a e^{a\mu+a^2/2}. \end{aligned}

On the other side, \(E[g'(X)]=aE[e^{aX}]=a e^{a\mu+a^2/2}\). This verifies Stein’s identity, including when \(a\) is negative or zero.

Worked Example: A Jump Leaves an Internal Boundary Term

Let \(X\sim N(\mu,1)\) and set \(g(x)=\mathbf 1\{x>0\}\). The function is bounded, and \(E[|(X-\mu)g(X)|]\) is finite. But it is not locally absolutely continuous across zero. Its ordinary derivative is zero everywhere except at the jump, so treating that derivative as zero would incorrectly predict a zero left side.

In fact, since \(\phi'(x-\mu)=-(x-\mu)\phi(x-\mu)\),

E[(X-\mu)g(X)] =\int_0^\infty (x-\mu)\phi(x-\mu)\,dx =-\int_0^\infty \phi'(x-\mu)\,dx =\phi(-\mu).

This is positive for every finite \(\mu\), whereas the expected ordinary derivative away from zero is zero. The missing amount is the contribution at the internal boundary \(x=0\). For example, when \(\mu=0\), the left side is \(\phi(0)=1/\sqrt{2\pi}\), not zero. The classical theorem does not apply to this discontinuous \(g\); the calculation shows why it cannot be applied by simply ignoring the jump.

Boundary Terms and Common Errors

On a finite interval, the integration-by-parts formula always displays endpoint terms. Stein’s identity on the whole real line follows only after those terms vanish in the limiting calculation. The theorem makes this requirement explicit on almost every slice. In the jump example, splitting the integral at zero exposes an internal boundary instead; omitting it loses a nonzero contribution.

Using a derivative without checking regularity. A derivative that exists almost everywhere does not, by itself, justify the classical integration-by-parts argument. The jump example has derivative zero almost everywhere but a nonzero expectation on the left.
Checking only that \(E[\partial_i g_i(X)]\) is finite. The other expectation must also be absolutely integrable, and the boundary limit must be controlled.
Assuming decay of the normal density settles every case. The product \(g_i(x)\phi(x_i-\mu_i)\), not the density alone, determines the boundary term. A rapidly growing \(g_i\) can offset the density’s decay.
Applying a one-dimensional argument without accounting for other coordinates. In several dimensions, the slice identity must hold for almost every fixed \(x_{-i}\), and the joint integrability conditions justify integrating those identities.

The practical lesson is to establish regularity, absolute integrability, and boundary decay before using the identity. When any condition is doubtful, return to integration by parts on a finite interval or on separate smooth regions and retain every endpoint term.

Check Your Understanding

For each question, identify which regularity, integrability, or boundary issue is relevant.

For \(X\sim N(\mu,1)\) and \(g(x)=x^5\), which simple growth condition ensures the hypotheses of Stein’s identity?
Why is absolute integrability of both terms useful when passing from coordinate slices to a multivariate expectation?
For \(g(x)=\mathbf 1\{x>0\}\), calculate \(E[(X-\mu)g(X)]\) and explain why the almost-everywhere derivative does not give the identity.
In the slice-wise theorem, what quantity must tend to zero at each endpoint: \(g_i(t,x_{-i})\), the normal density, or their product?
Does exponential growth automatically violate Stein’s identity? Use the example \(g(x)=e^{ax}\) to explain your answer.