Weak Derivatives in Stein’s Identity

Why Extend Stein’s Identity?

The usual Gaussian integration-by-parts formula is often introduced for smooth functions. But many useful functions are only piecewise smooth: a hinge has a corner, an absolute-value function has a cusp, and a coordinatewise maximum changes formula along a boundary. These functions still have derivatives almost everywhere, and under suitable integrability conditions those derivatives suffice for Stein’s identity.

The extension is not a license to ignore regularity. We need a function with an appropriate weak derivative and enough integrability to make both sides of the identity finite. The conditions below allow corners, but not arbitrary jumps.

Definition: A function \(g:\mathbb{R}\to\mathbb{R}\) is locally absolutely continuous if its restriction to every bounded interval is absolutely continuous. Its derivative \(g'\) then exists almost everywhere, is locally integrable, and recovers changes in \(g\) by integration. In one dimension, a function with a locally integrable weak derivative has a locally absolutely continuous representative; the weak derivative agrees almost everywhere with that representative’s ordinary derivative.

The Scalar Identity for Weak Derivatives

Let \(X\sim N(\mu,\sigma^2)\), with \(\sigma^2>0\). Write \(\phi_{\mu,\sigma}(x)\) for its density. A convenient sufficient set of conditions is that \(g\) have a locally absolutely continuous representative and \(E[|g'(X)|]<\infty\) and \(E[|X-\mu|\,|g(X)|]<\infty\). The second condition makes the left-hand side absolutely integrable; the first does the same for the right-hand side.

Theorem: If \(X\sim N(\mu,\sigma^2)\), \(g\) is locally absolutely continuous, and \(E[|g'(X)|]<\infty\) and \(E[|X-\mu|\,|g(X)|]<\infty\), then

E[(X-\mu)g(X)]=\sigma^2 E[g'(X)].

The derivative is the almost-everywhere derivative of the absolutely continuous representative. Equivalently, \(g'\) may be interpreted as the weak derivative.

Here is a proof that also explains why no boundary term remains. Put \(h(x)=g(x)\phi_{\mu,\sigma}(x)\). The two stated expectations imply that \(h'\) is integrable, since, almost everywhere,

h'(x)=g'(x)\phi_{\mu,\sigma}(x) -\frac{x-\mu}{\sigma^2}g(x)\phi_{\mu,\sigma}(x).

Also \(h\) is integrable. On a bounded interval this follows from local absolute continuity and boundedness of the Gaussian density. Outside any fixed interval containing \(\mu\), \(|x-\mu|\) is bounded away from zero, so integrability of \(|x-\mu|\,|g(x)|\phi_{\mu,\sigma}(x)\) implies integrability of \(|h(x)|\) there.

Since \(h\) is locally absolutely continuous and \(h'\) is integrable on the line, \(h\) has finite limits at both ends of the line: for example, for \(y>x\), \(|h(y)-h(x)|\leq\int_x^y |h'(t)|\,dt\), whose tail tends to zero. Because \(h\) is integrable, each of those limits must be zero. Integrating \(h'\) over the line therefore gives zero. Substituting the displayed expression for \(h'\) yields

0=E[g'(X)]-\frac{1}{\sigma^2}E[(X-\mu)g(X)],

which proves the identity. A weakly differentiable function in one dimension has a locally absolutely continuous representative, so the same proof applies to it. The Gaussian distribution has a positive density, and changing a representative on a Lebesgue-null set changes neither expectation. \(\square\)

Worked Example: A Hinge at a Fixed Threshold

Let \(X\sim N(0,1)\) and \(g(x)=(x-1)_+=\max(x-1,0)\). This function is locally absolutely continuous. Its weak derivative is \(g'(x)=\mathbf{1}\{x>1\}\) almost everywhere; the value assigned at \(x=1\) is immaterial. Both required expectations are finite because \(g(x)\) grows at most linearly.

The identity gives

E[X(X-1)_+]=E[\mathbf{1}\{X>1\}]=P(X>1)=1-\Phi(1)\approx 0.1587,

where \(\Phi\) is the standard normal distribution function. To check the left side directly, use the standard normal density \(\phi\) and \(\phi'(x)=-x\phi(x)\):

\begin{aligned} E[X(X-1)_+] &=\int_1^\infty x(x-1)\phi(x)\,dx\\ &=\int_1^\infty \bigl(x^2-x\bigr)\phi(x)\,dx\\ &=\left[-x\phi(x)\right]_1^\infty+\int_1^\infty\phi(x)\,dx -\int_1^\infty x\phi(x)\,dx\\ &=\phi(1)+\bigl(1-\Phi(1)\bigr)-\phi(1)\\ &=1-\Phi(1). \end{aligned}

The corner at \(1\) causes no extra term: the function is continuous there, and its almost-everywhere derivative is integrable.

Worked Example: The Absolute-Value Function

Let \(X\sim N(0,1)\) and \(g(x)=|x|\). The function is locally absolutely continuous, and its weak derivative is \(\operatorname{sign}(x)\) almost everywhere. Thus

E[X|X|]=E[\operatorname{sign}(X)].

By symmetry, \(E[\operatorname{sign}(X)]=P(X>0)-P(X<0)=\tfrac12-\tfrac12=0\). Directly, \(X|X|\) is an odd function and the standard normal density is symmetric, so its expectation is also zero. Both integrability conditions hold because \(E[X^2]=1\) and \(E[|X|]<\infty\). The derivative has a jump in value across zero, but the function itself is continuous; its weak derivative is the sign function, defined only up to its value at zero.

Vector Functions and General Covariance

The same extension applies component by component to a multivariate normal vector. Let \(X\sim N_p(\mu,\Sigma)\), where \(\Sigma\) is positive definite, and let \(g:\mathbb{R}^p\to\mathbb{R}^p\). Write \(Dg(x)\) for the Jacobian matrix with entries \((Dg(x))_{ij}=\partial g_i(x)/\partial x_j\). The following sufficient conditions are stated after whitening the Gaussian, so that the one-dimensional result can be applied along each coordinate.

Theorem: Let \(A\) be a nonsingular matrix with \(AA^T=\Sigma\), and define \(q(z)=g(\mu+Az)\). Suppose each component \(q_i\) is locally in \(W^{1,1}\) and, for every \(i,j\), \(E[|Z_jq_i(Z)|]<\infty\) and \(E[|\partial q_i(Z)/\partial z_j|]<\infty\), where \(Z\sim N_p(0,I)\). Then

E[(X-\mu)g(X)^T]=\Sigma E[Dg(X)^T].

The derivatives on the right are weak partial derivatives, defined almost everywhere.

To prove this, local \(W^{1,1}\) regularity gives, for almost every choice of the other coordinates, a locally absolutely continuous representative along each coordinate line. Fubini’s theorem and the stated integrability conditions permit application of the scalar identity to those lines and then integration over the remaining coordinates. For each \(i,j\), this gives \(E[Z_jq_i(Z)]=E[\partial q_i(Z)/\partial z_j]\), or, in matrix form, \(E[Zq(Z)^T]=E[Dq(Z)^T]\).

Since \(X=\mu+AZ\), the chain rule for weak derivatives under this invertible linear change of variables gives \(Dq(Z)=Dg(X)A\) almost everywhere. Therefore

\begin{aligned} E[(X-\mu)g(X)^T] &=A E[Zq(Z)^T]\\ &=A E[Dq(Z)^T]\\ &=A E[A^T Dg(X)^T]\\ &=\Sigma E[Dg(X)^T]. \end{aligned}

The integrability assumptions ensure that the entries on both sides are finite. This proves the vector identity. \(\square\)

Worked Example: Ordering Two Gaussian Coordinates

Let \(X=(X_1,X_2)^T\sim N_2(0,I)\), with independent standard normal coordinates, and define \(g(x_1,x_2)=(\max(x_1,x_2),\min(x_1,x_2))^T\). The function changes formula on the line \(x_1=x_2\), but is continuous and locally Lipschitz, hence locally in \(W^{1,1}\). Away from that line its Jacobian is

Dg(x)= \begin{cases} \begin{pmatrix}1&0\\0&1\end{pmatrix},&x_1>x_2,\\[4pt] \begin{pmatrix}0&1\\1&0\end{pmatrix},&x_1<x_2. \end{cases}

The event \(X_1=X_2\) has probability zero. By symmetry, each ordering has probability \(1/2\), so \(E[Dg(X)^T]\) is the matrix whose four entries are \(1/2\).

For a direct check of the left side, set \(U=(X_1+X_2)/\sqrt{2}\) and \(V=(X_1-X_2)/\sqrt{2}\). These are independent standard normals, and \(\max(X_1,X_2)=(U+|V|)/\sqrt{2}\), \(\min(X_1,X_2)=(U-|V|)/\sqrt{2}\). For either \(k=1,2\), expanding the products and using \(E[U^2]=E[V^2]=1\), independence, and symmetry gives

E[X_k\max(X_1,X_2)]=\frac12,\qquad E[X_k\min(X_1,X_2)]=\frac12.

For instance, the first expectation with \(k=1\) is \(\tfrac12 E[(U+V)(U+|V|)]=\tfrac12\): the \(U^2\) term contributes \(1\), while \(E[U|V|]\), \(E[UV]\), and \(E[V|V|]\) are all zero. The other three expectations follow by the same expansion, with \(X_2=(U-V)/\sqrt{2}\) and the minus sign for the minimum. Thus every entry of \(E[Xg(X)^T]\) is \(1/2\), matching the derivative calculation.

Conditions, Interpretation, and a Common Pitfall

A weak derivative is an almost-everywhere object: it describes how a function changes through integration, not what derivative value should be assigned at every point. For a locally absolutely continuous function, changing the derivative’s value at a single kink does not alter its weak derivative or any expectation under a continuous Gaussian distribution. This is why the hinge and absolute-value examples fit the identity.

The integrability conditions matter just as much as the weak differentiability. A formal integration by parts can fail if the expectations do not exist or if a boundary term survives. The theorem gives transparent sufficient conditions; it does not assert that every function with an almost-everywhere derivative satisfies Stein’s identity.

Common mistake: “The function is differentiable except at one point, so I can substitute its derivative into Stein’s identity without checking anything else.”
Correction: Verify local absolute continuity (or the corresponding weak-derivative condition) and the required integrability. A single kink is allowed when the function remains absolutely continuous, but mere almost-everywhere differentiability alone is not the stated hypothesis.
Another mistake: Assigning special importance to the derivative’s value exactly at a kink. The weak derivative is defined only almost everywhere; changing it on a probability-zero set does not change the expectation.

Key takeaway: Stein’s Gaussian identity extends beyond smooth functions. Local absolute continuity or suitable local weak differentiability handles kinks, while integrability controls the expectations and eliminates boundary terms.

Check Your Understanding

Use the hypotheses as well as the formula when deciding whether an application is justified.

For \(X\sim N(\mu,\sigma^2)\), state the two integrability conditions in the scalar weak-derivative identity.
Why does the value assigned to \(g'\) at the kink of \(g(x)=(x-c)_+\) not affect the identity?
For \(X\sim N(0,1)\) and \(g(x)=|x|\), what are \(E[Xg(X)]\) and \(E[g'(X)]\)?
In the multivariate theorem, what does local \(W^{1,1}\) regularity provide along almost every coordinate line?
Why is almost-everywhere differentiability alone not enough to invoke the theorem?

Weak Derivatives in Stein’s Identity

What You'll Learn