Stein’s Identity for a Scalar Normal Variable

Why the Scalar Identity Matters

For a normal variable, integration against the centered value $X-\mu$ can be converted into integration against a derivative. This is useful because expectations involving a random deviation can then be expressed in terms of how a function changes. The precise identity is simple to state, but its validity depends on regularity and boundary behavior: integration by parts cannot be used without checking those details.

Let $X\sim N(\mu,\sigma^2)$, where $\sigma^2>0$ is the variance, and let $f$ be a real-valued function. The scalar identity relates $E[(X-\mu)f(X)]$ to $E[f'(X)]$. It applies, for example, to polynomials and exponential functions. We first state conditions that make the calculation rigorous, then verify the identity in concrete cases.

Definition: The density of $X\sim N(\mu,\sigma^2)$ is $p(x)=(\sigma\sqrt{2\pi})^{-1}\exp(-(x-\mu)^2/(2\sigma^2))$. A function $f:\mathbb{R}\to\mathbb{R}$ is locally absolutely continuous if its restriction to every bounded interval is absolutely continuous. In particular, it has an almost-everywhere derivative and satisfies the fundamental theorem of calculus on each bounded interval.

Stein’s Identity in One Dimension

The assumptions below ensure that both expectations exist, that integration by parts is valid on finite intervals, and that its boundary term disappears as the intervals expand to the real line. They are convenient sufficient conditions; the identity can hold under other, more general hypotheses as well.

Theorem (Stein’s identity for a scalar normal variable): Suppose $X\sim N(\mu,\sigma^2)$ with $\sigma^2>0$. Suppose $f$ is locally absolutely continuous and $E[|f'(X)|]<\infty$, $E[|X-\mu||f(X)|]<\infty$, and $\lim_{x\to-\infty}p(x)f(x)=\lim_{x\to\infty}p(x)f(x)=0$. Then $$ E[(X-\mu)f(X)]=\sigma^2E[f'(X)]. $$

Proof. Differentiating the normal density gives $p'(x)=-(x-\mu)p(x)/\sigma^2$. On any finite interval $[-R,R]$, local absolute continuity permits integration by parts:

\int_{-R}^{R}(x-\mu)f(x)p(x)\,dx =-\sigma^2\int_{-R}^{R}f(x)p'(x)\,dx =-\sigma^2[f(x)p(x)]_{-R}^{R} +\sigma^2\int_{-R}^{R}f'(x)p(x)\,dx.

As $R\to\infty$, the boundary term tends to zero by the assumed limits. The two integrands in the remaining integrals are absolutely integrable: their integrals of absolute values are the two finite expectations assumed in the theorem. Thus the integrals over $[-R,R]$ converge to their integrals over $\mathbb{R}$. Taking limits in the displayed equality gives the claimed identity. $\square$

The factor multiplying $E[f'(X)]$ is the variance $\sigma^2$, not the standard deviation $\sigma$. It comes directly from the derivative of the density. The conditions also explain why the result is not merely a formal rearrangement: the density-weighted boundary values must vanish, and both sides must be integrable.

A Recurrence for Centered Normal Moments

Stein’s identity immediately yields a useful consequence for powers of the centered variable. Write $Z=X-\mu$, so $Z\sim N(0,\sigma^2)$. Applying the identity to $f(x)=(x-\mu)^{k-1}$ gives a recurrence. These polynomial functions satisfy the theorem’s conditions: their derivatives and their products with $x-\mu$ have finite normal expectations, and a polynomial times the normal density tends to zero at both tails.

Proposition: For every integer $k\geq 2$, $$ E[(X-\mu)^k]=(k-1)\sigma^2E[(X-\mu)^{k-2}]. $$ In addition, $E[X-\mu]=0$.

Derivation. For $k\geq2$, set $f(x)=(x-\mu)^{k-1}$. Its derivative is $(k-1)(x-\mu)^{k-2}$. Substitution into Stein’s identity gives $$ E[(X-\mu)^k] =\sigma^2 E[(k-1)(X-\mu)^{k-2}] =(k-1)\sigma^2E[(X-\mu)^{k-2}]. $$ For $f(x)=1$, the identity gives $E[X-\mu]=\sigma^2E[0]=0$. This establishes both claims. $\square$

For example, $E[(X-\mu)^2]=\sigma^2$, since the recurrence at $k=2$ reduces to $E[(X-\mu)^2]=\sigma^2E[1]$. At $k=3$, it gives $E[(X-\mu)^3]=2\sigma^2E[X-\mu]=0$. The recurrence is a consequence of the identity, not an additional assumption about moments.

Worked Example: A Quadratic Polynomial

Let $X\sim N(1,4)$, and take $f(x)=x^2$. The derivative is $f'(x)=2x$. We calculate each side of the identity, using $Z=X-1$, for which $E[Z]=0$ and $E[Z^2]=4$.

For the left side, $X=1+Z$, so $$ E[(X-1)X^2] =E[Z(1+Z)^2] =E[Z+2Z^2+Z^3] =0+2(4)+0 =8. $$ The third centered moment is zero by the recurrence above.

For the right side, $$ \sigma^2E[f'(X)] =4E[2X] =8E[X] =8(1) =8. $$ Thus both sides equal $8$. The conditions hold because a polynomial and its derivative are integrable under a normal density, and a polynomial multiplied by that density vanishes at both tails.

Worked Example: A Cubic Polynomial

Let $X\sim N(1,1)$ and $f(x)=x^3$, so $f'(x)=3x^2$. With $Z=X-1$, the recurrence gives $E[Z]=0$, $E[Z^2]=1$, and $E[Z^3]=0$.

The left side is $$ E[(X-1)X^3] =E[Z(1+Z)^3] =E[Z+3Z^2+3Z^3+Z^4]. $$ Using the recurrence at $k=4$, $E[Z^4]=3E[Z^2]=3$. Therefore the left side is $0+3(1)+3(0)+3=6$.

The right side is $$ \sigma^2E[f'(X)] =1\cdot 3E[X^2] =3E[(1+Z)^2] =3\{1+2E[Z]+E[Z^2]\} =3(1+0+1) =6. $$ The two sides agree. Again, the polynomial growth ensures the required integrability and boundary behavior.

Exponential Test Functions

Exponentials provide a different check because their growth is not polynomial. For any fixed real $t$, the function $f(x)=e^{tx}$ satisfies the theorem’s conditions under a normal distribution. To see why, the normal density has a negative quadratic exponent, which dominates the linear term $tx$ as $x$ tends to either tail. The same fact ensures the expectations of $e^{tX}$ and $|X|e^{tX}$ are finite.

Worked Example: An Exponential Function

Let $X\sim N(0,4)$, and choose $f(x)=e^{x/2}$, so $t=1/2$ and $f'(x)=\tfrac12 e^{x/2}$. The normal moment-generating function, obtained by completing the square in the density integral, is $$ M_X(t)=E[e^{tX}]=\exp\left(\frac{\sigma^2t^2}{2}\right) =\exp(2t^2). $$ At $t=1/2$, this gives $E[e^{X/2}]=e^{1/2}$.

To compute the left side, differentiate the moment-generating function: $M_X'(t)=E[Xe^{tX}]=4t\exp(2t^2)$. Differentiation under the integral is justified because, for $t$ in a bounded interval around $1/2$, the integrand and its derivative are bounded in absolute value by a constant times $(1+|X|)e^{T|X|}$ for some finite $T$, which is integrable under a normal distribution. Hence $$ E[Xe^{X/2}] =M_X'(1/2) =2e^{1/2}. $$ Since $\mu=0$, this is the left side. The right side is $$ \sigma^2E[f'(X)] =4\cdot\frac12 E[e^{X/2}] =2e^{1/2}. $$ Both sides are $2e^{1/2}$, as required.

Common Pitfalls and Practical Use

A frequent mistake is to replace $\sigma^2$ by $\sigma$. For $X\sim N(\mu,\sigma^2)$, the density derivative is $p'(x)=-(x-\mu)p(x)/\sigma^2$, so rearranging the integration-by-parts expression introduces $\sigma^2$, not $\sigma$. For instance, in the quadratic example the correct right side is $4E[2X]=8$; using $\sigma=2$ instead would give $2E[2X]=4$, which does not match the left side.

Another error is to assume that differentiability alone makes the identity valid. The proof uses local absolute continuity, integrability of both sides, and a vanishing boundary term. For the polynomial and exponential examples these conditions can be checked from the Gaussian tail, but a function with sufficiently rapid growth need not satisfy them. Always check the hypotheses before applying the formula.

The identity is important because it turns a centered normal factor into a derivative inside an expectation. In later calculations, that conversion can simplify expressions involving functions of normal observations. Here the essential point is its scalar form and its justification; the examples show that the same formula handles both polynomial and exponential test functions when the conditions are met.

Check Your Understanding

Use the assumptions and calculations above to answer the following questions.

For $X\sim N(\mu,\sigma^2)$, what derivative of the density leads to the factor $\sigma^2$ in Stein’s identity?
State the boundary condition used in the theorem, and explain where it enters the proof.
If $X\sim N(0,9)$ and $f(x)=x$, calculate both sides of the identity.
Use the moment recurrence to find $E[(X-\mu)^4]$ in terms of $\sigma^2$.
Why does $f(x)=e^{tx}$ satisfy the required tail conditions for any fixed real $t$ under a normal density?