Divergence Form of Stein’s Identity

From Coordinatewise Identities to Divergence

For a multivariate normal vector, Stein’s identity can be written as a sum of one-dimensional integration-by-parts identities, one for each coordinate. The vector-field form gathers those terms into a single expression: the inner product with the centered observation is paired with the divergence of the field. This notation is useful because it exposes exactly which derivatives contribute, without requiring the vector field to act separately on each coordinate.

Throughout, let \(X=(X_1,\ldots,X_p)^T\sim N_p(\mu,\sigma^2 I_p)\), where \(\mu=(\mu_1,\ldots,\mu_p)^T\), \(\sigma^2>0\), and \(I_p\) is the \(p\)-dimensional identity matrix. Let \(g:\mathbb R^p\to\mathbb R^p\) have component functions \(g_1,\ldots,g_p\). The dot product \((X-\mu)^Tg(X)\) is the sum of the coordinatewise products.

The trace selects the diagonal entries of the Jacobian: each component \(g_i\) is differentiated with respect to its own input coordinate \(x_i\). Derivatives such as \(\partial g_i/\partial x_j\) for \(i\ne j\) may be nonzero, but they do not appear in the divergence. This is not a claim that the field is coordinatewise; it is a specific sum of its own-coordinate derivatives.

Divergence Form of Stein’s Identity

Here is a sufficient set of conditions for the identity. Suppose each \(g_i\) is continuously differentiable, and, for every \(i\), the expectations \(E[|(X_i-\mu_i)g_i(X)|]\) and \(E[|\partial g_i(X)/\partial x_i|]\) are finite. Also suppose that for almost every fixed vector \(x_{-i}\) of the coordinates other than \(x_i\), the boundary terms \(g_i(x)\phi_i(x_i)\) tend to zero as \(x_i\to-\infty\) and \(x_i\to+\infty\), where \(\phi_i\) is the \(N(\mu_i,\sigma^2)\) density. These assumptions make the coordinatewise integrations by parts and the summation below valid.

Proof. The coordinates of \(X\) are independent, each with density $$ \phi_i(t)=\frac{1}{\sqrt{2\pi\sigma^2}} \exp\!\left(-\frac{(t-\mu_i)^2}{2\sigma^2}\right), \qquad \phi_i'(t)=-\frac{t-\mu_i}{\sigma^2}\phi_i(t). $$ Fix \(i\), and hold the other coordinates \(x_{-i}\) fixed at values for which the boundary condition holds. One-dimensional integration by parts gives $$ \int_{\mathbb R}(t-\mu_i)g_i(x)\phi_i(t)\,dt =-\sigma^2\int_{\mathbb R}g_i(x)\phi_i'(t)\,dt =\sigma^2\int_{\mathbb R}\frac{\partial g_i(x)}{\partial x_i}\phi_i(t)\,dt. $$ In the last equality, the boundary term \(-\sigma^2[g_i(x)\phi_i(t)]_{-\infty}^{\infty}\) is zero by assumption. Integrating over \(X_{-i}\) gives $$ E[(X_i-\mu_i)g_i(X)] =\sigma^2 E\!\left[\frac{\partial g_i(X)}{\partial x_i}\right]. $$ The stated absolute-integrability conditions allow these integrations and expectations to be taken as written. Summing over \(i=1,\ldots,p\), and using the definitions of dot product and divergence, yields $$ E\!\left[(X-\mu)^Tg(X)\right] =\sum_{i=1}^p E[(X_i-\mu_i)g_i(X)] =\sigma^2\sum_{i=1}^p E\!\left[\frac{\partial g_i(X)}{\partial x_i}\right] =\sigma^2 E[\operatorname{div}g(X)]. $$ \(\square\)

The proof shows how the vector formula relates to the coordinatewise identity: it adds the \(p\) scalar identities. The factor \(\sigma^2\) comes from differentiating the normal density, not from the definition of divergence. The regularity conditions are part of the result; the identity is not justified merely by writing down derivatives. More detailed classes of sufficient conditions require separate attention.

Calculating Divergence

For an affine field \(g(x)=Ax+b\), where \(A\) is a fixed \(p\times p\) matrix and \(b\) is a fixed vector, the Jacobian is constant and equals \(A\). Therefore its divergence is the trace of \(A\). This gives a useful check on both the derivative convention and the identity.

A particularly simple pattern occurs when each component depends only on its matching coordinate. If \(g_i(x)=h_i(x_i)\), then the Jacobian is diagonal and its \(i\)th diagonal entry is \(h_i'(x_i)\). Consequently, $$ \operatorname{div}g(x)=\sum_{i=1}^p h_i'(x_i). $$ No independence assumption about the functions is needed for this derivative calculation; the displayed formula follows from how the components are defined.

The divergence formula also applies when components depend on several coordinates. In that case, form the full Jacobian first, then select its diagonal. The next example makes explicit why “divergence” does not mean summing every entry of that matrix.

Why the Trace Matters

The divergence form compresses a collection of coordinate identities into a single geometric expression. In a calculation, the trace is often easier to inspect than a list of separate derivatives, especially when the Jacobian has a recognizable structure. It also makes clear that a field can couple coordinates while still having a simple divergence: only the diagonal of its Jacobian determines this quantity.

A frequent mistake is to sum all entries of \(J_g(x)\), or to use the determinant instead of the trace. The determinant describes a different feature of a matrix and is not the divergence. For the coupled field above, summing all four Jacobian entries would give \(3x_1+x_2\), whereas the divergence is \(x_2\). Another common error is to omit \(\sigma^2\) in Stein’s identity. The derivative of the \(N(\mu_i,\sigma^2)\) density includes the factor \(1/\sigma^2\), so integration by parts produces the multiplier \(\sigma^2\) on the divergence side.