Zero-Bias Transformations of Centered Variables

Why Zero-Biasing Is Useful

A centered random variable need not be close to normal. But the normal distribution is characterized by a derivative identity, and zero-biasing gives a way to express the corresponding identity for any centered variable with finite, positive variance. If a zero-biased version can be coupled near the original variable, that identity turns directly into a normal-approximation bound.

The transformation differs from size-biasing: it applies to centered real-valued variables, including variables that take both positive and negative values. Its defining identity also specifies the transformed law, so there is no need to assume that the original variable has a density.

Definition and Existence

Definition: Let $X$ be a real-valued random variable with $\mathbb{E}X=0$ and $0<\mathbb{E}X^2=\sigma^2<\infty$. A random variable $X^*$ has the zero-biased distribution of $X$ if, for every continuously differentiable function $f:\mathbb{R}\to\mathbb{R}$ with bounded derivative for which the expectations exist, $$ \mathbb{E}[Xf(X)]=\sigma^2\mathbb{E}[f'(X^*)]. $$

This test-function class is chosen to avoid ambiguity about derivatives at atoms. In particular, the definition uses continuously differentiable functions, not functions whose derivative is specified only almost everywhere. Since $f'$ is bounded, $f$ grows at most linearly, so $\mathbb{E}|Xf(X)|<\infty$ follows from the finite second moment.

Theorem: Every centered real-valued $X$ with $0<\sigma^2=\mathbb{E}X^2<\infty$ has a zero-biased distribution. One version of its density, with respect to Lebesgue measure, is $$ g(w)= \begin{cases} -\dfrac{\mathbb{E}[X\mathbf{1}_{\{X\leq w\}}]}{\sigma^2},&w<0,\\[6pt] \dfrac{\mathbb{E}[X\mathbf{1}_{\{X>w\}}]}{\sigma^2},&w\geq 0. \end{cases} $$

Proof. For $w<0$, every value of $X$ on $\{X\leq w\}$ is nonpositive, so the first numerator with its minus sign is nonnegative. For $w\geq0$, $X$ is positive on $\{X>w\}$, so the second numerator is nonnegative. These expressions define a measurable function. Tonelli’s theorem gives $$ \int_{-\infty}^0-\mathbb{E}[X\mathbf{1}_{\{X\leq w\}}]\,dw =\mathbb{E}[X^2\mathbf{1}_{\{X<0\}}], \qquad \int_0^\infty\mathbb{E}[X\mathbf{1}_{\{X>w\}}]\,dw =\mathbb{E}[X^2\mathbf{1}_{\{X>0\}}]. $$ For the first equality, a fixed value $X=x<0$ contributes $-x$ for $w\in[x,0)$, an interval of length $-x$; its total contribution is $x^2$. For the second, $x>0$ contributes $x$ for $w\in[0,x)$, giving $x^2$. Thus $\int_{\mathbb{R}}g(w)\,dw=1$, since $\mathbb{P}(X=0)$ contributes zero to the second moment. Hence $g$ is a probability density. It remains to verify the identity. For $x>0$, $xf(x)-xf(0)=x\int_0^x f'(w)\,dw$. For $x<0$, the same expression is $-x\int_x^0 f'(w)\,dw$. Taking expectations and interchanging the integrals gives $$ \mathbb{E}[Xf(X)] =\int_0^\infty f'(w)\mathbb{E}[X\mathbf{1}_{\{X>w\}}]\,dw +\int_{-\infty}^0 f'(w)\bigl(-\mathbb{E}[X\mathbf{1}_{\{X\leq w\}}]\bigr)\,dw. $$ The interchange is justified because the absolute value of the integrand before taking expectations is bounded in total by $\|f'\|_\infty\mathbb{E}X^2$. The right side is $\sigma^2\int f'(w)g(w)\,dw$, which is the required identity for $X^*$ with density $g$. $\square$

At $w=0$, the two expressions suggested by the positive- and negative-side formulas agree because $X\mathbf{1}_{\{X=0\}}=0$. The particular value assigned to a density at one point does not affect its probability law. More generally, density formulas may differ at individual points without defining different distributions.

Uniqueness of the Zero-Biased Law

Theorem: The zero-biased distribution is unique among probability laws satisfying the defining identity for all continuously differentiable functions with bounded derivative.

Proof. Suppose probability measures $\mu$ and $\nu$ both satisfy the identity. Let $h:\mathbb{R}\to\mathbb{R}$ be any continuous function with compact support, and set $$ f(x)=\int_{-\infty}^{x}h(t)\,dt. $$ Then $f$ is continuously differentiable, $f'=h$ is bounded, and $f$ is bounded because $h$ has compact support. Applying the defining identity to each candidate law gives $$ \int h\,d\mu =\frac{\mathbb{E}[Xf(X)]}{\sigma^2} =\int h\,d\nu. $$ Finite Borel measures on $\mathbb{R}$ that agree on all continuous compactly supported functions are equal. Therefore $\mu=\nu$. $\square$

This argument identifies the whole law without testing a function whose derivative has an exceptional point. That distinction matters when a candidate zero-biased variable might have an atom at that point.

Worked Example: A Centered Bernoulli Variable

Let $B$ be Bernoulli with $\mathbb{P}(B=1)=p$, where $0<p<1$, and put $X=B-p$. Write $q=1-p$. Then $X=q$ with probability $p$, and $X=-p$ with probability $q$. Its mean is $pq-qp=0$, and its variance is $pq^2+qp^2=pq$.

The density formula gives $g(w)=1$ for $-p<w<q$, and zero outside that interval, apart from possible endpoint values that do not affect the law. Indeed, when $-p<w<0$, $-\mathbb{E}[X\mathbf{1}_{\{X\leq w\}}]=pq$; when $0\leq w<q$, $\mathbb{E}[X\mathbf{1}_{\{X>w\}}]=pq$. Dividing by $\sigma^2=pq$ gives one in either interval. At $w=-p$, the strict event in the positive-side formula has probability $p$, so its numerator is $pq$ and the density value is one; this single endpoint value does not alter the uniform law.

Thus $X^*$ is uniform on $[-p,q]$. Directly, for any continuously differentiable $f$ with bounded derivative, $$ \mathbb{E}[Xf(X)] =pqf(q)-pqf(-p) =pq\int_{-p}^{q}f'(w)\,dw =pq\,\mathbb{E}[f'(X^*)]. $$ The interval has length $q+p=1$, so the uniform expectation in the last equality is exactly the integral over the interval.

Worked Example: A Symmetric Two-Point Variable

Let $X$ take the values $1$ and $-1$ with probability $1/2$ each. It is centered with variance one. For $-1<w<0$, the negative-side numerator is $-\mathbb{E}[X\mathbf{1}_{\{X\leq w\}}]=1/2$; for $0\leq w<1$, the positive-side numerator is $\mathbb{E}[X\mathbf{1}_{\{X>w\}}]=1/2$. The density is therefore $1/2$ on $(-1,1)$, so $X^*$ is uniform on $[-1,1]$.

To check the identity independently, the left side is $$ \mathbb{E}[Xf(X)]=\frac{f(1)-f(-1)}{2}. $$ For $U$ uniform on $[-1,1]$, the fundamental theorem of calculus gives $$ \mathbb{E}[f'(U)] =\frac12\int_{-1}^1 f'(w)\,dw =\frac{f(1)-f(-1)}2. $$ The two sides agree for every test function in the definition.

Zero-Biasing Independent Sums

Worked Example: A Zero-Biased Version of an Independent Sum

Let $X_1,\ldots,X_n$ be independent, centered variables with finite variances $\sigma_i^2$, and suppose $s^2=\sum_{i=1}^n\sigma_i^2>0$. Write $W=\sum_iX_i$. For each $i$ with $\sigma_i^2>0$, let $X_i^*$ have the zero-biased law of $X_i$. Choose an index $I$ independently with $\mathbb{P}(I=i)=\sigma_i^2/s^2$. In the construction, take the selected $X_I^*$ independent of all the original summands, and set $W^*=\sum_{j\ne I}X_j+X_I^*$. Indices of zero variance have probability zero and need no zero-biased variable.

For a continuously differentiable $f$ with bounded derivative, independence and $\mathbb{E}X_i=0$ yield $$ \mathbb{E}[Wf(W)] =\sum_i\mathbb{E}[X_i f(W)] =\sum_i \sigma_i^2\mathbb{E}\!\left[f'\!\left(\sum_{j\ne i}X_j+X_i^*\right)\right]. $$ For each term, condition on the other summands and apply the zero-bias identity for $X_i$ to the function $x\mapsto f(\sum_{j\ne i}X_j+x)$. By the definition of $I$, the last sum equals $s^2\mathbb{E}[f'(W^*)]$. Thus $W^*$ has the zero-biased law of $W$, whose variance is $s^2$.

A Coupling Bound for Normal Approximation

Let $Z$ have the standard normal distribution. The standard normal Stein equation for a Lipschitz test function $h$ is $$ f_h'(z)-zf_h(z)=h(z)-\mathbb{E}h(Z). $$ The normal Stein derivative bound gives a solution with $\|f_h''\|_\infty\leq 2\|h'\|_\infty$. This is the derivative estimate used in the Wasserstein bounds from Stein’s method.

Theorem: Let $X$ be centered with $0<\sigma^2=\mathbb{E}X^2<\infty$, and let $(X,X^*)$ be any coupling whose marginals are $X$ and its zero-biased law. If $\mathbb{E}|X-X^*|<\infty$, then $$ d_{\mathrm W}\!\left(\frac{X}{\sigma},Z\right) \leq \frac{2}{\sigma}\mathbb{E}|X-X^*|, $$ where $d_{\mathrm W}(U,V)=\sup_{\|h'\|_\infty\leq1}|\mathbb{E}h(U)-\mathbb{E}h(V)|$, with the supremum over Lipschitz functions.

Proof. Fix a Lipschitz $h$ with $\|h'\|_\infty\leq1$, and let $f_h$ solve the displayed Stein equation. The zero-bias identity applied to $x\mapsto f_h(x/\sigma)$ gives $$ \mathbb{E}\!\left[\frac{X}{\sigma}f_h\!\left(\frac{X}{\sigma}\right)\right] =\mathbb{E}\!\left[f_h'\!\left(\frac{X^*}{\sigma}\right)\right]. $$ Using the Stein equation and $\mathbb{E}h(Z)$ as the normal target, we obtain $$ \left|\mathbb{E}h\!\left(\frac{X}{\sigma}\right)-\mathbb{E}h(Z)\right| =\left|\mathbb{E}\!\left[f_h'\!\left(\frac{X}{\sigma}\right)-f_h'\!\left(\frac{X^*}{\sigma}\right)\right]\right|. $$ The derivative bound makes $f_h'$ Lipschitz with constant at most $2$. For the given coupling, therefore, $$ \left|\mathbb{E}h\!\left(\frac{X}{\sigma}\right)-\mathbb{E}h(Z)\right| \leq \frac{2}{\sigma}\mathbb{E}|X-X^*|. $$ Taking the supremum over such $h$ proves the bound. $\square$

Worked Example: A Normal Bound for Rademacher Sums

Let $X_1,\ldots,X_n$ be independent, each taking $1$ and $-1$ with probability $1/2$, and let $W=\sum_iX_i$. Then $\operatorname{Var}(W)=n$. Each summand has a zero-biased version $U$ uniform on $[-1,1]$, as shown above. The sum construction selects $I$ uniformly from $\{1,\ldots,n\}$ and replaces $X_I$ by an independent $U$. Consequently $W^*-W=U-X_I$.

Conditional on $X_I=1$, $|U-X_I|=1-U$ throughout $[-1,1]$, so its expectation is one. Conditional on $X_I=-1$, $|U-X_I|=U+1$, also with expectation one. Hence $\mathbb{E}|W^*-W|=1$. Applying the coupling bound with $\sigma=\sqrt n$ gives $$ d_{\mathrm W}\!\left(\frac{W}{\sqrt n},Z\right)\leq\frac{2}{\sqrt n}. $$ This is an explicit bound from a concrete coupling; it does not claim that the constant is optimal.

Interpretation and a Common Pitfall

The bound depends on the average distance in a coupling, not merely on the separate distributions of $X$ and $X^*$. A good construction makes the replacement $X\mapsto X^*$ small on average. The independent-sum construction illustrates the mechanism: choose a summand with probability proportional to its variance, then replace only that summand.

A common normalization error is to compare $X$ directly with a standard normal when $X$ has variance $\sigma^2$. The Stein identity first gives a comparison for $X/\sigma$; the factor $1/\sigma$ in the coupling bound follows from scaling the replacement distance. Also, positive variance is essential: if $\sigma^2=0$, the defining identity cannot be normalized to specify a zero-biased probability law.

Check Your Understanding

Try to answer these questions using the definition and derivations above.

Why does the zero-bias identity require $\sigma^2>0$, and what property of $X$ makes the density construction nonnegative?
Why is a continuously differentiable test-function class useful when proving uniqueness?
For a centered Bernoulli variable $B-p$, what is the support and density of its zero-biased law?
In the independent-sum construction, why is the replacement index selected in proportion to the summands’ variances?
Where does the factor $1/\sigma$ enter the Wasserstein coupling bound?

Zero-Bias Transformations of Centered Variables

What You'll Learn

Why Zero-Biasing Is Useful

Definition and Existence

Uniqueness of the Zero-Biased Law

Worked Example: A Centered Bernoulli Variable

Worked Example: A Centered Bernoulli Variable

Worked Example: A Symmetric Two-Point Variable

Worked Example: A Symmetric Two-Point Variable

Zero-Biasing Independent Sums

Worked Example: A Zero-Biased Version of an Independent Sum

A Coupling Bound for Normal Approximation

Worked Example: A Normal Bound for Rademacher Sums

Worked Example: A Normal Bound for Rademacher Sums

Interpretation and a Common Pitfall

Check Your Understanding

Previous

Next