Size-Bias Couplings for Nonnegative Sums

Why Size Biasing Is Useful

A nonnegative random variable contributes to an expectation in proportion to its size when it appears as a multiplicative weight. Size biasing turns that weighted expectation into an ordinary expectation under a new probability distribution. For sums of indicators, this new distribution has a direct construction: select an indicator with probability proportional to its mean, then arrange for it to equal one.

The resulting coupling can be inserted into the normal Stein equation. Its error is controlled by how much the size-biased sum differs from the original sum, and by how predictable that difference is after the original sum is known.

Definition and Basic Identity

Definition: Let $W$ be a nonnegative random variable with finite mean $\mu=\mathbb{E}W>0$. A random variable $W^s$ has the size-biased distribution of $W$ if, for every bounded measurable function $f$, $$ \mathbb{E}[Wf(W)]=\mu\,\mathbb{E}[f(W^s)]. $$ Equivalently, its probability distribution is given by $\mathbb{P}(W^s\in A)=\mathbb{E}[W\mathbf{1}_{\{W\in A\}}]/\mu$ for every measurable set $A$.

The definition weights outcomes of $W$ by their value. In particular, values near zero receive little weight, while larger values receive more. The condition $\mu>0$ is essential: if $\mu=0$ and $W\geq 0$, then $W=0$ almost surely, and the formula would divide by zero.

Taking $f\equiv 1$ verifies that the specified probabilities have total mass one. If $W$ has a finite second moment, taking $f(w)=w$ gives $\mathbb{E}W^s=\mathbb{E}W^2/\mu$. Consequently, $$ \mathbb{E}(W^s-W)=\frac{\operatorname{Var}(W)}{\mu}. $$ This determines the average size of the difference in any coupling of $W$ and $W^s$, though not its distribution or its dependence on $W$.

Worked Example: A Two-Point Distribution

Suppose $W=2$ with probability $1/3$, and $W=0$ with probability $2/3$. Then $\mu=2/3$. The size-biased distribution assigns probability $\mathbb{E}[W\mathbf{1}_{\{W=2\}}]/\mu=(2/3)/(2/3)=1$ to the value 2, and probability zero to 0. Thus $W^s=2$ almost surely.

For any bounded measurable $f$, the identity can be checked directly: $$ \mathbb{E}[Wf(W)] =\frac{1}{3}\,2f(2)+\frac{2}{3}\,0f(0) =\frac{2}{3}f(2) =\mu\,\mathbb{E}[f(W^s)]. $$ The size-biased version is not an independent copy of $W$; it has a different distribution, concentrated on the positive value.

Size Biasing a Sum of Indicators

Let $X_1,\ldots,X_n$ be indicators, with $p_i=\mathbb{P}(X_i=1)$, and set $W=\sum_{i=1}^n X_i$ and $\mu=\sum_{i=1}^n p_i>0$. Choose an index $I$ with $\mathbb{P}(I=i)=p_i/\mu$, omitting indices with $p_i=0$. Given $I=i$, construct a vector with the distribution of $(X_1,\ldots,X_n)$ conditional on $X_i=1$, and let $W^s$ be the sum of its coordinates. Then, for bounded measurable $f$, $$ \mathbb{E}f(W^s) =\sum_{i:p_i>0}\frac{p_i}{\mu}\mathbb{E}[f(W)\mid X_i=1] =\frac{1}{\mu}\sum_{i=1}^n\mathbb{E}[X_i f(W)] =\frac{1}{\mu}\mathbb{E}[Wf(W)]. $$ Thus $W^s$ has the size-biased distribution of $W$. The conditional vector in this construction may be coupled to the original vector in many ways; the choice of coupling affects the quality of an approximation bound.

Worked Example: Independent Bernoulli Indicators

Suppose $X_1,\ldots,X_n$ are independent Bernoulli variables with common success probability $p$, where $0<p<1$. Then $W=\sum_i X_i$ has mean $\mu=np$. Choose $I$ uniformly from $\{1,\ldots,n\}$, independently of the indicators. Define $X_I^s=1$, and leave every other coordinate unchanged. Conditional on $I=i$, independence means the unchanged coordinates have exactly their original joint distribution, while the $i$th coordinate is fixed at one. The resulting sum $W^s$ is size-biased.

In this coupling, $\Delta=W^s-W=1-X_I$, so $\Delta$ is either zero or one. In particular, $\mathbb{E}\Delta=1-p$. Since $\operatorname{Var}(W)=np(1-p)$, the general mean-difference identity gives the same value: $$ \frac{\operatorname{Var}(W)}{\mu} =\frac{np(1-p)}{np} =1-p =\mathbb{E}\Delta. $$ The coupling changes the sum only when the selected indicator was zero.

A Normal-Approximation Bound from the Coupling

For probability distributions on the real line, the Wasserstein distance $d_W$ is the supremum of the difference in expectations over all functions with Lipschitz constant at most one. Let $N$ be standard normal. We use the standard normal Stein equation: for a 1-Lipschitz test function $h$, a solution $f$ of $f'(x)-xf(x)=h(x)-\mathbb{E}h(N)$ can be chosen with $\lVert f''\rVert_\infty\leq 2$. This derivative bound is the input that converts the size-biased identity into a quantitative estimate.

Theorem: Let $W\geq 0$ have finite second moment, mean $\mu>0$, and variance $\sigma^2>0$. Let $(W,W^s)$ be any coupling for which $W^s$ has the size-biased distribution of $W$, and put $\Delta=W^s-W$. Then $$ d_W\!\left(\frac{W-\mu}{\sigma},N\right) \leq \frac{2\mu}{\sigma^2} \sqrt{\operatorname{Var}\!\left(\mathbb{E}[\Delta\mid W]\right)} +\frac{\mu}{\sigma^3}\mathbb{E}[\Delta^2]. $$

Proof. Set $Z=(W-\mu)/\sigma$. Fix a 1-Lipschitz test function $h$, and let $f$ solve its normal Stein equation with $\lVert f''\rVert_\infty\leq 2$. The size-biased identity, applied to $w\mapsto f((w-\mu)/\sigma)$, gives $$ \mathbb{E}[Zf(Z)] =\frac{\mu}{\sigma}\mathbb{E}\!\left[f\!\left(\frac{W^s-\mu}{\sigma}\right)-f(Z)\right]. $$ Taylor's formula with remainder yields $$ f\!\left(\frac{W^s-\mu}{\sigma}\right)-f(Z) =\frac{\Delta}{\sigma}f'(Z)+R, \qquad |R|\leq \frac{\lVert f''\rVert_\infty}{2\sigma^2}\Delta^2. $$ Writing $B=\mathbb{E}[\Delta\mid W]$, we therefore have $$ \mathbb{E}[h(Z)]-\mathbb{E}h(N) =\mathbb{E}\!\left[f'(Z)\left(1-\frac{\mu B}{\sigma^2}\right)\right] -\frac{\mu}{\sigma}\mathbb{E}R. $$ The size-biased identity implies $\mathbb{E}W^s=\mathbb{E}W^2/\mu$, so $\mathbb{E}B=\mathbb{E}\Delta=\sigma^2/\mu$. Hence the first term is a covariance: $$ \mathbb{E}\!\left[f'(Z)\left(1-\frac{\mu B}{\sigma^2}\right)\right] =-\operatorname{Cov}\!\left(f'(Z),\frac{\mu B}{\sigma^2}\right). $$ By Cauchy–Schwarz, its absolute value is at most $\sqrt{\operatorname{Var}(f'(Z))}\,(\mu/\sigma^2)\sqrt{\operatorname{Var}(B)}$. If $Z'$ is an independent copy of $Z$, then $$ \operatorname{Var}(f'(Z)) =\frac12\mathbb{E}\big[(f'(Z)-f'(Z'))^2\big] \leq\frac{\lVert f''\rVert_\infty^2}{2} \mathbb{E}\big[(Z-Z')^2\big] =\lVert f''\rVert_\infty^2, $$ because $\operatorname{Var}(Z)=1$. The remainder term has absolute value at most $\mu\lVert f''\rVert_\infty\mathbb{E}[\Delta^2]/(2\sigma^3)$. Using $\lVert f''\rVert_\infty\leq 2$, and then taking the supremum over $h$, proves the stated bound. $\square$

The first term is small when the conditional mean of the coupling difference, given $W$, varies little. The second is small when the coupling usually changes the sum by a small amount. Both features matter: a small average difference alone does not guarantee a strong normal approximation.

Worked Example: A Bound for an Independent Bernoulli Sum

Continue with the independent, identically distributed indicators above. Write $q=1-p$. The variance of $W$ is $\sigma^2=npq$, and $\Delta=1-X_I$. Given $W=k$, the selected index is uniform, so the conditional probability that $X_I=1$ is $k/n$. Therefore $$ \mathbb{E}[\Delta\mid W]=1-\frac{W}{n}, \qquad \operatorname{Var}\!\left(\mathbb{E}[\Delta\mid W]\right) =\frac{\operatorname{Var}(W)}{n^2} =\frac{pq}{n}. $$ Also $\Delta^2=\Delta$, giving $\mathbb{E}\Delta^2=q$. Substituting into the theorem, $$ d_W\!\left(\frac{W-np}{\sqrt{npq}},N\right) \leq \frac{2np}{npq}\sqrt{\frac{pq}{n}} +\frac{np}{(npq)^{3/2}}q = 2\sqrt{\frac{p}{nq}}+\frac{1}{\sqrt{npq}}. $$ For fixed $p$ strictly between zero and one, both terms tend to zero at rate $n^{-1/2}$. The calculation also shows why the conditional mean term can be sharper than controlling $\Delta$ by its raw variability alone.

A Dependent Indicator Example

Worked Example: Mutually Exclusive Indicators

Let $(X_1,X_2)$ equal $(0,0)$, $(1,0)$, or $(0,1)$ with probabilities $1/2$, $1/4$, and $1/4$, respectively. The indicators are dependent and cannot both be one. For $W=X_1+X_2$, we have $\mu=1/2$ and $\sigma^2=1/4$.

Each indicator has mean $1/4$, so the size-biased construction selects either index with probability $1/2$. Conditional on the selected indicator being one, the vector must be the corresponding singleton vector. Thus $W^s=1$ almost surely. Couple this $W^s$ with the original $W$; then $\Delta=1-W$, so $\mathbb{E}[\Delta\mid W]=1-W$ and $\operatorname{Var}(\mathbb{E}[\Delta\mid W])=\operatorname{Var}(W)=1/4$. Moreover, $\mathbb{E}\Delta^2=\mathbb{P}(W=0)=1/2$. The theorem gives $$ d_W\!\left(\frac{W-1/2}{1/2},N\right) \leq \frac{2(1/2)}{1/4}\sqrt{\frac14} +\frac{(1/2)(1/2)}{(1/2)^3} =2+2=4. $$ This bound is valid but not informative numerically; a bound need not be sharp for a small, non-normal example. The construction still illustrates how conditioning on a selected indicator produces the size-biased law even when the indicators are dependent.

Interpreting and Using the Bound

The size-biased identity is exact, while the normal-approximation bound is an inequality whose usefulness depends on the coupling. For indicator sums, the conditional construction gives a general way to obtain the correct marginal law of $W^s$, but it does not prescribe how the conditional vector should be joined to the original vector. A carefully chosen joint construction can make $\Delta$ small or make its conditional mean nearly constant.

A common pitfall is to use only $\mathbb{E}\Delta=\sigma^2/\mu$ and conclude that the approximation error is small. This expectation is fixed by the size-biased distribution, but it does not control either $\operatorname{Var}(\mathbb{E}[\Delta\mid W])$ or $\mathbb{E}\Delta^2$. Those are the quantities in the theorem. In applications, calculate them for the actual coupling rather than inferring them from the mean difference.

Another point is normalization: the theorem concerns $(W-\mu)/\sigma$, with $\sigma^2=\operatorname{Var}(W)>0$. The powers of $\sigma$ in the two error terms arise from scaling the difference $\Delta$ into the standardized variable. Keeping this scale explicit prevents errors when comparing bounds or studying a sequence of sums.

Check Your Understanding

Use the definitions and bound to check the following points.

Why must a nonnegative random variable with mean zero be excluded from the size-bias definition?
For an indicator sum, why should index $i$ be selected with probability $p_i/\mu$, rather than uniformly when the $p_i$ differ?
What identity determines $\mathbb{E}(W^s-W)$ in terms of the variance of $W$?
In the normal-approximation theorem, what aspects of the coupling are measured by the two error terms?
For independent Bernoulli indicators with common success probability $p$, why is $\mathbb{E}[\Delta\mid W]=1-W/n$?