Density of Bounded Random Variables in Lp

Why Clipping Is a Useful Approximation

An \(L^p\) random variable need not be bounded. It may take very large values, provided those values are sufficiently rare for its \(p\)-th moment to remain finite. This raises a natural question: can an unbounded variable be approximated, in the \(L^p\) norm, by variables that are bounded?

The basic method is to clip values beyond a chosen level. As the level increases, the clipped variable retains more of the original variable. The key point is that the difference is small in \(L^p\), not necessarily that the original variable is bounded or that the clipping error vanishes uniformly over the sample space.

Definition of Clipping

Work on a probability space \((\Omega,\mathcal F,\mathbb P)\), and let \(X\) be a real-valued random variable. For a level \(a>0\), define the clipping function \(c_a:\mathbb R\to\mathbb R\) by

c_a(x)= \begin{cases} -a, & x<-a,\\ x, & -a\le x\le a,\\ a, & x>a. \end{cases}

The clipped random variable is \(X^{(a)}=c_a(X)\). Equivalently, \(X^{(a)}=\max(-a,\min(X,a))\). Since \(c_a\) is continuous and \(|c_a(x)|\le a\) for every real \(x\), \(X^{(a)}\) is measurable and bounded.

Definition: For \(n\in\mathbb N\), the clipping of \(X\) at level \(n\) is \(X_n=c_n(X)\). Thus \(X_n=X\) whenever \(|X|\le n\), and otherwise \(X_n\) has the same sign as \(X\) and absolute value \(n\).

For \(1\le p<\infty\), the \(L^p\) norm is \(\|Y\|_p=(\mathbb E|Y|^p)^{1/p}\) when \(\mathbb E|Y|^p<\infty\). As usual, \(L^p\) identifies random variables that are equal almost surely. Changing a representative on a null set does not change the \(L^p\) class or any norm calculation below.

The Clipping Error

The error has a simple pointwise description. If \(|X|\le n\), clipping changes nothing. If \(|X|>n\), the clipped value is \(n\) units from zero in the direction of \(X\), so the error is \(|X|-n\). In particular,

|X-X_n|=(|X|-n)_+ \le |X|\,\mathbf 1_{\{|X|>n\}},

where \(u_+=\max(u,0)\), and \(\mathbf 1_A\) denotes the indicator of an event \(A\). The inequality follows by considering the two cases: both sides are zero when \(|X|\le n\), and \(|X|-n\le |X|\) when \(|X|>n\).

Theorem: If \(1\le p<\infty\) and \(X\in L^p\), then its clippings \(X_n\) satisfy \(\|X-X_n\|_p\to 0\) as \(n\to\infty\).

Proof: For every \(\omega\in\Omega\), the value \(X(\omega)\) is finite, so \(X_n(\omega)=X(\omega)\) for all integers \(n\ge |X(\omega)|\). Therefore \(|X-X_n|^p\to0\) pointwise. The error inequality gives

0\le |X-X_n|^p \le |X|^p\mathbf 1_{\{|X|>n\}} \le |X|^p.

The dominating random variable \(|X|^p\) is integrable because \(X\in L^p\). The dominated convergence theorem therefore gives \(\mathbb E|X-X_n|^p\to0\). Taking the \(p\)-th root proves \(\|X-X_n\|_p\to0\).

\(\square\)

This argument does not require \(X_n\) to approach \(X\) uniformly over \(\Omega\). The error may remain large on some outcomes at each fixed level. What matters is that the \(p\)-th power of the error is controlled by the integrable variable \(|X|^p\), and that the error tends to zero outcome by outcome.

Density of Bounded Random Variables

A subset \(D\) of a normed space is dense if every element of the space can be approximated arbitrarily closely in norm by elements of \(D\). Here, the relevant subset consists of the \(L^p\) classes that have bounded representatives. Every clipping \(X_n\) belongs to this subset, since \(|X_n|\le n\).

Corollary: For \(1\le p<\infty\), bounded random variables are dense in \(L^p\).

Proof: Let \(X\in L^p\) and let \(\varepsilon>0\). By the theorem, there is an integer \(n\) large enough that \(\|X-X_n\|_p<\varepsilon\). The variable \(X_n\) is bounded by \(n\), so it is a bounded \(L^p\) random variable within \(\varepsilon\) of \(X\). Since \(X\) and \(\varepsilon\) were arbitrary, the bounded random variables are dense in \(L^p\).

\(\square\)

The proof gives more than an existence statement: it specifies an approximation that can be used for any \(X\). One does not need to search for a separate bounded variable for each desired accuracy. Raising the clipping level produces a sequence of bounded approximants with \(L^p\) error tending to zero.

Worked Examples

Worked Example: A Finite-Valued Random Variable

Suppose \(\mathbb P(X=2)=1/3\) and \(\mathbb P(X=-4)=2/3\). Clip at level \(1\). Then \(X_1=1\) on \(\{X=2\}\) and \(X_1=-1\) on \(\{X=-4\}\). The respective absolute errors are \(1\) and \(3\). For any \(1\le p<\infty\),

\|X-X_1\|_p^p =\frac13|2-1|^p+\frac23|-4-(-1)|^p =\frac13+\frac23\,3^p.

In particular, for \(p=2\), the squared error norm is \(1/3+(2/3)9=19/3\), so \(\|X-X_1\|_2=\sqrt{19/3}\). At every integer level \(n\ge4\), \(X_n=X\) at both possible values, and the error is exactly zero. This finite example illustrates the pointwise error calculation; for an unbounded variable, the errors need not become exactly zero at any finite level.

Worked Example: An Exponential Random Variable

Let \(X\) have the exponential distribution with density \(e^{-x}\) on \(x\ge0\), and use \(1\le p<\infty\). Since \(X\ge0\), clipping at level \(n\) is \(X_n=\min(X,n)\). The error is zero for \(X\le n\), and equals \(X-n\) for \(X>n\). Thus

\mathbb E|X-X_n|^p =\int_n^\infty (x-n)^p e^{-x}\,dx.

Set \(u=x-n\), so \(x=u+n\) and \(dx=du\). The lower endpoint becomes \(u=0\), giving

\mathbb E|X-X_n|^p =e^{-n}\int_0^\infty u^p e^{-u}\,du =e^{-n}\Gamma(p+1).

The integral is finite, and therefore \(X\in L^p\); in fact \(\mathbb E X^p=\Gamma(p+1)\). The clipping error is exactly \(\|X-X_n\|_p=e^{-n/p}\Gamma(p+1)^{1/p}\), which tends to zero as \(n\to\infty\).

Worked Example: A Pareto Variable with a Polynomial Error Rate

Let \(X\) have density \(3x^{-4}\) on \(x\ge1\), and zero elsewhere. Its second moment is

\mathbb E X^2=3\int_1^\infty x^{-2}\,dx=3,

so \(X\in L^2\). For an integer \(n\ge1\), \(X\) is nonnegative, and the clipping error is \(X-n\) when \(X>n\), zero otherwise. Consequently,

\mathbb E|X-X_n|^2 =3\int_n^\infty (x-n)^2x^{-4}\,dx =3n^{-1}\int_1^\infty (y-1)^2y^{-4}\,dy,

where the last equality uses \(x=ny\). Expanding the integrand gives \((y-1)^2y^{-4}=y^{-2}-2y^{-3}+y^{-4}\). The three integrals over \([1,\infty)\) are \(1\), \(1/2\), and \(1/3\), respectively. Hence the integral is \(1-2(1/2)+1/3=1/3\), and

\mathbb E|X-X_n|^2=\frac1n, \qquad \|X-X_n\|_2=\frac1{\sqrt n}.

This example has a slower, polynomial error rate than the exponential example, but the error still tends to zero. The moment assumption matters: for this density, \(\mathbb E X^p=3\int_1^\infty x^{p-4}\,dx\) is finite exactly when \(p<3\). The theorem applies only for those \(p\) for which \(X\) actually belongs to \(L^p\).

What the Result Does and Does Not Say

The theorem is about approximation of one fixed \(L^p\) variable. It does not say that the clipping errors are small uniformly over all variables in an \(L^p\) ball. The level needed for a given accuracy can depend on the distribution of \(X\). Nor does the proof require a separate estimate on the probability of the clipping event: integrability of \(|X|^p\) and dominated convergence already control the error.

There is also a distinction between finite \(p\) and \(p=\infty\). Every \(L^\infty\) variable is essentially bounded, so bounded random variables already represent all of \(L^\infty\). Moreover, for \(X\in L^\infty\), if \(n\) exceeds its essential bound, then \(X_n=X\) almost surely. The substantive density statement is for finite \(p\), where \(L^p\) can contain genuinely unbounded variables.

Clipping is useful because it reduces arguments about an unbounded variable to arguments about bounded approximants, while retaining quantitative control in the \(L^p\) norm. It is important, however, to use the norm relevant to the claim. Small \(L^p\) error is an averaged statement; it does not assert that the maximum pointwise error is small.

Check Your Understanding

Use the definition of clipping and the error estimate to answer these questions.

For a real number \(x\), what is \(|x-c_n(x)|\) in the cases \(|x|\le n\) and \(|x|>n\)?
Why is \(|X|^p\) a valid dominating random variable when \(X\in L^p\)?
What property of \(X_n\) makes it an admissible bounded approximant to \(X\)?
For the exponential example, why does the substitution \(u=x-n\) produce a factor \(e^{-n}\)?
For the Pareto density \(3x^{-4}\), for which values of \(p\ge1\) does the moment \(\mathbb E X^p\) exist?