Tutorials › Stochastic Processes › Lp Convergence and Moment Error

Home  /  Tutorials  /  Convergence of Random Variables

Convergence of Random Variables · Tutorial 48 of 994

Lp Convergence and Moment Error

Learn to measure approximation error in \(L^p\) and calculate it for constants, Gaussian variables, and rare large values.

Intermediate 9 min read

What You'll Learn

  • Define \(L^p\) convergence using the expected absolute \(p\)th-power error.
  • Distinguish the \(p\)th-power error from its \(p\)th root, the \(L^p\) norm.
  • Calculate \(L^p\) errors for deterministic sequences and scaled Gaussian variables.
  • Find the exact error for a random variable supported on an event.
  • See how the same scaled-indicator sequence can converge for one \(p\) and fail for another.
  • Apply scalar multiplication and addition to \(L^p\)-convergent sequences.

Measuring Error in \(L^p\)

Convergence of random variables can be measured by asking how large their errors are on average. For a chosen exponent \(p\), the quantity \(\mathbb{E}|X_n-X|^p\) gives extra weight to large errors as \(p\) increases. The resulting notion, \(L^p\) convergence, is useful when the size of the error matters—not just whether an error occurs.

Throughout, let \(X,X_1,X_2,\ldots\) be real-valued random variables on the same probability space \((\Omega,\mathcal{F},\mathbb{P})\). Keeping them on the same space makes the difference \(X_n-X\) well-defined. Fix a finite exponent \(p\ge1\). A random variable \(Y\) has a finite \(p\)th absolute moment if \(\mathbb{E}|Y|^p<\infty\).

Definition: For \(p\ge1\), write \(\|Y\|_p=(\mathbb{E}|Y|^p)^{1/p}\) when \(\mathbb{E}|Y|^p<\infty\). The sequence \((X_n)\) converges to \(X\) in \(L^p\) if \(\mathbb{E}|X_n-X|^p\to0\), or equivalently if \(\|X_n-X\|_p\to0\). In this definition, the differences must have finite \(p\)th moments.

The expectation inside the definition is the moment error; its \(p\)th root is the \(L^p\) error or norm of the difference. Both describe the same convergence because taking a \(p\)th root preserves convergence to zero. For \(p=1\), the error is the mean absolute difference. For \(p=2\), the norm is the root-mean-square difference. In particular, if \(X_n\) and \(X\) have finite second moments, then \(\|X_n-X\|_2^2=\mathbb{E}(X_n-X)^2\).

The exponent affects how strongly large errors count. At a fixed event, an error of size \(c\) contributes \(c^p\) times the event's probability to the moment error. Thus a small probability does not by itself guarantee a small \(L^p\) error: the magnitude of the error matters as well.

Three Basic Calculations

Worked Example: A Deterministic Sequence

Regard each number as a constant random variable on any probability space. Let \(X_n=3+\tfrac{2}{n}\) and \(X=3\). Find the \(L^p\) error for a fixed \(p\ge1\).

For every outcome \(\omega\), the difference is \(X_n(\omega)-X(\omega)=\tfrac{2}{n}\). Its absolute value is constant, so taking the expectation leaves it unchanged after raising it to the \(p\)th power:

$$ \mathbb{E}|X_n-X|^p =\mathbb{E}\left(\frac{2}{n}\right)^p =\left(\frac{2}{n}\right)^p, \qquad \|X_n-X\|_p=\frac{2}{n}. $$

Since \(2/n\to0\), the sequence converges to \(3\) in \(L^p\) for every finite \(p\ge1\). For instance, at \(n=4\) the pointwise error is \(2/4=1/2\), the moment error is \((1/2)^p\), and its \(p\)th root is \(1/2\). This check also shows why the moment error and the norm error are different quantities.

Worked Example: Scaled Standard Gaussian Variables

Let \(Z\) have the standard normal distribution on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), and set \(X_n=(1+\tfrac1n)Z\) and \(X=Z\). Compute the errors for \(p=1\) and \(p=2\), and give the formula for general finite \(p\ge1\).

The difference uses the same \(Z\) in both variables: \(X_n-X=\tfrac1n Z\). Hence \(\mathbb{E}|X_n-X|^p=n^{-p}\mathbb{E}|Z|^p\). To calculate the Gaussian moment, use the standard normal density and symmetry:

$$ \mathbb{E}|Z|^p =\frac{2}{\sqrt{2\pi}}\int_0^\infty x^p e^{-x^2/2}\,dx =\frac{2^{p/2}}{\sqrt{\pi}}\, \Gamma\left(\frac{p+1}{2}\right), $$

where \(\Gamma(a)=\int_0^\infty u^{a-1}e^{-u}\,du\) for \(a>0\). For the second equality, substitute \(u=x^2/2\); then \(x^p\,dx=2^{(p-1)/2}u^{(p-1)/2}\,du\). Therefore

$$ \mathbb{E}|X_n-X|^p =\frac{2^{p/2}}{\sqrt{\pi}}\, \Gamma\left(\frac{p+1}{2}\right)n^{-p}, \qquad \|X_n-X\|_p =\left(\frac{2^{p/2}}{\sqrt{\pi}}\, \Gamma\left(\frac{p+1}{2}\right)\right)^{1/p}\frac1n. $$

At \(p=1\), the moment is \(\mathbb{E}|Z|=\sqrt{2/\pi}\), so \(\mathbb{E}|X_n-X|=\sqrt{2/\pi}/n\). At \(p=2\), \(\mathbb{E}|Z|^2=1\), so \(\mathbb{E}|X_n-X|^2=1/n^2\) and \(\|X_n-X\|_2=1/n\). For a numerical-index check at \(n=2\), the difference is \(Z/2\): its mean absolute error is \(\sqrt{2/\pi}/2\), and its squared error is \(\mathbb{E}[Z^2/4]=1/4\). Both formulas give errors tending to zero as \(n\) increases.

Worked Example: Large Values on Small Events

On \(\Omega=[0,1]\) with the uniform probability measure, let \(A_n=[0,4^{-n}]\), \(X_n=2^n\mathbf{1}_{A_n}\), and \(X=0\). The interval includes its endpoints; those endpoints have probability zero. Calculate the moment error for \(p=1\) and \(p=2\).

The event \(A_n\) has probability \(4^{-n}\). On \(A_n\), the absolute difference is \(2^n\); outside \(A_n\), it is zero. Thus, for any fixed \(p\ge1\),

$$ \mathbb{E}|X_n-X|^p =(2^n)^p\,4^{-n} =2^{n(p-2)}. $$

For \(p=1\), the moment error is \(2^{-n}\), which tends to zero, and the \(L^1\) error is also \(2^{-n}\). For \(p=2\), the moment error is \(1\) for every \(n\), so the \(L^2\) error is \(1\) and does not tend to zero. For example, at \(n=2\), the event has probability \(1/16\) and the error on it is \(4\): the first absolute moment is \(4(1/16)=1/4\), while the second moment is \(4^2(1/16)=1\). This verifies the formulas at a specific index and shows that convergence in \(L^p\) depends on the chosen \(p\).

Results for Calculating and Combining Errors

Proposition: If \(x_n\) and \(x\) are real numbers, regarded as constant random variables, then \(x_n\to x\) in \(L^p\) if and only if the ordinary numerical sequence \(x_n\) converges to \(x\). More precisely, \(\|x_n-x\|_p=|x_n-x|\).

Proof. A constant random variable has expectation equal to its constant value. Therefore \(\mathbb{E}|x_n-x|^p=|x_n-x|^p\), and taking the \(p\)th root gives \(\|x_n-x\|_p=|x_n-x|\). The norm tends to zero exactly when the ordinary absolute difference tends to zero. \(\square\)

Proposition: Let \(p\ge1\), and suppose \(X_n\to X\) and \(Y_n\to Y\) in \(L^p\), with all variables on the same probability space. For fixed real numbers \(a,b\), the sequence \(aX_n+bY_n\) converges in \(L^p\) to \(aX+bY\).

Proof. Put \(U_n=X_n-X\) and \(V_n=Y_n-Y\). The error is \(aU_n+bV_n\), whose absolute value is at most \(|a||U_n|+|b||V_n|\). For nonnegative \(r,s\), convexity of \(t\mapsto t^p\) on \([0,\infty)\) gives \(((r+s)/2)^p\le(r^p+s^p)/2\), or \((r+s)^p\le2^{p-1}(r^p+s^p)\). Apply this with \(r=|a||U_n|\) and \(s=|b||V_n|\), then take expectations:

$$ \mathbb{E}|aU_n+bV_n|^p \le 2^{p-1}\left( |a|^p\mathbb{E}|U_n|^p+ |b|^p\mathbb{E}|V_n|^p \right). $$

Both expectations on the right tend to zero by the assumed \(L^p\) convergence. The left side is nonnegative and bounded above by a quantity tending to zero, so it tends to zero. This is precisely \(aX_n+bY_n\to aX+bY\) in \(L^p\). The argument includes \(a=0\) or \(b=0\), since the corresponding term is then zero. \(\square\)

A particularly useful special case makes the role of event probabilities explicit.

Proposition: For events \(A_n\) on a probability space, real constants \(c_n\), and the zero random variable \(X=0\), \(X_n=c_n\mathbf{1}_{A_n}\) satisfies \(\mathbb{E}|X_n-X|^p=|c_n|^p\mathbb{P}(A_n)\). Consequently, \(X_n\to0\) in \(L^p\) exactly when \(|c_n|^p\mathbb{P}(A_n)\to0\).

Proof. On \(A_n\), the indicator is \(1\), so \(|X_n|^p=|c_n|^p\). On the complement of \(A_n\), the indicator is \(0\), so \(|X_n|^p=0\). Taking the expectation of these two values gives \(\mathbb{E}|X_n|^p=|c_n|^p\mathbb{P}(A_n)\). By the definition of \(L^p\) convergence, this expectation tends to zero if and only if \(X_n\to0\) in \(L^p\). \(\square\)

Common Mistake: Taking the Moment of the Mean

A common incorrect calculation replaces \(\mathbb{E}|X_n-X|^p\) with \(|\mathbb{E}X_n-\mathbb{E}X|^p\). These are different quantities: the first averages the size of the error, while the second takes the size of the average error. For the indicator example, \(\mathbb{E}X_n=2^n4^{-n}=2^{-n}\). Squaring this mean gives \(4^{-n}\), which tends to zero. But the actual squared error is \(\mathbb{E}|X_n|^2=(2^n)^2 4^{-n}=1\), so the sequence does not converge to zero in \(L^2\). The correction is to raise the pointwise absolute error to the \(p\)th power first, and only then take its expectation.

When calculating an \(L^p\) limit, first write the difference \(X_n-X\) on the common probability space. Then raise its absolute value to the specified power, take its expectation, and finally check whether the resulting nonnegative numbers tend to zero. For scaled indicators, this procedure separates the size of the error from the probability of the event where it occurs; for Gaussian variables, it reduces the calculation to a Gaussian absolute moment.

Check Your Understanding

For each question, state the moment error before taking its \(p\)th root.

  1. If \(X_n=5-\tfrac{3}{n}\) and \(X=5\) are constant random variables, what are \(\mathbb{E}|X_n-X|^p\) and \(\|X_n-X\|_p\)?
  2. For \(X_n=(1+\tfrac{2}{n})Z\) and \(X=Z\), where \(Z\) is standard normal, calculate the squared error and the \(L^2\) error.
  3. If \(X_n=3^n\mathbf{1}_{A_n}\), \(X=0\), and \(\mathbb{P}(A_n)=9^{-n}\), does \(X_n\to0\) in \(L^1\)? In \(L^2\)?
  4. Why must the variables \(X_n\) and \(X\) be defined on the same probability space to use the difference \(X_n-X\)?
  5. For \(p\ge1\), if \(X_n\to X\) and \(Y_n\to Y\) in \(L^p\), what inequality in the proof controls the error of \(X_n+Y_n\)?

Previous

Next

Back to Tutorials