Escaping Rare Mass and Degenerate Distributional Limits

Introduction

A sequence of random variables can take increasingly large values and still converge in distribution to zero. The key is that those large values may occur with probabilities that vanish. At any fixed finite threshold, the probability of seeing one of them eventually becomes negligible, even though the possible values themselves keep moving away.

This tutorial studies the concrete sequence \(X_n\) that equals zero with probability \(1-1/n\) and equals \(n\) with probability \(1/n\). We will verify its distributional limit directly from its cumulative distribution functions (CDFs), then use the same reasoning for related rare-event constructions. The example also helps separate convergence of distributions from the behavior of expectations and other moments.

The Rare-Mass Sequence

Definition: For each positive integer \(n\), let \(X_n\) be a random variable with \[ P(X_n=0)=1-\frac{1}{n}, \qquad P(X_n=n)=\frac{1}{n}. \] Its CDF is \(F_n(x)=P(X_n\leq x)\). The proposed limit is the constant random variable zero, whose CDF is \[ F_0(x)= \begin{cases} 0, & x<0,\\ 1, & x\geq 0. \end{cases} \]

A CDF has a jump at zero, so \(F_0\) is continuous at every real number other than zero. By the definition of convergence in distribution, \(X_n\xrightarrow{d}0\) if \(F_n(x)\to F_0(x)\) at every continuity point \(x\) of \(F_0\). The value at the jump is not required for this check, although we can calculate it as well.

Worked Example: Checking the CDF Limit

First write the CDF by locating \(x\) relative to the two possible values, zero and \(n\). If \(x<0\), neither value is at most \(x\). If \(0\leq x<n\), only the value zero is at most \(x\). If \(x\geq n\), both values are at most \(x\). Therefore

F_n(x)= \begin{cases} 0, & x<0,\\[3pt] 1-\dfrac{1}{n}, & 0\leq x<n,\\[3pt] 1, & x\geq n. \end{cases}

Now fix a continuity point \(x\) of \(F_0\). If \(x<0\), then \(F_n(x)=0\) for every \(n\), so \(F_n(x)\to 0=F_0(x)\). If \(x>0\), then \(n>x\) for all sufficiently large \(n\). For those \(n\), \(F_n(x)=1-1/n\), and hence \(F_n(x)\to 1=F_0(x)\). These are all the continuity points of \(F_0\), so \(X_n\xrightarrow{d}0\).

At \(x=0\), which is a discontinuity point of \(F_0\), the formula gives \(F_n(0)=1-1/n\to1\). This extra fact is consistent with the limit but is not needed to establish convergence in distribution.

Why the CDF Check Works

The CDF check examines fixed thresholds. For any fixed positive \(x\), the exceptional value \(n\) eventually lies above \(x\). The CDF at that threshold then counts only the mass at zero, which tends to one. At a fixed negative threshold, the CDF remains zero. This is why the location of the rare value can diverge without preventing the stated distributional limit.

The central feature is not specifically that the rare value equals \(n\). It is that the probability of encountering a value different from zero goes to zero. The exceptional values can vary in size or sign; for this argument, their locations do not need to converge to any finite number.

Theorem: Let \(X_n\) be real-valued random variables, and suppose there are events \(A_n\) such that \(P(A_n)\to0\) and \(X_n=0\) on \(A_n^c\). Then \(X_n\xrightarrow{d}0\). No restriction is needed on the values \(X_n\) takes on \(A_n\).

Proof. Let \(F_n\) be the CDF of \(X_n\), and let \(F_0\) be the CDF of the constant zero. If \(x<0\), then the event \(\{X_n\leq x\}\) cannot occur on \(A_n^c\), because \(X_n=0>x\) there. Thus

0\leq F_n(x)=P(X_n\leq x)\leq P(A_n)\longrightarrow 0=F_0(x).

If \(x>0\), then \(\{X_n>x\}\) cannot occur on \(A_n^c\), again because \(X_n=0\) there. Therefore

0\leq 1-F_n(x)=P(X_n>x)\leq P(A_n)\longrightarrow 0,

so \(F_n(x)\to1=F_0(x)\). The continuity points of \(F_0\) are exactly the real numbers other than zero. We have proved CDF convergence at every such point, and therefore \(X_n\xrightarrow{d}0\). \(\square\)

Worked Example: A More Distant Rare Value

Let \(Y_n=0\) with probability \(1-1/n\) and \(Y_n=n^2\) with probability \(1/n\). The exceptional value now moves away faster, but the probability of the exceptional event still tends to zero. For a fixed \(x<0\), \(P(Y_n\leq x)=0\). For a fixed \(x>0\), \(n^2>x\) eventually, so

P(Y_n\leq x)=1-\frac{1}{n}\longrightarrow1.

Thus \(Y_n\xrightarrow{d}0\), by the CDF criterion. The same conclusion follows from the theorem with \(A_n=\{Y_n=n^2\}\), since \(P(A_n)=1/n\to0\).

The mean, however, is

E[Y_n]=0\left(1-\frac{1}{n}\right)+n^2\left(\frac{1}{n}\right)=n.

It diverges rather than tending to the mean zero of the limit. This calculation does not contradict the distributional limit: it shows that the rare values can have a substantial effect on an expectation even when their total probability vanishes.

Distributional Convergence and Probability of a Large Error

For the original sequence, the same rare-event structure also lets us check convergence in probability directly. For any fixed \(\varepsilon>0\), choose an integer \(n>\varepsilon\). The value zero is within \(\varepsilon\) of zero, while the value \(n\) is farther than \(\varepsilon\). Consequently,

P(|X_n-0|>\varepsilon)=\frac{1}{n}\longrightarrow0.

This is exactly the definition of \(X_n\xrightarrow{p}0\). Thus, in this example, the large exceptional value does not prevent convergence in probability: the chance of seeing it becomes negligible. The earlier result that convergence in probability to a constant implies convergence in distribution is consistent with the CDF calculation; the direct CDF proof makes the limit visible at its continuity points.

Worked Example: Rare Values on Both Sides of Zero

For each \(n\geq2\), let \(Z_n\) equal \(-n\) with probability \(1/n\), equal \(n\) with probability \(1/n\), and equal zero with probability \(1-2/n\). These probabilities are nonnegative and sum to

\frac{1}{n}+\frac{1}{n}+1-\frac{2}{n}=1.

Fix \(x<0\). For all sufficiently large \(n\), \(-n<x<0<n\). The only possible value at most \(x\) is then \(-n\), so \(P(Z_n\leq x)=1/n\to0\). Fix \(x>0\). For all sufficiently large \(n\), \(-n<0<x<n\), and the values at most \(x\) are \(-n\) and zero. Hence

P(Z_n\leq x)=\frac{1}{n}+1-\frac{2}{n} =1-\frac{1}{n}\longrightarrow1.

These are all the continuity points of the CDF of the constant zero, so \(Z_n\xrightarrow{d}0\). In addition, for any fixed \(\varepsilon>0\), once \(n>\varepsilon\) the event \(\{|Z_n|>\varepsilon\}\) consists of the two nonzero outcomes. Its probability is \(2/n\to0\), confirming convergence in probability as well.

The mean and variance illustrate a different aspect of the same construction. By symmetry, \(E[Z_n]=(-n)(1/n)+n(1/n)=0\). Since the mean is zero,

\operatorname{Var}(Z_n)=E[Z_n^2] =n^2\left(\frac{1}{n}\right)+n^2\left(\frac{1}{n}\right)=2n.

The means stay at zero, while the variances grow without bound. A distributional limit alone does not force these moment calculations to approach those of the limit.

What the Example Does—and Does Not—Show

For the original \(X_n\), direct calculation gives

E[X_n]=0\left(1-\frac{1}{n}\right)+n\left(\frac{1}{n}\right)=1, \qquad E[X_n^2]=0+n^2\left(\frac{1}{n}\right)=n.

Thus the mean does not tend to zero, and the second moment grows. Neither fact changes the CDF limit. A CDF check at a fixed \(x\) asks how much probability lies at or below that threshold; it does not weight an outcome in proportion to its numerical size. An increasingly large outcome can therefore matter greatly in a moment calculation while contributing only a vanishing amount of probability.

Common Mistake

A tempting but incorrect argument is: “The value \(n\) goes to infinity, so \(X_n\) cannot converge to zero.” This treats the possible value as if it occurred with fixed positive probability. In fact, for every \(\varepsilon>0\), once \(n>\varepsilon\), the probability that \(X_n\) is more than \(\varepsilon\) from zero is only \(1/n\), which tends to zero. Likewise, the CDF criterion concerns fixed continuity points, not the moving threshold \(x=n\).

A second mistake is to demand CDF convergence at every real number, including the jump at zero. The definition requires convergence only at continuity points of the proposed limit CDF. In this particular example the CDFs also converge at zero, but that is an extra calculation, not a requirement of the criterion.

Key takeaway: If a variable equals zero except on events whose probabilities vanish, then its CDF converges to the CDF of zero at every continuity point, regardless of how large the exceptional values become. Check probabilities at fixed thresholds, and do not infer moment convergence from a distributional limit alone.

Check Your Understanding

Use the CDF criterion and the rare-event bounds developed above.

For the original \(X_n\), write \(F_n(x)\) for each of the three regions \(x<0\), \(0\leq x<n\), and \(x\geq n\).
Why is convergence of \(F_n(x)\) at \(x=0\) not required to establish convergence in distribution to the constant zero?
Suppose \(W_n=0\) except on an event \(A_n\) with \(P(A_n)=1/\sqrt{n}\). What condition on \(W_n\) is needed on \(A_n^c\) for the rare-event theorem to apply?
For the variable \(Y_n\) that equals \(n^2\) with probability \(1/n\) and zero otherwise, calculate \(E[Y_n]\). Why does its value not contradict \(Y_n\xrightarrow{d}0\)?
For \(Z_n\) in the two-sided example, calculate \(P(|Z_n|>\varepsilon)\) when \(n>\varepsilon\). What does this probability show?