Equal Marginals Do Not Guarantee Convergence in Probability

Same Distribution Is Not the Same as Being Close

A sequence can have exactly the same distribution as a target random variable at every index and therefore converge in distribution to that target. This says nothing by itself about whether the sequence gets close to the target on the same probability space. Convergence in probability concerns the joint behavior of the sequence and target, not just their separate distributions.

Independent copies provide a sharp illustration. If the common distribution is nondegenerate, an independent copy has a positive chance of landing a definite distance away from the target. That chance does not shrink as the index grows.

Definition: A real-valued random variable $X$ is nondegenerate if there is no constant $c$ such that $P(X=c)=1$. A sequence $X_n$ converges in probability to a real-valued random variable $Y$ if, for every $\varepsilon>0$, $$ P(|X_n-Y|>\varepsilon)\longrightarrow 0. $$ The sequence converges in distribution to $Y$ if its distribution functions $F_n$ converge to the distribution function $F_Y$ at every continuity point of $F_Y$.

Equal Marginals Give Distributional Convergence

Suppose $X_0,X_1,X_2,\ldots$ are independent and identically distributed real-valued random variables. Take $X_0$ as the specified target, and consider the sequence $X_n$, $n\geq 1$. Each $X_n$ has the same distribution function as $X_0$.

Result: If $X_n$ and $Y$ have the same distribution for every $n$, then $X_n\xrightarrow{d}Y$. Indeed, for every real $x$, $F_n(x)=F_Y(x)$, so the required CDF convergence holds at every continuity point of $F_Y$. No independence assumption is needed for this conclusion.

Thus, for independent copies, $X_n\xrightarrow{d}X_0$ holds immediately. To determine whether $X_n\xrightarrow{p}X_0$, however, we must examine $P(|X_n-X_0|>\varepsilon)$. Independence enters precisely here.

A Fixed Error Probability for Independent Copies

The key fact is that a nondegenerate distribution puts positive probability on both sides of some finite threshold. Moreover, the probability above that threshold can be found a strictly positive distance farther to the right. These two events produce a positive lower bound on the chance that two independent copies are separated.

Theorem: Let $X_0,X_1,X_2,\ldots$ be independent and identically distributed real-valued random variables, and suppose their common distribution is nondegenerate. Then there exist $\varepsilon>0$ and $c>0$ such that, for every $n\geq1$, $$ P(|X_n-X_0|>\varepsilon)\geq c. $$ Consequently, $X_n$ does not converge in probability to $X_0$, although $X_n\xrightarrow{d}X_0$.

Proof. We first show that there is a real number $t$ for which $0<P(X_0\leq t)<1$. Suppose instead that $P(X_0\leq t)$ were always either zero or one. Since $X_0$ is real-valued, $P(X_0\leq m)\to1$ as integers $m\to\infty$, and $P(X_0\leq -m)\to0$. Therefore the set $S=\{t\in\mathbb R:P(X_0\leq t)=1\}$ is nonempty and bounded below. Let $a=\inf S$. For every $k\geq1$, choose $s_k\in S$ with $s_k<a+1/k$. Since $a\leq s_k$, we have $P(X_0\leq a+1/k)=1$. Also $a-1/k\notin S$, so the assumed zero-or-one property gives $P(X_0\leq a-1/k)=0$. Taking the countable intersection of the probability-one events $X_0\leq a+1/k$ shows $X_0\leq a$ almost surely. Taking the countable union of the probability-zero events $X_0\leq a-1/k$ shows $X_0\geq a$ almost surely. Hence $X_0=a$ almost surely, contradicting nondegeneracy. Thus such a $t$ exists.

Write $q=P(X_0\leq t)>0$. Since $P(X_0>t)>0$, and $$ \{X_0>t\}=\bigcup_{k=1}^{\infty}\{X_0>t+1/k\}, $$ at least one integer $k\geq1$ has $r=P(X_0>t+1/k)>0$. (If every event in the union had probability zero, the union would have probability zero.) Set $\varepsilon=1/k$ and $c=qr$. By independence of $X_0$ and $X_n$, $$ P(X_0\leq t,\ X_n>t+1/k) =P(X_0\leq t)P(X_n>t+1/k) =qr=c. $$ On this event, $X_n-X_0>1/k=\varepsilon$, so it is contained in $\{|X_n-X_0|>\varepsilon\}$. Therefore $P(|X_n-X_0|>\varepsilon)\geq c>0$ for every $n$. This fixed positive lower bound contradicts the defining requirement for convergence in probability. The equal-distribution result above gives convergence in distribution. $\square$

Worked Examples

Worked Example: Independent Bernoulli Copies

Let $X_0,X_1,\ldots$ be independent Bernoulli random variables with $P(X_n=1)=0.3$ and $P(X_n=0)=0.7$. Every $X_n$ has the same Bernoulli distribution as the target $X_0$, so the sequence converges in distribution to $X_0$.

For $\varepsilon=0.5$, the event $|X_n-X_0|>0.5$ occurs exactly when one variable is zero and the other is one. Independence gives $$ P(|X_n-X_0|>0.5) =P(X_0=0,X_n=1)+P(X_0=1,X_n=0) =(0.7)(0.3)+(0.3)(0.7)=0.42. $$ This probability is $0.42$ for every $n$, not a quantity tending to zero. The copies do not converge in probability to the target.

Worked Example: Independent Uniform Copies

Let $U_0,U_1,\ldots$ be independent Uniform$(0,1)$ random variables, with $U_0$ as the target. For every $n$, $U_n$ and $U_0$ have the same continuous distribution, so $U_n\xrightarrow{d}U_0$.

The pair $(U_0,U_n)$ has constant joint density $1$ on the unit square. The event $|U_n-U_0|>1/2$ consists of two right triangles, one above and one below the diagonal band. Each triangle has legs of length $1/2$, so its area is $\frac12(1/2)(1/2)=1/8$. Therefore $$ P(|U_n-U_0|>1/2)=1/8+1/8=1/4 $$ for every $n$. A quarter of the probability remains outside the fixed tolerance $1/2$, so convergence in probability fails.

Worked Example: Independent Standard Normal Copies

Let $Z_0,Z_1,\ldots$ be independent $N(0,1)$ random variables. They all have the same distribution, so $Z_n\xrightarrow{d}Z_0$. Consider the fixed tolerance $\varepsilon=2$. The event $\{Z_0\leq-1,\ Z_n\geq1.1\}$ implies $Z_n-Z_0\geq2.1>2$, and is therefore contained in $\{|Z_n-Z_0|>2\}$.

Let $\Phi$ denote the standard normal distribution function. Independence and symmetry give $$ P(Z_0\leq-1,\ Z_n\geq1.1) =\Phi(-1)\bigl[1-\Phi(1.1)\bigr] =\bigl[1-\Phi(1)\bigr]\bigl[1-\Phi(1.1)\bigr]>0. $$ Both factors are positive because the standard normal density is positive on every real interval and its upper tail is nonzero at finite arguments. This is a positive lower bound, the same for every $n$, on the probability of exceeding distance $2$. Thus convergence in probability to $Z_0$ fails.

Why Dependence Matters

The examples do not say that a sequence with the same marginals as a target can never converge in probability to it. They say that equal marginals alone do not ensure such convergence; the joint arrangement matters. To see the contrast, define $Y_n=X_0$ for every $n$. Each $Y_n$ has exactly the target's distribution, and $P(|Y_n-X_0|>\varepsilon)=0$ for every $\varepsilon>0$. Thus $Y_n$ converges in probability to $X_0$, but the sequence is not a sequence of independent copies.

The distinction can be summarized as follows: a marginal distribution describes one variable at a time, while convergence in probability compares two variables on the same outcome. Independence prevents the copies from tracking the target closely; repeated reuse of the target makes them identical to it.

Common Mistake

Incorrect reasoning: “Every $X_n$ has the same distribution as $X_0$, so $X_n$ is effectively the same as $X_0$ and must converge to it in probability.”

Equal distributions mean that $P(X_n\in A)=P(X_0\in A)$ for each measurable set $A$. They do not mean $P(X_n=X_0)=1$, nor do they determine the probability of a joint event such as $\{|X_n-X_0|>\varepsilon\}$. For independent, nondegenerate copies, the theorem gives a fixed positive lower bound for such an error event at some tolerance. Equality of marginals does guarantee convergence in distribution, but it does not guarantee convergence in probability.

Key takeaway: Identical marginal distributions imply distributional convergence to a target with that distribution. For independent copies of a nondegenerate target, however, some fixed-distance error has positive probability at every index, so convergence in probability fails.

Check Your Understanding

Use the definitions and argument above to answer these questions.

If $X_n$ and $Y$ have identical distributions for every $n$, why does $X_n\xrightarrow{d}Y$ hold?
In the Bernoulli example with success probability $0.3$, what is $P(|X_n-X_0|>0.5)$, and why does it not depend on $n$?
Why does nondegeneracy imply that there is a threshold $t$ with $0<P(X_0\leq t)<1$?
Where does independence enter the theorem's lower-bound argument?
How can a sequence have the same marginal distribution as a target and still converge in probability to it?