Joint Convergence Of Random Vectors

Why Joint Convergence Matters

Econometric arguments often involve several quantities at once: an estimated intercept and slope, a sample mean and a sample variance, or several moments computed from the same observations. Knowing the limiting behavior of each quantity separately does not always tell us how the quantities behave together. That distinction matters whenever a conclusion depends on their joint relationship.

A random vector is a collection of random variables treated as one object. This tutorial defines convergence in probability and convergence in distribution for such vectors, then separates two facts that are easy to confuse: coordinatewise convergence in probability does give convergence in probability of the vector, whereas coordinatewise convergence in distribution does not generally determine a joint distributional limit.

Definitions for Random Vectors

Let $X_n=(X_{n1},\ldots,X_{nk})$ be a sequence of random vectors taking values in $\mathbb{R}^k$. Write $X=(X_1,\ldots,X_k)$ for a possible limiting random vector and $c=(c_1,\ldots,c_k)$ for a fixed vector. The dimension $k$ is fixed as $n$ increases. For measuring the size of a vector, use the maximum norm $\|x\|_\infty=\max_{1\leq j\leq k}|x_j|$.

Definition: The sequence $X_n$ converges in probability to the random vector $X$, written $X_n\xrightarrow{p}X$, if for every $\varepsilon>0$, $$ P\bigl(\|X_n-X\|_\infty>\varepsilon\bigr)\longrightarrow 0. $$ It converges in distribution to $X$, written $X_n\xrightarrow{d}X$, if $E[f(X_n)]\longrightarrow E[f(X)]$ for every bounded continuous real-valued function $f$ on $\mathbb{R}^k$. Equivalently, the joint cumulative distribution functions converge at every continuity point of the limiting joint cumulative distribution function.

The joint cumulative distribution function at $x=(x_1,\ldots,x_k)$ is $P(X_1\leq x_1,\ldots,X_k\leq x_k)$. It records probabilities involving all coordinates together, rather than only the distribution of each coordinate separately. Convergence in probability is defined by the probability that the whole vector is farther than a given distance from its limit; convergence in distribution concerns the limiting joint law.

Convergence in Probability: The Coordinatewise Equivalence

For a fixed finite number of coordinates, convergence in probability of a vector is equivalent to convergence in probability of every coordinate to the corresponding coordinate of the limit. The finite-dimensional qualification matters: the argument uses a finite union of events.

Theorem: For fixed $k$, $X_n\xrightarrow{p}X$ if and only if $X_{nj}\xrightarrow{p}X_j$ for every $j=1,\ldots,k$.

To justify the forward direction, for each coordinate $j$ the event $\{|X_{nj}-X_j|>\varepsilon\}$ is contained in $\{\|X_n-X\|_\infty>\varepsilon\}$. Thus its probability tends to zero if the vector converges in probability. For the reverse direction, the event that the maximum coordinate error exceeds $\varepsilon$ is the union of the coordinate-error events. The union bound gives

P\bigl(\|X_n-X\|_\infty>\varepsilon\bigr) \leq \sum_{j=1}^{k}P\bigl(|X_{nj}-X_j|>\varepsilon\bigr) \longrightarrow 0.

The sum tends to zero because it has only $k$ terms and every term tends to zero. No independence between coordinates is needed. The same reasoning applies whether the limiting vector is fixed or random.

Worked Example: Two Sample Averages Converge Together

Suppose $(A_i,B_i)$, for $i=1,\ldots,n$, are independent and identically distributed random pairs, with $E[A_i]=2$, $E[B_i]=-1$, $\operatorname{Var}(A_i)=4$, and $\operatorname{Var}(B_i)=9$. The two variables within a pair need not be independent. Let $\bar A_n=n^{-1}\sum_{i=1}^n A_i$ and $\bar B_n=n^{-1}\sum_{i=1}^n B_i$.

Finite variances and independence across observations allow Chebyshev's inequality to be applied to each sample average. For any $\varepsilon>0$, $$ P(|\bar A_n-2|>\varepsilon)\leq \frac{4}{n\varepsilon^2}, \qquad P(|\bar B_n+1|>\varepsilon)\leq \frac{9}{n\varepsilon^2}. $$ Both upper bounds tend to zero. Therefore $\bar A_n\xrightarrow{p}2$ and $\bar B_n\xrightarrow{p}-1$, so the coordinatewise equivalence gives $(\bar A_n,\bar B_n)\xrightarrow{p}(2,-1)$.

For a numerical check, take $\varepsilon=0.5$ and $n=400$. The union bound gives $$ P\left(\max\{|\bar A_n-2|,|\bar B_n+1|\}>0.5\right) \leq \frac{4}{400(0.5)^2}+\frac{9}{400(0.5)^2} =\frac{16}{400}+\frac{36}{400}=0.13. $$ This is an upper bound, not an exact probability. Its value at $n=400$ is not itself the convergence claim; the bound tends to zero as $n$ increases. Dependence between $A_i$ and $B_i$ does not invalidate the union bound.

Convergence in Distribution: What Follows Coordinate by Coordinate

If a vector converges in distribution, then each coordinate converges in distribution to the corresponding coordinate. A coordinate is obtained by applying a continuous projection, such as $\pi_1(x_1,\ldots,x_k)=x_1$. The continuous mapping theorem for distributional limits says that applying a continuous function to a convergent random vector preserves convergence in distribution. Applying it to each coordinate projection gives the result.

Takeaway: $X_n\xrightarrow{d}X$ implies $X_{nj}\xrightarrow{d}X_j$ for every coordinate $j$. The reverse implication is not valid in general: marginal limits alone do not specify how coordinates are jointly related.

The next example uses a scalar central limit result together with a continuous transformation to obtain a joint limit. This does not require a multivariate central limit theorem: the vector is a function of one scalar random variable.

Worked Example: A Joint Limit From One Scalar Limit

Let $Z_n\sim N(0,1+1/n)$, and define $V_n=(Z_n,2Z_n+1)$. Since $Z_n$ has the same distribution as $\sqrt{1+1/n}\,Z$, where $Z\sim N(0,1)$, and $\sqrt{1+1/n}\to 1$, the continuous mapping theorem gives $Z_n\xrightarrow{d}Z$.

Now define the continuous function $g(z)=(z,2z+1)$. Applying it to the scalar limit yields $$ V_n=g(Z_n)\xrightarrow{d}g(Z)=(Z,2Z+1). $$ In the bounded-continuous-function definition, the justification is that for any bounded continuous $f:\mathbb{R}^2\to\mathbb{R}$, the composition $f\circ g$ is bounded and continuous on $\mathbb{R}$. Hence $E[f(V_n)]=E[(f\circ g)(Z_n)]\to E[(f\circ g)(Z)]$.

For example, if $n=4$ and a hypothetical realized value is $Z_4=0.8$, then $V_4=(0.8,2(0.8)+1)=(0.8,2.6)$. The coordinates are linked by construction: the second is twice the first plus one. The limiting vector retains this relationship, so the joint limit contains information that would be lost by reporting only the two marginal limits.

Why Marginal Distributional Limits Are Not Enough

The failure of the reverse implication for distributional convergence is not a technicality. Two coordinates can each have a stable marginal distribution while their dependence changes with $n$. In that case the joint distribution may fail to settle to any limit.

Worked Example: Fixed Marginals, No Joint Limit

Let $X_n$ be Bernoulli with probability $1/2$ of being 1 for every $n$. For even $n$, set $Y_n=X_n$; for odd $n$, set $Y_n=1-X_n$. Each $X_n$ and each $Y_n$ has the same Bernoulli distribution, so both coordinate sequences converge in distribution to a Bernoulli random variable with success probability $1/2$.

But the joint behavior alternates. For even $n$, the pair is either $(0,0)$ or $(1,1)$, each with probability $1/2$. For odd $n$, it is either $(0,1)$ or $(1,0)$, each with probability $1/2$. To see that there is no joint distributional limit, define $q(t)=\min(1,\max(0,t))$ and $f(x,y)=q(x)q(y)$. This is a bounded continuous function on $\mathbb{R}^2$. For even $n$, $E[f(X_n,Y_n)]=1/2$; for odd $n$, $E[f(X_n,Y_n)]=0$. These expectations do not converge, so the bounded-continuous-function criterion rules out convergence in distribution of the vectors.

Thus both marginal distributions converge, but the joint distribution does not. The changing feature is dependence: the coordinates agree for even $n$ and disagree for odd $n$.

How to Use the Distinction

When a problem concerns several statistics, first identify whether its conclusion is about each statistic separately or about their joint behavior. For convergence in probability, separate coordinate checks are sufficient in fixed finite dimension, because the union bound controls the probability of any coordinate failing. For convergence in distribution, a collection of marginal limits is not generally enough; a joint argument must account for dependence.

A useful way to establish a joint distributional limit is to start with a known vector limit or to express the vector as a continuous function of a random object with a known limit. The worked transformation illustrates the latter. In later applications, joint limits provide the foundation for studying functions of multiple sample moments together. The multivariate central limit theorem supplies one important source of such limits, but it is not needed to understand the definitions or the distinction established here.

Common Mistake

Wrong reasoning: “Every coordinate has a limiting distribution, so the vector has a limiting distribution.”
Correction: Coordinatewise convergence in distribution establishes marginal limits only. Check whether the dependence also stabilizes, or prove joint convergence directly. The alternating Bernoulli example has convergent marginals but no joint limit.
Related error: Treating the union-bound argument for convergence in probability as proof of the corresponding claim for convergence in distribution. They are different convergence modes and require different reasoning.

Key takeaway: In fixed finite dimension, vector convergence in probability is equivalent to convergence in probability of every coordinate. Vector convergence in distribution implies coordinatewise distributional convergence, but marginal distributional convergence alone does not ensure a joint limit.

Check Your Understanding

Answer each question by identifying whether it concerns convergence in probability, convergence in distribution, or both.

State the maximum-norm definition of $X_n\xrightarrow{p}X$.
Why does coordinatewise convergence in probability imply convergence in probability of a vector when its dimension is fixed?
If $X_n\xrightarrow{d}X$, what can be concluded about each coordinate, and which result justifies that conclusion?
In the alternating Bernoulli example, what changes with $n$: the marginal distributions, the dependence, or both?
Why does the bounded continuous function $f(x,y)=q(x)q(y)$ show that the alternating pairs do not converge in distribution?