Joint Convergence with a Component Converging in Probability

Why a Constant Limit Changes the Joint-Convergence Problem

Knowing the limiting behavior of two coordinates separately does not usually determine the limiting behavior of the pair. The dependence between coordinates can matter: two random variables may each have fixed marginal distributions while their joint distribution changes.

A useful special case avoids that difficulty. If one coordinate converges in probability to a constant, then it concentrates near that constant with probability tending to one. This concentration is enough to combine it with a coordinate that converges weakly. No independence assumption is needed.

Central idea: On a region where the weakly convergent coordinate is confined to a compact set, a bounded continuous test function changes little when the other coordinate is close to its constant limit. The probability of being outside that region or away from the constant can be made small.

Definitions and Statement

For random vectors $X_n$ and $X$ in $\mathbb{R}^d$, weak convergence $X_n\Rightarrow X$ means that the expectations of every bounded continuous function $g:\mathbb{R}^d\to\mathbb{R}$ converge:

\mathbb{E}[g(X_n)]\longrightarrow \mathbb{E}[g(X)].

For random vectors $Y_n$ and a fixed vector $c\in\mathbb{R}^q$, convergence in probability $Y_n\xrightarrow{\mathbb{P}}c$ means that, for every $\delta>0$, the probability that $Y_n$ is farther than $\delta$ from $c$ tends to zero:

\mathbb{P}(\|Y_n-c\|\geq\delta)\longrightarrow 0.

Here and below, $\|\cdot\|$ denotes the Euclidean norm in the relevant space. The pair $(X_n,Y_n)$ is regarded as a random vector in $\mathbb{R}^{d+q}$.

Theorem: Suppose $X_n\Rightarrow X$ in $\mathbb{R}^d$, and $Y_n\xrightarrow{\mathbb{P}}c$ in $\mathbb{R}^q$, where $c$ is a fixed, nonrandom vector. Then $$ (X_n,Y_n)\Rightarrow(X,c) $$ in $\mathbb{R}^{d+q}$. The random vectors $X_n$ and $Y_n$ need not be independent, and no independence assumption involving $X$ is required.

Worked Examples

Worked Example: A Bernoulli Coordinate and a Vanishing Change

Let $Z$ be Bernoulli with $\mathbb{P}(Z=1)=p$, where $0<p<1$. For a fixed $c\in\mathbb{R}$, set $X_n=Z$ and $Y_n=c+Z/n$. Since $X_n$ has the same distribution as $Z$ for every $n$, $X_n\Rightarrow Z$. Also,

|Y_n-c|=\frac{Z}{n}\leq\frac{1}{n}.

For any $\delta>0$, the right side is less than $\delta$ for all sufficiently large $n$; hence $Y_n\xrightarrow{\mathbb{P}}c$. The theorem gives $(X_n,Y_n)\Rightarrow(Z,c)$.

For every $n$, the events $\{X_n=1\}$ and $\{Y_n=c+1/n\}$ are both $\{Z=1\}$. Their intersection has probability $p$, whereas the product of their probabilities is $p^2$. Since $0<p<1$, $p\ne p^2$, so the coordinates are dependent. The conclusion therefore applies in a genuinely dependent example. If $p=0$ or $p=1$, both coordinates are constant and this dependence claim would not hold.

Worked Example: Dependence That Becomes Rare

Let $U$ be uniform on $[0,1]$, and define $X_n=U$ and $Y_n=c+\mathbf{1}_{\{U\leq 1/n^2\}}$, where $\mathbf{1}_A$ is one on the event $A$ and zero otherwise. The law of $X_n$ is the same for every $n$, so $X_n\Rightarrow U$. For every $\delta>0$, once $\delta\leq1$,

\mathbb{P}(|Y_n-c|\geq\delta) = \mathbb{P}(U\leq1/n^2) = \frac{1}{n^2} \longrightarrow 0.

For $\delta>1$, the probability on the left is zero, so convergence in probability holds for every $\delta>0$. Thus $(X_n,Y_n)\Rightarrow(U,c)$.

For $n\geq2$, let $A=\{U\leq1/n^2\}$ and $B=\{Y_n=c+1\}$. These events are equal, so $\mathbb{P}(A\cap B)=1/n^2$, while $\mathbb{P}(A)\mathbb{P}(B)=1/n^4$. Since $1/n^2\ne1/n^4$, the coordinates are dependent for $n\geq2$. At $n=1$, $U\leq1$ with probability one, so $Y_1=c+1$ is constant and is independent of $X_1$. The rare event responsible for dependence has probability tending to zero, which is consistent with the joint limit having second coordinate $c$.

Worked Example: A Continuous Dependent Pair

Let $Z$ have the standard normal distribution, and set $X_n=Z$ and $Y_n=c+Z/n$. The distribution of $X_n$ does not change with $n$, so $X_n\Rightarrow Z$. For each $\delta>0$,

\mathbb{P}(|Y_n-c|\geq\delta) = \mathbb{P}(|Z|\geq n\delta) \longrightarrow 0,

because the tail probability of a real-valued random variable tends to zero as the threshold tends to infinity. Thus $Y_n\xrightarrow{\mathbb{P}}c$, and the theorem yields $(X_n,Y_n)\Rightarrow(Z,c)$.

For every $n$, the events $\{X_n>0\}$ and $\{Y_n>c\}$ are equal to $\{Z>0\}$. Their intersection has probability $1/2$, but the product of their probabilities is $1/4$. Hence the coordinates are dependent for every $n$. This example shows that the result is not limited to discrete variables or to dependence occurring on a rare event.

Proof of the Joint-Convergence Theorem

We prove the theorem using the bounded-continuous-function characterization of weak convergence. First, weak convergence of $X_n$ provides the compact-region control needed in the argument.

Proof. Let $f:\mathbb{R}^{d+q}\to\mathbb{R}$ be bounded and continuous, and write $M=\sup_{(x,y)}|f(x,y)|$. If $M=0$, then $f$ is identically zero and the desired convergence of expectations is immediate. Suppose $M>0$.

Fix $\eta>0$. Since the balls $\{x:\|x\|<R\}$ increase to $\mathbb{R}^d$ as $R$ increases, choose $R>0$ such that $\mathbb{P}(\|X\|<R)>1-\eta$. The Portmanteau theorem for the open ball gives

\liminf_{n\to\infty}\mathbb{P}(\|X_n\|<R) \geq \mathbb{P}(\|X\|<R)>1-\eta.

Consequently, $\limsup_{n\to\infty}\mathbb{P}(\|X_n\|\geq R)\leq\eta$. This is the only compactness control needed: for large $n$, $X_n$ lies in the closed ball of radius $R$ with high probability.

The set consisting of $x$ in that closed ball and $y$ in the closed unit ball centered at $c$ is compact in $\mathbb{R}^{d+q}$. A continuous function is uniformly continuous on a compact set. Therefore, given $\varepsilon>0$, there is a $\delta\in(0,1]$ such that

\|x\|\leq R,\quad \|y-c\|<\delta \quad\Longrightarrow\quad |f(x,y)-f(x,c)|<\varepsilon.

On the event $\{\|X_n\|<R,\ \|Y_n-c\|<\delta\}$, this inequality applies. On its complement, the absolute difference is at most $2M$. Splitting the expectation according to these two events and using the union bound gives

\begin{aligned} \left|\mathbb{E}f(X_n,Y_n)-\mathbb{E}f(X_n,c)\right| &\leq \mathbb{E}\left|f(X_n,Y_n)-f(X_n,c)\right|\\ &\leq \varepsilon +2M\left[ \mathbb{P}(\|X_n\|\geq R) +\mathbb{P}(\|Y_n-c\|\geq\delta) \right]. \end{aligned}

The first probability has limit superior at most $\eta$, as shown above. The second tends to zero by convergence in probability. Hence

\limsup_{n\to\infty} \left|\mathbb{E}f(X_n,Y_n)-\mathbb{E}f(X_n,c)\right| \leq \varepsilon+2M\eta.

Both $\varepsilon$ and $\eta$ can be chosen arbitrarily small, so the difference of expectations tends to zero.

Finally, $g(x)=f(x,c)$ is bounded and continuous on $\mathbb{R}^d$. By $X_n\Rightarrow X$,

\mathbb{E}f(X_n,c) = \mathbb{E}g(X_n) \longrightarrow \mathbb{E}g(X) = \mathbb{E}f(X,c).

Combining this limit with the expectation comparison proves $\mathbb{E}f(X_n,Y_n)\to\mathbb{E}f(X,c)$ for every bounded continuous $f$. This is precisely $(X_n,Y_n)\Rightarrow(X,c)$.

End of proof.
$\square$

What the Argument Does—and Does Not—Use

The proof does not factor any joint probability or expectation into marginal terms. In particular, it never assumes that $X_n$ and $Y_n$ are independent. Instead, it compares $f(X_n,Y_n)$ with $f(X_n,c)$ on a high-probability event. Uniform continuity controls the difference there; boundedness controls it on the remaining event.

The constant limit is essential to this argument. Convergence in probability to $c$ places $Y_n$ near the same fixed point, irrespective of the value of $X_n$. If the second coordinate instead had a nonconstant weak limit, separate convergence of the two coordinates would generally not determine their joint limit: dependence could affect the joint distribution.

A common pitfall is to think that dependence must disappear before joint convergence can hold. It need not. The examples show that the coordinates can be dependent at every index, while the second coordinate still concentrates near $c$. The result concerns the limiting joint distribution, not independence of the coordinates at finite $n$.

Check Your Understanding

Use the definitions and proof above to answer these questions.

Which two different modes of convergence are assumed in the theorem, and which mode is concluded for the pair?
Where does the proof use that the limit $c$ is a constant?
Why is it useful to restrict attention to a compact region for $X_n$ when applying continuity of the test function?
In the rare-event example, why are the coordinates dependent for $n\geq2$, but not necessarily at $n=1$?
At what point, if any, does the proof require independence?