Convolution of Independent Random Vectors

Why Vector Convolution Requires a Region

For scalar random variables, a sum has one output coordinate. For random vectors, the sum is also a vector, and its joint law depends on all coordinates together. To find the density or mass at an output vector \(z\), we must account for every pair of input vectors whose sum is \(z\). In the continuous case, these pairs form a region of integration; in the discrete case, they form a collection of summands.

The coordinates within each random vector need not be independent. The convolution uses the joint density or mass function of each vector, not products of its coordinate marginals unless those coordinates are themselves independent.

Definition: Let \(X\) and \(Y\) be independent random vectors in \(\mathbb{R}^d\), and let \(Z=X+Y\), with addition coordinate by coordinate. If \(X\) and \(Y\) have densities \(f_X\) and \(f_Y\) with respect to \(d\)-dimensional Lebesgue measure, the convolution formula is

f_Z(z)=\int_{\mathbb{R}^d} f_X(x)f_Y(z-x)\,dx

for almost every \(z\in\mathbb{R}^d\). The integration region can be restricted to points \(x\) for which both \(f_X(x)\) and \(f_Y(z-x)\) are nonzero. For countable-valued independent vectors with mass functions \(p_X\) and \(p_Y\), the corresponding formula is

p_Z(z)=\sum_{x}p_X(x)p_Y(z-x),

where the sum is over the possible values of \(X\).

Why the Continuous Formula Holds

The formula follows by changing variables from the two input vectors \((X,Y)\) to \((X,Z)\), where \(Z=X+Y\). Independence gives the joint density \(f_X(x)f_Y(y)\). The change of variables is invertible: \(y=z-x\). Its absolute Jacobian determinant is one, so it does not introduce an additional factor.

Theorem: If \(X,Y\in\mathbb{R}^d\) are independent and have densities \(f_X,f_Y\), then \(Z=X+Y\) has a density given almost everywhere by the multivariate convolution formula.

Proof. Let \(B\) be any Borel subset of \(\mathbb{R}^d\). Independence gives the joint density \(f_X(x)f_Y(y)\) for \((X,Y)\). Therefore

\begin{aligned} P(Z\in B) &=\int_{\mathbb{R}^d}\int_{\mathbb{R}^d} \mathbf{1}_B(x+y)f_X(x)f_Y(y)\,dy\,dx\\ &=\int_{\mathbb{R}^d}\int_{\mathbb{R}^d} \mathbf{1}_B(z)f_X(x)f_Y(z-x)\,dz\,dx\\ &=\int_B\left[\int_{\mathbb{R}^d} f_X(x)f_Y(z-x)\,dx\right]dz. \end{aligned}

The second equality uses the translation \(z=x+y\) in the inner integral, whose Jacobian determinant is one. The integrand is nonnegative and measurable, so Tonelli’s theorem justifies changing the order of integration. The final expression shows that the function in brackets is a density of \(Z\). Taking \(B=\mathbb{R}^d\) also shows it integrates to one. \(\square\)

At a fixed output \(z\), the region that contributes is

R_z=\{x\in\mathbb{R}^d:f_X(x)>0\ \text{and}\ f_Y(z-x)>0\}.

Equivalently, if the two densities are supported on sets \(A\) and \(C\), integrate over \(A\cap(z-C)\), where \(z-C=\{z-y:y\in C\}\). If this region is empty, the convolution is zero at that output (up to the usual almost-everywhere qualification for densities). The sum’s possible values lie in \(A+C=\{a+c:a\in A,\ c\in C\}\).

Worked Continuous Convolution

Worked Example: Adding Two Uniform Square Vectors

Let \(X\) and \(Y\) be independent, each uniform on the unit square \([0,1]^2\). Find the density of \(Z=X+Y\), and evaluate it at \(z=(0.75,1.5)\).

Each vector has joint density \(1\) on the square and \(0\) elsewhere. Write \(z=(z_1,z_2)\) and \(x=(x_1,x_2)\). A point \(x\) contributes only when \(x\in[0,1]^2\) and \(z-x\in[0,1]^2\). In coordinate \(i\), these conditions are \(0\leq x_i\leq1\) and \(z_i-1\leq x_i\leq z_i\). Thus

\max(0,z_i-1)\leq x_i\leq\min(1,z_i),\qquad i=1,2.

The two coordinate intervals form a rectangle. Since the integrand is one on that rectangle, its area is the density:

f_Z(z_1,z_2)= \begin{cases} \bigl(\min(1,z_1)-\max(0,z_1-1)\bigr) \bigl(\min(1,z_2)-\max(0,z_2-1)\bigr), &0\leq z_1,z_2\leq2,\\ 0,&\text{otherwise}. \end{cases}

For \(0\leq t\leq2\), each interval length is \(t\) when \(t\leq1\) and \(2-t\) when \(t\geq1\). Hence the same density can be written as

f_Z(z_1,z_2)= \begin{cases} (1-|z_1-1|)(1-|z_2-1|),&0\leq z_1,z_2\leq2,\\ 0,&\text{otherwise}. \end{cases}

At \((0.75,1.5)\), the two interval lengths are \(0.75\) and \(0.5\), so \(f_Z(0.75,1.5)=0.75(0.5)=0.375\). The support is the square \([0,2]^2\); the density integrates to one because each one-dimensional triangular factor has area \(1\), and the joint density is their product.

Discrete Vector Convolution

For countable-valued vectors, the possible output \(z\) can be obtained from many different pairs of input values. Independence makes the probability of each pair a product. The probabilities of all pairs that produce the same \(z\) must then be added.

Theorem: If \(X\) and \(Y\) are independent countable-valued vectors, then for every \(z\),

P(X+Y=z)=\sum_x P(X=x)P(Y=z-x).

Proof. The events \(\{X=x,\ Y=z-x\}\), as \(x\) ranges over the possible values of \(X\), are disjoint. Their union is exactly \(\{X+Y=z\}\). Countable additivity and independence therefore give

P(X+Y=z)=\sum_x P(X=x,Y=z-x) =\sum_xP(X=x)P(Y=z-x).

Each term is nonnegative, so the sum is well-defined, including when written as an infinite series. \(\square\)

Worked Example: Adding Two Finite-Support Vectors

Suppose independent vectors \(X,Y\in\mathbb{Z}^2\) have the following distributions:

\(X=(0,0)\) with probability \(0.4\), and \(X=(1,0)\) with probability \(0.6\).
\(Y=(0,0)\) with probability \(0.5\), \(Y=(0,1)\) with probability \(0.25\), and \(Y=(1,1)\) with probability \(0.25\).

For each possible pair, multiply the probabilities and record the vector sum:

\(X\)	\(Y\)	\(X+Y\)	Pair probability
\((0,0)\)	\((0,0)\)	\((0,0)\)	\(0.4(0.5)=0.20\)
\((0,0)\)	\((0,1)\)	\((0,1)\)	\(0.4(0.25)=0.10\)
\((0,0)\)	\((1,1)\)	\((1,1)\)	\(0.4(0.25)=0.10\)
\((1,0)\)	\((0,0)\)	\((1,0)\)	\(0.6(0.5)=0.30\)
\((1,0)\)	\((0,1)\)	\((1,1)\)	\(0.6(0.25)=0.15\)
\((1,0)\)	\((1,1)\)	\((2,1)\)	\(0.6(0.25)=0.15\)

The output \((1,1)\) is produced by two distinct pairs, so their probabilities must be combined. The mass function is

\begin{aligned} P(Z=(0,0))&=0.20,& P(Z=(0,1))&=0.10,\\ P(Z=(1,0))&=0.30,& P(Z=(1,1))&=0.10+0.15=0.25,\\ P(Z=(2,1))&=0.15. \end{aligned}

These are all possible output vectors. As a normalization check, \(0.20+0.10+0.30+0.25+0.15=1\).

One Continuous Vector and One Discrete Vector

A useful extension follows by conditioning on the discrete vector’s value. If \(X\) has density \(f_X\), \(Y\) is countable-valued with mass function \(p_Y\), and they are independent, then \(Z=X+Y\) has density

f_Z(z)=\sum_y p_Y(y)f_X(z-y)

almost everywhere. Each possible value \(y\) shifts the density of \(X\), and the shifted densities are weighted by the probabilities of those values. The sum is a density because its terms are nonnegative and their integrals sum to \(\sum_y p_Y(y)=1\), by Tonelli’s theorem.

Worked Example: A Discrete Shift of a Uniform Square

Let \(X\) be uniform on \([0,1]^2\). Independently, let \(Y=(0,0)\) with probability \(1/3\) and \(Y=(1/2,0)\) with probability \(2/3\). Find the density of \(Z=X+Y\).

The first value of \(Y\) contributes a uniform density on \([0,1]\times[0,1]\), weighted by \(1/3\). The second contributes a uniform density on \([1/2,3/2]\times[0,1]\), weighted by \(2/3\). Both squares have area one. Their overlap means the two contributions must be added there. Writing \(z=(u,v)\), the result is

f_Z(u,v)= \begin{cases} \frac13,&0\leq u<\frac12,\ 0\leq v\leq1,\\ 1,&\frac12\leq u\leq1,\ 0\leq v\leq1,\\ \frac23,&1<u\leq\frac32,\ 0\leq v\leq1,\\ 0,&\text{otherwise}. \end{cases}

The middle region has density \(1/3+2/3=1\). Boundary values do not affect a density. The total integral is \((1/2)(1/3)+(1/2)(1)+(1/2)(2/3)=1\), as required.

Common Pitfalls and a Practical Check

The main difficulty is usually not multiplying densities; it is identifying the correct region or the correct set of contributing pairs. At a proposed output \(z\), a continuous input \(x\) is admissible only if \(x\) is possible for \(X\) and \(z-x\) is possible for \(Y\). In a discrete calculation, every pair yielding \(z\) contributes, including pairs that yield the same output.

Wrong: multiply coordinate-wise marginal densities as though vector coordinates were independent. Correction: use each vector’s joint density; factor it only when independence of its coordinates is known.
Wrong: integrate over the support of \(X\) alone. Correction: intersect that support with \(z-\operatorname{supp}(Y)\), so the required value \(z-x\) is possible for \(Y\).
Wrong: keep just one pair when several discrete input pairs produce the same sum. Correction: sum the probabilities of all such pairs.

A reliable calculation has three checks: the region enforces both support restrictions, the possible outputs lie in the sum of the two supports, and the resulting density or mass function has total probability one. These checks are particularly useful when the geometry is not rectangular or when different pairs lead to the same output.

Check Your Understanding

Use the support restrictions and independence carefully in each problem.

For independent vectors \(X,Y\in\mathbb{R}^2\) with densities, write the convolution integral for the density of \(X+Y\).
If \(X\) is supported on \(A\) and \(Y\) on \(C\), describe the integration region at output \(z\).
For countable-valued independent vectors, explain why probabilities from different input pairs must be added when they produce the same sum.
If a discrete vector \(Y\) has two possible values, describe how each value contributes to the density of \(X+Y\) when \(X\) has a density.
Why does a product of coordinate-wise marginal densities not generally give the joint density of a random vector?