Convolution as the Distribution of a Sum

Why a Sum Requires a Transformation

Knowing the densities of $X$ and $Y$ does not by itself give the density of their sum. The value $S=X+Y$ can arise from many pairs: for a fixed $s$, every pair $(x,s-x)$ has sum $s$. To find the density of $S$, we must account for all such pairs, not just one inverse value.

When $X$ and $Y$ are independent and continuous, their joint density is the product of their marginal densities. Transforming the pair to a sum and one retained coordinate then reduces the problem to integrating that transformed joint density over the coordinate that is no longer needed.

Definition: For measurable, nonnegative functions $f$ and $g$ on the real line, their convolution is the function, possibly infinite at some points, $$ (f*g)(s)=\int_{-\infty}^{\infty} f(x)g(s-x)\,dx. $$ For independent random variables $X$ and $Y$ with densities $f_X$ and $f_Y$, this integral gives a version of the density of $X+Y$, as established below.

Deriving the Convolution Density

Assume $X$ and $Y$ are independent real-valued random variables with Lebesgue densities $f_X$ and $f_Y$. Thus these functions are measurable, nonnegative, and each integrates to one. Independence gives the joint density $f_{X,Y}(x,y)=f_X(x)f_Y(y)$.

Set $S=X+Y$, and retain $V=X$ as the second coordinate. The transformation and its inverse are

(s,v)=(x+y,x),\qquad (x,y)=(v,s-v).

The absolute value of the Jacobian determinant of the inverse transformation is one. The change-of-variables formula therefore gives the joint density of $(S,V)$:

f_{S,V}(s,v)=f_X(v)f_Y(s-v).

To obtain the marginal density of $S$, integrate out $v$. This yields the convolution formula.

Theorem: If $X$ and $Y$ are independent real-valued random variables with Lebesgue densities $f_X$ and $f_Y$, then $S=X+Y$ has density $$ f_S(s)=\int_{-\infty}^{\infty} f_X(x)f_Y(s-x)\,dx $$ for almost every $s$. The integral is finite for almost every $s$, is measurable as a function of $s$, and integrates to one.

Proof. The joint density of $(X,Y)$ is $f_X(x)f_Y(y)$. The transformation $(x,y)\mapsto(s,v)=(x+y,x)$ is one-to-one, with inverse $(s,v)\mapsto(v,s-v)$. The inverse Jacobian matrix is

\begin{pmatrix} 0 & 1\\ 1 & -1 \end{pmatrix}, \qquad \left|\det \begin{pmatrix} 0 & 1\\ 1 & -1 \end{pmatrix}\right|=1.

The change-of-variables formula gives $f_{S,V}(s,v)=f_X(v)f_Y(s-v)$. Integrating this nonnegative measurable joint density over $v$ gives a marginal density for $S$; Tonelli's theorem justifies the integral and ensures its measurability. Its total integral is

\begin{aligned} \int_{-\infty}^{\infty}\int_{-\infty}^{\infty} f_X(v)f_Y(s-v)\,dv\,ds &=\int_{-\infty}^{\infty} f_X(v) \left(\int_{-\infty}^{\infty} f_Y(s-v)\,ds\right)dv\\ &=\int_{-\infty}^{\infty} f_X(v)\,dv=1. \end{aligned}

The inner integral equals one by translation invariance of Lebesgue integration. Since the convolution is nonnegative and has finite integral one, it can be infinite only on a set of measure zero. Thus the displayed formula defines a density almost everywhere. $\square$

Support and Integrability

The formula also shows how the supports of the summands constrain the sum. Suppose $f_X$ vanishes outside a measurable set $A$, and $f_Y$ vanishes outside a measurable set $B$, up to sets of measure zero. For a given $s$, the integrand can be nonzero only when $x\in A$ and $s-x\in B$. Equivalently, $x\in A\cap(s-B)$, where $s-B=\{s-y:y\in B\}$. If no such $x$ exists except on a null set, the convolution is zero at that $s$.

In particular, if the densities are concentrated on intervals $[a,b]$ and $[c,d]$, respectively, the sum is concentrated on $[a+c,b+d]$. If each density is positive almost everywhere in the interior of its interval, the sum has support $[a+c,b+d]$ in the usual topological sense. For general supports, the set of possible sums suggests the candidate support, but the convolution integral—not set addition alone—determines where the density is positive.

The assumptions that the marginal densities are integrable and independence holds are enough for the convolution to be a density, but not to make its defining integral finite at every single $s$. The theorem guarantees finiteness almost everywhere. Particular models, such as the examples below, may yield a finite formula everywhere on the relevant support.

Worked Example: The Sum of Two Independent Exponentials

Let $X$ and $Y$ be independent exponential random variables with rate $1$, so each has density $f(x)=e^{-x}$ for $x>0$ and zero otherwise. For $s\leq 0$, no pair of positive values can sum to $s$, so $f_S(s)=0$. For $s>0$, the integrand is nonzero only if $0<x<s$. Therefore

f_S(s)=\int_0^s e^{-x}e^{-(s-x)}\,dx =e^{-s}\int_0^s 1\,dx =se^{-s},\qquad s>0.

The density integrates to one:

\int_0^\infty se^{-s}\,ds =\left[-(s+1)e^{-s}\right]_0^\infty=1.

The support is $[0,\infty)$, and the density is zero at the boundary $s=0$. The factor $s$ arises because, for a fixed positive sum, the possible values of $x$ range across an interval of length $s$.

Worked Example: The Sum of Two Independent Normal Variables

Let $X\sim N(\mu_1,\sigma_1^2)$ and $Y\sim N(\mu_2,\sigma_2^2)$ be independent, with $\sigma_1^2,\sigma_2^2>0$. Write $a=\sigma_1^2$, $b=\sigma_2^2$, and $m=s-\mu_1-\mu_2$. The convolution integrand is the product of the two normal densities. Its exponent can be rewritten as

\frac{(x-\mu_1)^2}{a}+ \frac{(s-x-\mu_2)^2}{b} = \frac{a+b}{ab} \left(x-\mu_1-\frac{a m}{a+b}\right)^2 +\frac{m^2}{a+b}.

Substituting this identity into the convolution integral separates the terms that depend on $x$ from those that depend only on $s$:

\begin{aligned} f_S(s) &=\frac{1}{2\pi\sqrt{ab}}\, e^{-m^2/[2(a+b)]} \int_{-\infty}^{\infty} \exp\left\{-\frac{a+b}{2ab} \left(x-\mu_1-\frac{a m}{a+b}\right)^2\right\}\,dx\\ &=\frac{1}{2\pi\sqrt{ab}}\, e^{-m^2/[2(a+b)]} \sqrt{\frac{2\pi ab}{a+b}}\\ &=\frac{1}{\sqrt{2\pi(a+b)}}\, \exp\left\{-\frac{(s-\mu_1-\mu_2)^2}{2(a+b)}\right\}. \end{aligned}

The integral in the second line is the Gaussian integral after rescaling. The result is the density of $N(\mu_1+\mu_2,\sigma_1^2+\sigma_2^2)$. Its support is the entire real line, consistent with adding two variables whose densities are positive everywhere.

Worked Example: A Uniform Variable Plus an Exponential Variable

Let $X$ be uniform on $(0,1)$, and let $Y$ be exponential with rate $1$, independent of $X$. Here $f_X(x)=1$ for $0<x<1$, while $f_Y(y)=e^{-y}$ for $y>0$. A sum must be positive. For $s>0$, the integrand is nonzero when $0<x<1$ and $x<s$, so the integration interval is $0<x<\min(1,s)$. Hence

f_S(s)= \begin{cases} \displaystyle\int_0^s e^{-(s-x)}\,dx=1-e^{-s},&0<s<1,\\[6pt] \displaystyle\int_0^1 e^{-(s-x)}\,dx=(e-1)e^{-s},&s\geq 1,\\[6pt] 0,&s\leq 0. \end{cases}

The support is $[0,\infty)$. To check normalization, integrate the two positive pieces:

\int_0^1(1-e^{-s})\,ds=e^{-1}, \qquad \int_1^\infty(e-1)e^{-s}\,ds=1-e^{-1}.

Their sum is $1$, as required. The change in formula at $s=1$ occurs because the allowable interval for $x$ stops growing once it reaches the full support of $X$.

Convolution Can Be Reordered

The density of a sum should not depend on which summand is named first. The convolution formula makes this symmetry explicit. It also extends naturally to three independent variables.

Theorem: For nonnegative integrable densities $f,g,h$ on the real line, convolution is commutative and associative almost everywhere: $$ f*g=g*f,\qquad (f*g)*h=f*(g*h). $$

Proof. In the defining integral for $g*f$, substitute $u=s-x$. Then $dx$ has absolute value $du$, and

(g*f)(s)=\int_{-\infty}^{\infty}g(s-u)f(u)\,du=(f*g)(s).

For associativity, expand the left-hand side and apply Tonelli's theorem, which permits changing the order of integration because the integrands are nonnegative:

\begin{aligned} ((f*g)*h)(s) &=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} f(x)g(u-x)h(s-u)\,dx\,du\\ &=\int_{-\infty}^{\infty}f(x) \left(\int_{-\infty}^{\infty}g(u-x)h(s-u)\,du\right)dx. \end{aligned}

In the inner integral put $y=u-x$. It becomes $\int g(y)h(s-x-y)\,dy=(g*h)(s-x)$. Thus the result is $\int f(x)(g*h)(s-x)\,dx=(f*(g*h))(s)$. These equalities hold almost everywhere, since integrable densities and their convolutions are identified up to changes on null sets. $\square$

Consequently, for three independent variables with densities $f_X,f_Y,f_Z$, the density of their sum is $(f_X*f_Y)*f_Z$, or equivalently $f_X*(f_Y*f_Z)$. The grouping of the summands does not alter the resulting distribution.

Common Mistake: Using Only One Pair

A tempting but incorrect argument is to write $S=s$ as $Y=s-X$, substitute this relation into $f_Y$, and conclude that the density is just $f_Y(s-X)$. That expression still depends on the random variable $X$, and it accounts for neither the density of $X$ nor the contributions from all possible values of $X$.

The correction is to transform the joint density of the pair. Independence gives $f_X(x)f_Y(y)$; the sum transformation sets $y=s-x$; and marginalizing over the full range of $x$ gives $\int f_X(x)f_Y(s-x)\,dx$. This integration is essential: a sum can be formed by many different pairs of values.

Key takeaway: For independent continuous random variables, the density of a sum is the convolution of their densities. Use the supports to determine the integration region, and remember that the formula is guaranteed to be finite and valid as a density almost everywhere.

Check Your Understanding

Try these questions using the transformation and support reasoning developed above.

For $S=X+Y$, explain why retaining $V=X$ makes the transformation one-to-one, and state its inverse.
If $X$ is supported on $[2,5]$ and $Y$ on $[-1,4]$, on what interval must $X+Y$ be supported?
Why does the convolution theorem guarantee a finite density only almost everywhere, rather than at every real number?
Two independent exponential variables have rate $1$. Use the convolution formula to explain why their sum density contains a factor of $s$.
State the additional assumption that lets the joint density of two variables be written as the product of their marginal densities.