Differences of Independent Continuous Variables

Why Differences Need Their Own Convolution

A difference $X-Y$ is a sum with a sign change: it compares the two random quantities on the same scale. For example, a difference can represent a treatment-minus-control measurement or the gap between two independent waiting times. Even when the densities of $X$ and $Y$ are simple, the density of their difference depends on how their possible values overlap after one distribution is shifted.

The key is to account for every pair of values that produces the same difference. If $X-Y=z$, then $X=z+Y$. Integrating over the possible values of $Y$ gives the convolution formula. Keeping track of this shift is also what determines the support and prevents a common sign error.

Definition: Let $X$ and $Y$ be independent continuous random variables with densities $f_X$ and $f_Y$. The density of $Z=X-Y$ is $$ f_Z(z)=\int_{-\infty}^{\infty} f_X(z+y)f_Y(y)\,dy =\int_{-\infty}^{\infty} f_X(x)f_Y(x-z)\,dx. $$ The integrals are interpreted as Lebesgue integrals; their integrands are nonnegative, so the value is well-defined, possibly infinite on a set of measure zero.

Why the Convolution Formula Is Correct

Independence matters: it gives the joint density $f_{X,Y}(x,y)=f_X(x)f_Y(y)$. The difference then collects the joint probability over all points $(x,y)$ lying along lines $x-y=z$. The following argument establishes the formula as a density, without assuming in advance that a derivative can be passed through an integral.

Theorem: If $X$ and $Y$ are independent and have densities $f_X$ and $f_Y$, then $Z=X-Y$ has the density given in the definition above.

Start with an arbitrary measurable set.
For any Borel set $A\subseteq\mathbb{R}$, independence gives $P(Z\in A)=\int_{\mathbb{R}}\int_{\mathbb{R}}\mathbf{1}_A(x-y)f_X(x)f_Y(y)\,dx\,dy$.

Change variables for each fixed $y$.
Set $z=x-y$, so $x=z+y$ and $dx=dz$. Tonelli's theorem applies because the integrand is nonnegative, giving $P(Z\in A)=\int_A\int_{\mathbb{R}}f_X(z+y)f_Y(y)\,dy\,dz$.

Identify the density.
The inner integral is nonnegative and measurable in $z$. Taking $A=\mathbb{R}$ shows its integral over $z$ is $P(Z\in\mathbb{R})=1$. Thus it is a density of $Z$, proving the first formula. In the inner integral, the substitution $x=z+y$ gives the equivalent second formula.

This proof uses neither continuity of the densities nor pointwise differentiation of a cumulative distribution function. The density formula holds almost everywhere, as usual for densities. If either input density is zero outside a specified range, the integrand also vanishes whenever the shifted ranges do not overlap.

Support and Overlap

Suppose $X$ takes values in $[a,b]$ and $Y$ takes values in $[c,d]$, where $a<b$ and $c<d$. Every possible difference lies between the smallest possible value $a-d$ and the largest $b-c$. More precisely, in the integral written with $x$, only values in the overlap of the two intervals $[a,b]$ and $[z+c,z+d]$ can contribute.

Support takeaway: If $X$ and $Y$ have interval supports $[a,b]$ and $[c,d]$, then $X-Y$ is supported within $[a-d,b-c]$. If each input density is positive almost everywhere in the interior of its interval, the support of $X-Y$ is the closure $[a-d,b-c]$. For a particular $z$, the density integral is nonzero only when the shifted intervals overlap in a set of positive length (or, for general densities, when their product contributes on a set of positive measure).

Thinking in terms of overlap is often more useful than trying to guess a piecewise formula. As $z$ changes, the shifted interval moves; the density can increase, remain level, and then decrease as the overlap grows and shrinks.

Worked Example: Two Uniform Variables With Unequal Ranges

Let $X\sim\operatorname{Uniform}(0,3)$ and $Y\sim\operatorname{Uniform}(1,2)$ be independent. Find the density of $Z=X-Y$.

The densities are $f_X(x)=1/3$ on $[0,3]$ and $f_Y(y)=1$ on $[1,2]$, and are zero elsewhere. The support of $Z$ is $[0-2,3-1]=[-2,2]$. Using the $x$-integral, for each $z$ the integrand is $1/3$ where $x\in[0,3]\cap[z+1,z+2]$, and zero elsewhere. Therefore the density is one third of the length of this overlap.

For $-2\le z\le -1$, the shifted interval $[z+1,z+2]$ enters $[0,3]$ from the left. The overlap length is $(z+2)-0=z+2$.

For $-1\le z\le 1$, the shifted interval lies entirely inside $[0,3]$. Its length, and hence the overlap length, is $1$.

For $1\le z\le 2$, the shifted interval exits $[0,3]$ on the right. The overlap length is $3-(z+1)=2-z$. Outside $[-2,2]$ there is no overlap. Thus

f_Z(z)= \begin{cases} \dfrac{z+2}{3}, & -2\le z\le -1,\\[4pt] \dfrac{1}{3}, & -1<z\le 1,\\[4pt] \dfrac{2-z}{3}, & 1<z\le 2,\\[4pt] 0, & \text{otherwise}. \end{cases}

The choices at the two breakpoints do not affect the distribution because changing a density at individual points does not change its integrals. As a check, the total mass is the area of two triangles and a rectangle: $\tfrac12(1)(1/3)+2(1/3)+\tfrac12(1)(1/3)=1$.

Two Further Calculations

Worked Example: Difference of Independent Normal Variables

Let $X\sim N(\mu_X,\sigma_X^2)$ and $Y\sim N(\mu_Y,\sigma_Y^2)$ be independent, with $\sigma_X^2>0$ and $\sigma_Y^2>0$. Use convolution to find the density of $Z=X-Y$.

Write $q=\sigma_X^2+\sigma_Y^2$ and $\delta=z-(\mu_X-\mu_Y)$. Substitution into the convolution formula gives

f_Z(z)=\frac{1}{2\pi\sigma_X\sigma_Y} \int_{-\infty}^{\infty} \exp\left\{-\frac{1}{2}\left[ \frac{(z+y-\mu_X)^2}{\sigma_X^2} +\frac{(y-\mu_Y)^2}{\sigma_Y^2} \right]\right\}\,dy.

Complete the square in $y$. The quadratic expression inside the brackets satisfies

\frac{(z+y-\mu_X)^2}{\sigma_X^2} +\frac{(y-\mu_Y)^2}{\sigma_Y^2} = \frac{q}{\sigma_X^2\sigma_Y^2}(y-m)^2 +\frac{\delta^2}{q}, \qquad m=\frac{\sigma_Y^2(\mu_X-z)+\sigma_X^2\mu_Y}{q}.

To check the constant term, the minimum over $y$ of the left side is $(z-\mu_X+\mu_Y)^2/q=\delta^2/q$; the coefficient of $y^2$ is $q/(\sigma_X^2\sigma_Y^2)$, and the minimizing value is $m$. The remaining Gaussian integral is $\sqrt{2\pi\sigma_X^2\sigma_Y^2/q}$. Substitution and simplification yield

f_Z(z)=\frac{1}{\sqrt{2\pi q}} \exp\left\{-\frac{[z-(\mu_X-\mu_Y)]^2}{2q}\right\}.

Hence $X-Y\sim N(\mu_X-\mu_Y,\sigma_X^2+\sigma_Y^2)$. The mean is a difference, but the variances add: subtracting $Y$ changes the sign of its centered values, not their squared spread.

Worked Example: Difference of Independent Exponential Variables

Let $X$ and $Y$ be independent exponential variables with rates $2$ and $1$, respectively. Their densities are $f_X(x)=2e^{-2x}$ and $f_Y(y)=e^{-y}$ for nonnegative arguments, and zero otherwise. Find the density of $Z=X-Y$.

For $z\ge0$, use $f_Z(z)=\int f_X(z+y)f_Y(y)\,dy$. Both arguments are nonnegative when $y\ge0$, so

f_Z(z)=\int_0^\infty 2e^{-2(z+y)}e^{-y}\,dy =2e^{-2z}\int_0^\infty e^{-3y}\,dy =\frac{2}{3}e^{-2z}.

For $z<0$, use $f_Z(z)=\int f_X(x)f_Y(x-z)\,dx$. The density factors are nonzero for $x\ge0$; because $z<0$, this also ensures $x-z\ge0$. Thus

f_Z(z)=\int_0^\infty 2e^{-2x}e^{-(x-z)}\,dx =2e^z\int_0^\infty e^{-3x}\,dx =\frac{2}{3}e^z.

The result has support on the whole real line. Its total mass is $\int_{-\infty}^0(2/3)e^z\,dz+\int_0^\infty(2/3)e^{-2z}\,dz=2/3+1/3=1$. The different decay rates on the two sides reflect the different rates of the two input variables.

A Sign Check and a Common Mistake

The two equivalent convolution expressions are $f_X(z+y)f_Y(y)$, integrated over $y$, and $f_X(x)f_Y(x-z)$, integrated over $x$. The minus sign in the second expression follows from solving $z=x-y$ for $y$: $y=x-z$.

Wrong reasoning: Replace $f_Y(x-z)$ by $f_Y(z-x)$ in the second integral because both expressions contain $x$ and $z$.
Correction: The equation defining the difference is $z=x-y$, so $y=x-z$, not $z-x$. Reversing the argument generally changes the distribution. It is harmless only in special cases, such as when $f_Y$ is symmetric about zero.

A practical check is to test the support before integrating. If $X\in[a,b]$ and $Y\in[c,d]$, the answer must vanish outside $[a-d,b-c]$. Also check that the resulting density integrates to one; this catches many incorrect limits and missing pieces.

Why this matters: The difference distribution answers comparison questions directly. For independent continuous variables, $P(X>Y)=P(X-Y>0)=\int_0^\infty f_Z(z)\,dz$. The same density also describes the magnitude and direction of random gaps, while its support and shape reveal which gaps are possible and how much probability they receive.

Check Your Understanding

Use the convolution formula and support arguments to answer these questions.

If $X$ is supported on $[2,5]$ and $Y$ on $[-1,4]$, what interval must contain the support of $X-Y$?
Write the density formula for $X-Y$ in both the $y$-integral and $x$-integral forms. How does solving $z=x-y$ determine the sign?
If $X\sim\operatorname{Uniform}(0,2)$ and $Y\sim\operatorname{Uniform}(0,1)$ independently, on what interval is $X-Y$ supported?
Why does the variance of a difference of independent normal variables equal the sum of their variances?
For independent continuous $X$ and $Y$, express $P(X>Y)$ using the density of $X-Y$.