Distributions of Differences of Independent Variables

Why Subtraction Requires a Reflection

A difference \(Z=X-Y\) is not obtained by using the sum formula unchanged. To see why, fix a possible value \(z\) of the difference. Whenever \(Y=y\), the equation \(X-Y=z\) requires \(X=z+y\). The density of \(X\) must therefore be evaluated at a shifted value, and this shift determines the correct convolution formula.

Throughout, assume \(X\) and \(Y\) are independent continuous random variables with densities \(f_X\) and \(f_Y\). Densities are understood up to changes on sets of Lebesgue measure zero; the formulas below specify a density for \(Z\).

Definition: The reflected density associated with \(Y\) is the density of \(-Y\), given by \(f_{-Y}(u)=f_Y(-u)\). The difference \(X-Y\) can be written as the sum \(X+(-Y)\), so its density is the convolution of \(f_X\) with this reflected density.

The Density Formula

Independence gives the joint density \(f_{X,Y}(x,y)=f_X(x)f_Y(y)\). Transform the variables by setting \(Z=X-Y\) and retaining \(Y\). The inverse transformation is \(x=z+y\), \(y=y\), and its Jacobian determinant has absolute value 1. Thus the joint density of \((Z,Y)\) is \(f_X(z+y)f_Y(y)\). Integrating out \(y\) gives the density of \(Z\).

Theorem: If \(X\) and \(Y\) are independent and have densities \(f_X\) and \(f_Y\), then \(Z=X-Y\) has density

f_Z(z)=\int_{-\infty}^{\infty} f_X(z+y)f_Y(y)\,dy =\int_{-\infty}^{\infty} f_X(x)f_Y(x-z)\,dx.

Equivalently, using the reflected density,

f_Z(z)=\int_{-\infty}^{\infty} f_X(x)f_{-Y}(z-x)\,dx =(f_X*f_{-Y})(z).

The equivalence of the first two integrals follows from the substitution \(x=z+y\). The last expression is the usual convolution formula applied to \(X\) and \(-Y\).

Why the Formula Is a Density

The change of variables above also justifies the formula directly. For any Borel set \(A\subseteq\mathbb{R}\), the probability that \(Z\) lies in \(A\) is

P(Z\in A) =\int_{\mathbb{R}}\int_{\mathbb{R}} \mathbf{1}_{A}(x-y)f_X(x)f_Y(y)\,dx\,dy.

The integrand is nonnegative, so Tonelli’s theorem permits changing the order of integration and making the substitution \(z=x-y\) in the inner integral. Since \(x=z+y\) and \(dx=dz\),

P(Z\in A) =\int_A\left[\int_{\mathbb{R}}f_X(z+y)f_Y(y)\,dy\right]dz.

The bracketed function is nonnegative and measurable, and this identity shows that it is a density of \(Z\). Taking \(A=\mathbb{R}\) also verifies its total integral:

\int_{\mathbb{R}} f_Z(z)\,dz =\int_{\mathbb{R}}\int_{\mathbb{R}}f_X(z+y)f_Y(y)\,dy\,dz =\int_{\mathbb{R}}f_Y(y)\left[\int_{\mathbb{R}}f_X(z+y)\,dz\right]dy =1.

The final equality holds because a translation does not change the integral of \(f_X\), and both \(f_X\) and \(f_Y\) integrate to 1. The interchange is valid by Tonelli’s theorem, again because the integrand is nonnegative. \(\square\)

Determining the Support

A useful first step is to identify where the two factors in the integral can both be positive. In the \(x\)-form of the formula, a contribution to \(f_Z(z)\) requires both \(f_X(x)>0\) and \(f_Y(x-z)>0\). Equivalently, \(x\) must be a possible value of \(X\) while \(x-z\) is a possible value of \(Y\).

For interval supports, this gives a particularly simple rule. Suppose \(X\) has support \([a,b]\) and \(Y\) has support \([c,d]\), with each density positive almost everywhere in the interior of its interval and zero outside it. Then the support of \(X-Y\) is \([a-d,b-c]\). Indeed, differences between values in those intervals cannot fall outside this range. Conversely, every value strictly between the endpoints can be expressed as \(x-y\) with \(x\) and \(y\) in the interiors, and the overlap contributing to the density has positive length. The endpoints belong to the support as limits of such values.

For distributions whose supports have gaps, the interval rule need not describe the support exactly. In that case, use the density formula and check where the two density factors have a positive overlap.

Worked Example: Difference of Shifted Uniform Variables

Let \(X\sim U(0,2)\) and \(Y\sim U(1,4)\), independently. Find the density and support of \(Z=X-Y\).

The support rule gives \([0-4,2-1]=[-4,1]\). The densities are \(f_X(x)=1/2\) on \([0,2]\) and \(f_Y(y)=1/3\) on \([1,4]\), so the product in the integral is \(1/6\) wherever both support conditions hold. For fixed \(z\), these conditions are \(0\leq x\leq2\) and \(1\leq x-z\leq4\), or \(x\in[z+1,z+4]\). Thus the density is one-sixth of the length of \([0,2]\cap[z+1,z+4]\).

For \(-4\leq z\leq-2\), this intersection is \([0,z+4]\), with length \(z+4\). For \(-2\leq z\leq-1\), it is \([0,2]\), with length 2. For \(-1\leq z\leq1\), it is \([z+1,2]\), with length \(1-z\). Outside \([-4,1]\), there is no overlap. Therefore,

f_Z(z)= \begin{cases} (z+4)/6, & -4\leq z\leq-2,\\ 1/3, & -2\leq z\leq-1,\\ (1-z)/6, & -1\leq z\leq1,\\ 0, & \text{otherwise}. \end{cases}

As a normalization check, the integral of the numerator over the three pieces is \(2+2+2=6\), so the density integrates to \(6/6=1\). The support is bounded but not symmetric about zero, reflecting the locations and widths of the two original intervals.

Worked Examples with Unbounded Supports

Worked Example: Difference of Two Exponential Variables

Let \(X\) and \(Y\) be independent exponential variables with respective rates 2 and 3. Their densities are \(f_X(x)=2e^{-2x}\) for \(x\geq0\) and \(f_Y(y)=3e^{-3y}\) for \(y\geq0\). Since each variable is nonnegative, their difference can be any real number.

Use \(f_Z(z)=\int f_X(z+y)f_Y(y)\,dy\). The integrand is nonzero only when \(y\geq0\) and \(z+y\geq0\), so the lower integration limit is \(\max(0,-z)\). The integrand on that range is \(6e^{-2z}e^{-5y}\).

If \(z\geq0\), the lower limit is 0, and

f_Z(z)=6e^{-2z}\int_0^\infty e^{-5y}\,dy =\frac{6}{5}e^{-2z}.

If \(z<0\), the lower limit is \(-z\), and

f_Z(z)=6e^{-2z}\int_{-z}^\infty e^{-5y}\,dy =\frac{6}{5}e^{-2z}e^{5z} =\frac{6}{5}e^{3z}.

Hence \(f_Z(z)=(6/5)e^{3z}\) for \(z<0\), and \(f_Z(z)=(6/5)e^{-2z}\) for \(z\geq0\). The negative tail decays at rate 3 and the positive tail at rate 2, so the difference has an asymmetric density. Its total integral is \((6/5)(1/3+1/2)=1\), as required.

Worked Example: A Uniform Variable Minus an Exponential Variable

Let \(X\sim U(1,3)\) and let \(Y\) be exponential with rate 1, independently. The possible differences extend without bound in the negative direction because \(Y\) can be arbitrarily large. The largest possible difference is 3, so the support is \((-\infty,3]\).

The density \(f_X\) is \(1/2\) on \([1,3]\), and \(f_Y(y)=e^{-y}\) for \(y\geq0\). In the formula involving \(y\), we need \(y\geq0\) and \(1\leq z+y\leq3\). Thus \(y\) ranges from \(\max(0,1-z)\) to \(3-z\), provided \(z\leq3\). It follows that

f_Z(z)=\frac12\int_{\max(0,1-z)}^{3-z} e^{-y}\,dy,\qquad z\leq3.

When \(z<1\), the limits are \(1-z\) and \(3-z\), giving \(f_Z(z)=\tfrac12(e^{z-1}-e^{z-3})\). When \(1\leq z\leq3\), the limits are 0 and \(3-z\), giving \(f_Z(z)=\tfrac12(1-e^{z-3})\). For \(z>3\), the density is zero. In full,

f_Z(z)= \begin{cases} \frac12 e^{z-1}(1-e^{-2}), & z<1,\\ \frac12(1-e^{z-3}), & 1\leq z\leq3,\\ 0, & z>3. \end{cases}

The two expressions agree at \(z=1\), where both equal \((1-e^{-2})/2\). The density also tends to zero at the upper endpoint \(z=3\). To check the integral, the part below 1 integrates to \((1-e^{-2})/2\), while the part from 1 to 3 integrates to \((1+e^{-2})/2\). Their sum is 1.

A Common Mistake

A common error is to use the sum formula \( \int f_X(z-y)f_Y(y)\,dy \) for \(X-Y\) without changing a sign. That expression is the convolution formula for \(X+Y\), not \(X-Y\): it enforces \(X=z-y\), or \(X+Y=z\). For a difference, the defining equation is \(X=z+Y\), which gives \( \int f_X(z+y)f_Y(y)\,dy \). Equivalently, reflect \(Y\) first and convolve \(X\) with \(-Y\).

Another useful check is to ask whether the resulting support is plausible. If \(X\) and \(Y\) are both nonnegative, their difference need not be nonnegative; it can extend in either direction. By contrast, if \(X\) is bounded above by \(b\) and \(Y\) is nonnegative, then \(X-Y\leq b\), which rules out any positive density above \(b\).

Key takeaway: For \(Z=X-Y\), reflect the density of \(Y\) and use convolution. Determine support from the overlap in the density integral, then check the piecewise formula at its breakpoints and verify that it integrates to 1.

Check Your Understanding

Use the difference-density formula and support reasoning to answer these questions.

If \(X\) has support \([2,5]\) and \(Y\) has support \([-1,3]\), with densities positive almost everywhere in their interiors, what is the support of \(X-Y\)?
Write the density integral for \(Z=X-Y\) using \(f_X\) and \(f_Y\), and explain why the argument of \(f_X\) is \(z+y\).
For independent \(X\sim U(0,1)\) and \(Y\sim U(2,5)\), identify the range of \(x\) that contributes to \(f_Z(z)\).
Why is the difference of two independent exponential variables not generally symmetric about zero when their rates differ?
What does it indicate if a proposed difference density integrates to less than 1?