Tutorials › VEE Mathematical Statistics › Conditioning to Find a Sum Distribution

Home  /  Tutorials  /  VEE Mathematical Statistics / Sums and Linear Combinations

VEE Mathematical Statistics · Sums and Linear Combinations · Tutorial 337 of 880

Conditioning to Find a Sum Distribution

Use the conditional distribution of one summand to derive the distribution of a sum, including when the summands are dependent.

Intermediate 10 min read

What You'll Learn

  • Apply the law of total probability to find the CDF or probability mass function of a sum.
  • Integrate conditional probabilities to obtain a sum distribution when the conditioning variable is continuous.
  • Distinguish conditional distributions from marginal distributions when summands are dependent.
  • Derive a sum density from its conditional density when the required regularity conditions hold.
  • Use the law of total expectation to check a calculated sum distribution.
  • Check support, boundary values, and total probability for errors.

Why Condition on One Summand?

A sum \(S=X+Y\) can be difficult to study directly, especially when the summands are dependent or their joint distribution is easier to describe conditionally than jointly. Conditioning on \(X\) fixes its value. Once \(X=x\), the event \(S\leq s\) is exactly the event \(Y\leq s-x\). Averaging that conditional probability over the possible values of \(X\) gives the distribution of the sum.

For independent summands, convolution provides a direct method. Conditioning is a broader framework: it also works when \(Y\)'s distribution changes with \(X\). The essential input is the conditional distribution of \(Y\) given \(X\), not just the two marginal distributions.

Definition: Let \(S=X+Y\). For any real number \(s\), the conditional method finds the distribution function of \(S\) from $$ F_S(s)=P(S\leq s)=E\left[P(Y\leq s-X\mid X)\right]. $$ If \(X\) is discrete, this expectation is a sum over its possible values. If \(X\) is continuous with density \(f_X\), it is an integral with respect to that density. The formula does not require \(X\) and \(Y\) to be independent.

The Conditioning Formula

For a discrete \(X\) with possible values \(x\), the law of total probability gives

$$ F_S(s)=\sum_x P(Y\leq s-x\mid X=x)P(X=x). $$

The same reasoning applies to individual probabilities. For any value \(t\),

$$ P(S=t)=\sum_x P(Y=t-x\mid X=x)P(X=x). $$

Terms for which \(t-x\) is not a possible value of \(Y\) given \(X=x\) contribute zero. This form is particularly useful for constructing a probability mass function.

If \(X\) is continuous with density \(f_X\), then

$$ F_S(s)=\int_{-\infty}^{\infty}P(Y\leq s-x\mid X=x)f_X(x)\,dx. $$

When the conditional distribution of \(Y\) given \(X=x\) has density \(f_{Y\mid X}(y\mid x)\), and the resulting integral can be differentiated under standard regularity conditions, the sum has density

$$ f_S(s)=\int_{-\infty}^{\infty}f_{Y\mid X}(s-x\mid x)f_X(x)\,dx. $$

The CDF formula is the safer starting point: it handles discrete distributions and boundary points directly. The density formula applies when the relevant variables have densities and differentiation is justified; it should not be used to represent point masses.

Why the Formula Holds

The possible values of a discrete \(X\) divide the sample space into disjoint events. On the event \(X=x\), the inequality \(X+Y\leq s\) becomes \(Y\leq s-x\). Summing the probabilities of this event over the disjoint values of \(X\) proves the discrete formula:

$$ \begin{aligned} P(S\leq s) &=\sum_x P(S\leq s,\ X=x)\\ &=\sum_x P(Y\leq s-x,\ X=x)\\ &=\sum_x P(Y\leq s-x\mid X=x)P(X=x). \end{aligned} $$

For continuous \(X\), the corresponding argument averages the same conditional probability across the density of \(X\), giving the integral. This is the law of total probability expressed as a conditional expectation. In particular, the law of total expectation also gives a useful check:

$$ E[S]=E\left[E[S\mid X]\right] =E\left[X+E[Y\mid X]\right], $$

provided the expectations exist. This identity checks a mean; it does not by itself determine the distribution.

Worked Example: A Discrete Sum with Dependent Summands

A hypothetical two-stage process produces a nonnegative count \(X\) with probabilities \(P(X=0)=0.25\), \(P(X=1)=0.50\), and \(P(X=2)=0.25\). The second-stage count \(Y\) has the following conditional probabilities. Find the probability mass function of \(S=X+Y\).

Value of \(X\)Conditional probabilities for \(Y\)
0\(P(Y=0\mid X=0)=0.60,\ P(Y=2\mid X=0)=0.40\)
1\(P(Y=0\mid X=1)=0.20,\ P(Y=1\mid X=1)=0.50,\ P(Y=2\mid X=1)=0.30\)
2\(P(Y=1\mid X=2)=0.40,\ P(Y=3\mid X=2)=0.60\)

Method. For each value of \(X\), add that value to each possible conditional value of \(Y\), multiply by \(P(X=x)\), and combine contributions that give the same sum.

When \(X=0\), the contributions are \(P(S=0)=0.25(0.60)=0.15\) and \(P(S=2)=0.25(0.40)=0.10\).

When \(X=1\), the contributions are \(P(S=1)=0.50(0.20)=0.10\), \(P(S=2)=0.50(0.50)=0.25\), and \(P(S=3)=0.50(0.30)=0.15\).

When \(X=2\), the contributions are \(P(S=3)=0.25(0.40)=0.10\) and \(P(S=5)=0.25(0.60)=0.15\).

Combine contributions for the same sum:

$$ \begin{aligned} P(S=0)&=0.15,\\ P(S=1)&=0.10,\\ P(S=2)&=0.10+0.25=0.35,\\ P(S=3)&=0.15+0.10=0.25,\\ P(S=4)&=0,\\ P(S=5)&=0.15. \end{aligned} $$

Check. The probabilities sum to \(0.15+0.10+0.35+0.25+0+0.15=1.00\). The support also agrees with the possible conditional outcomes. The common mistake to avoid is treating \(X\) and \(Y\) as independent and multiplying their marginal probabilities; the table specifies conditional probabilities, so the conditional method is required.

A Continuous Conditional Distribution

When \(X\) is continuous, a sum over its values becomes an integral. The integration limits must reflect where the conditional event is possible and where it is certain. A changing conditional support can make these limits especially important.

Worked Example: A Sum with a Support that Depends on \(X\)

Let \(X\) have the uniform distribution on \((0,1)\). Given \(X=x\), let \(Y\) have the uniform distribution on \((0,x)\). Find the CDF and density of \(S=X+Y\).

Method. Conditional on \(X=x\), \(S=x+Y\) is uniform on \((x,2x)\). Thus its conditional CDF is zero for \(s\leq x\), equals \((s-x)/x\) for \(x<s<2x\), and equals one for \(s\geq2x\). Since \(f_X(x)=1\) on \((0,1)\), integrate these conditional probabilities over \(x\).

If \(0<s<1\), the conditional probability is one for \(0<x\leq s/2\), is \((s-x)/x\) for \(s/2<x<s\), and is zero for \(x\geq s\). Therefore,

$$ \begin{aligned} F_S(s) &=\int_0^{s/2}1\,dx+\int_{s/2}^{s}\frac{s-x}{x}\,dx\\ &=\frac{s}{2}+\left[s\ln x-x\right]_{s/2}^{s}\\ &=\frac{s}{2}+\left(s\ln 2-\frac{s}{2}\right)\\ &=s\ln 2. \end{aligned} $$

If \(1\leq s<2\), every \(x\leq s/2\) makes the event certain, while \(s/2<x<1\) gives a partial conditional probability. Hence

$$ \begin{aligned} F_S(s) &=\int_0^{s/2}1\,dx+\int_{s/2}^{1}\frac{s-x}{x}\,dx\\ &=\frac{s}{2}+\left[s\ln x-x\right]_{s/2}^{1}\\ &=s\ln(2/s)+s-1. \end{aligned} $$

The support is \(0<S<2\), so the complete CDF is

$$ F_S(s)= \begin{cases} 0, & s\leq 0,\\ s\ln 2, & 0<s<1,\\ s\ln(2/s)+s-1, & 1\leq s<2,\\ 1, & s\geq 2. \end{cases} $$

Differentiating within the two continuous intervals gives

$$ f_S(s)= \begin{cases} \ln 2, & 0<s<1,\\ \ln(2/s), & 1<s<2,\\ 0, & \text{otherwise}. \end{cases} $$

Check. The CDF is continuous at \(s=1\), where both expressions give \(\ln 2\), and at \(s=2\), where it gives \(1\). The density integrates to \(\ln 2+\int_1^2\ln(2/s)\,ds=\ln 2+(1-\ln 2)=1\). The common mistake is to integrate the partial-probability expression over the whole range of \(x\), ignoring the regions where the event is already certain or impossible.

Using Conditioning to Check a Distribution

A conditional model can also give a quick mean check. It is important to use the conditional mean of \(Y\) at each value of \(X\); substituting the unconditional mean of \(Y\) inside every conditional case generally loses the dependence.

Worked Example: Conditional Binomial Counts

Suppose \(P(X=0)=0.40\) and \(P(X=2)=0.60\). Given \(X=0\), let \(Y\sim\operatorname{Binomial}(2,0.25)\); given \(X=2\), let \(Y\sim\operatorname{Binomial}(2,0.75)\). Find the probability mass function of \(S=X+Y\), then check its mean.

Method. For a binomial random variable with parameters \(2\) and \(p\), the probabilities at \(0,1,2\) are \((1-p)^2,\ 2p(1-p),\ p^2\). When \(X=0\), these are \(0.5625,\ 0.3750,\ 0.0625\), giving weighted contributions to \(S=0,1,2\) of \(0.2250,\ 0.1500,\ 0.0250\).

When \(X=2\), the conditional probabilities at \(Y=0,1,2\) are \(0.0625,\ 0.3750,\ 0.5625\). Multiplying by \(P(X=2)=0.60\) gives contributions to \(S=2,3,4\) of \(0.0375,\ 0.2250,\ 0.3375\).

Combining the contributions,

$$ \begin{aligned} P(S=0)&=0.2250, & P(S=1)&=0.1500,\\ P(S=2)&=0.0250+0.0375=0.0625, & P(S=3)&=0.2250,\\ P(S=4)&=0.3375. \end{aligned} $$

The probabilities sum to \(0.2250+0.1500+0.0625+0.2250+0.3375=1.0000\). Their implied mean is

$$ E[S]=0(0.2250)+1(0.1500)+2(0.0625)+3(0.2250)+4(0.3375)=2.3000. $$

Independently, the law of total expectation gives \(E[S]=E[X]+E[E[Y\mid X]]\). Here \(E[X]=0(0.40)+2(0.60)=1.20\), and \(E[Y\mid X=0]=2(0.25)=0.50\), while \(E[Y\mid X=2]=2(0.75)=1.50\). Thus \(E[E[Y\mid X]]=0.40(0.50)+0.60(1.50)=1.10\), so \(E[S]=1.20+1.10=2.30\), matching the mass function.

The common mistake is to use one fixed binomial parameter for both conditional cases. The model assigns \(p=0.25\) when \(X=0\) and \(p=0.75\) when \(X=2\); those conditional distributions determine the sum probabilities.

Common Errors and Practical Checks

Conditioning is a bookkeeping method as well as a probability identity. For every possible value of the conditioning variable, track the corresponding shifted event or support before summing or integrating. The following checks catch many errors:

  • Do not assume independence. Independence would allow \(P(Y\leq y\mid X=x)=P(Y\leq y)\). Without independence, use the conditional distribution actually specified.
  • Shift by the conditioned value. For \(S=X+Y\), the event is \(Y\leq s-x\), not \(Y\leq s\).
  • Respect the support. In a continuous calculation, separate regions where the conditional event is impossible, partial, or certain.
  • Check total mass and endpoints. A discrete probability mass function must sum to one. A CDF must be nondecreasing, stay between zero and one, and reach one beyond the support.
  • Use means only as checks. The law of total expectation can verify a mean, but a matching mean does not prove that the whole distribution is correct.
Key takeaway: Condition on one summand, express the sum event in terms of the other summand, and average using the distribution of the conditioning variable. This yields a valid sum distribution whether or not the summands are independent.

Check Your Understanding

For each problem, condition on the stated value of \(X\) and keep track of which outcomes produce the same sum.

  1. \(P(X=0)=0.30\) and \(P(X=1)=0.70\). Given \(X=0\), \(Y=0\) with probability \(0.80\) and \(Y=2\) with probability \(0.20\). Given \(X=1\), \(Y=0\) with probability \(0.25\) and \(Y=1\) with probability \(0.75\). Find the mass function of \(S=X+Y\). (Answer: \(P(S=0)=0.240,\ P(S=1)=0.175,\ P(S=2)=0.585\).)
  2. Let \(X\) be uniform on \((0,1)\), and, given \(X=x\), let \(Y\) be uniform on \((0,2x)\). Find \(P(X+Y\leq1)\). (Answer: \(\frac{\ln 3}{2}\approx0.5493\).)
  3. \(P(X=0)=0.40\) and \(P(X=2)=0.60\). Given \(X=0\), \(Y\) is Bernoulli with success probability \(0.20\); given \(X=2\), \(Y\) is Bernoulli with success probability \(0.70\). Find \(E[S]\) and \(P(S\geq3)\), where \(S=X+Y\). (Answer: \(E[S]=1.70\) and \(P(S\geq3)=0.42\).)
  4. \(P(X=1)=0.60\) and \(P(X=3)=0.40\). Given \(X=1\), \(Y=0\) or \(2\), each with probability \(0.50\). Given \(X=3\), \(P(Y=0\mid X=3)=0.25\) and \(P(Y=1\mid X=3)=0.75\). Find \(P(S\leq3)\) and \(E[S]\). (Answer: \(P(S\leq3)=0.70\) and \(E[S]=2.70\).)

Previous

Next

Back to Tutorials
Clinical Biostats