Product Laws for Independent Discrete Variables

Why Independence Gives a Product Law

When two discrete random variables are independent, knowing the value of one does not change the probability distribution of the other. This condition gives a direct way to construct their joint probability mass function: multiply the two marginal mass functions. The resulting rule applies both to a single specified pair and, by summing, to regions containing many pairs.

The product rule is simple, but it depends on independence. Marginal distributions alone do not generally determine a joint distribution. The distinction matters whenever a probability concerns two variables together—for example, the probability that a pair of outcomes falls in a specified set.

Definition: A discrete random variable \(X\) has probability mass function \(p_X(x)=P(X=x)\). Its support is \(S_X=\{x:p_X(x)>0\}\). For discrete random variables \(X\) and \(Y\), their joint probability mass function is \(p_{X,Y}(x,y)=P(X=x,Y=y)\). They are independent if \(P(X\in A,Y\in B)=P(X\in A)P(Y\in B)\) for every pair of sets \(A\) and \(B\) of possible values.

The Product-Law Theorem

For a pair of independent discrete random variables, the joint pmf is determined at every pair of values by the marginal pmfs. Conversely, a joint pmf that factors in this way describes independent variables.

Theorem: Let \(X\) and \(Y\) be discrete random variables with joint pmf \(p_{X,Y}\) and marginal pmfs \(p_X\) and \(p_Y\). Then \(X\) and \(Y\) are independent if and only if

p_{X,Y}(x,y)=p_X(x)p_Y(y)

for every pair \(x,y\). In particular, if \(X\) and \(Y\) are independent, their joint pmf is zero outside \(S_X\times S_Y\), and on that support it is the product of the marginal pmfs.

To prove the forward direction, take \(A=\{x\}\) and \(B=\{y\}\) in the definition of independence. Then \(P(X=x,Y=y)=P(X=x)P(Y=y)\), which is the claimed factorization.

For the converse, suppose the factorization holds for every pair. For any sets \(A\) and \(B\) of possible values, sum the joint pmf over all pairs in \(A\times B\):

\begin{aligned} P(X\in A,Y\in B) &=\sum_{x\in A}\sum_{y\in B}p_{X,Y}(x,y)\\ &=\sum_{x\in A}\sum_{y\in B}p_X(x)p_Y(y)\\ &=\left(\sum_{x\in A}p_X(x)\right) \left(\sum_{y\in B}p_Y(y)\right)\\ &=P(X\in A)P(Y\in B). \end{aligned}

The sums have nonnegative terms, so the factorization of these countable sums is valid, including when either set is infinite. This is exactly the definition of independence. Also, the product rule gives a normalized joint pmf: summing it over the whole product support gives \((\sum_x p_X(x))(\sum_y p_Y(y))=1\). Thus the product formula both constructs a valid joint law and characterizes independence. \(\square\)

Calculating Probabilities of Pairs and Regions

A single pair \((x,y)\) has probability \(p_X(x)p_Y(y)\) when the variables are independent. A region is any set \(C\) of pairs in the joint support. Its probability is obtained by adding the joint masses of all pairs in \(C\). For a finite region this is a finite sum; for a countably infinite region it is a nonnegative series.

Region formula: If \(X\) and \(Y\) are independent and \(C\) is a set of pairs, then

P\bigl((X,Y)\in C\bigr) =\sum_{(x,y)\in C}p_X(x)p_Y(y).

The sum is taken over the pairs in \(C\) that have positive joint mass.

This formula follows by expressing the event \(\{(X,Y)\in C\}\) as the disjoint union of the events \(\{X=x,Y=y\}\) over \((x,y)\in C\), then applying countable additivity and the product-law theorem. In some regions, summing first across one coordinate makes the calculation shorter. The allowed order of summation does not change the result because all terms are nonnegative.

Worked Example: A Joint Table for Two Bernoulli Variables

Let \(X\sim\operatorname{Bernoulli}(0.3)\) and \(Y\sim\operatorname{Bernoulli}(0.6)\), independently. Thus \(P(X=1)=0.3\), \(P(X=0)=0.7\), \(P(Y=1)=0.6\), and \(P(Y=0)=0.4\). Construct the joint pmf and find \(P(X\geq Y)\).

By the product law, each joint mass is the product of the two corresponding marginal masses:

\(x\)	\(y\)	\(p_{X,Y}(x,y)\)
0	0	\(0.7(0.4)=0.28\)
0	1	\(0.7(0.6)=0.42\)
1	0	\(0.3(0.4)=0.12\)
1	1	\(0.3(0.6)=0.18\)

The four masses sum to \(0.28+0.42+0.12+0.18=1\). The event \(X\geq Y\) consists of the pairs \((0,0),(1,0),(1,1)\). Therefore,

P(X\geq Y)=0.28+0.12+0.18=0.58.

Equivalently, when \(Y=0\), either value of \(X\) qualifies, while when \(Y=1\), only \(X=1\) qualifies. This gives \(0.4(0.7+0.3)+0.6(0.3)=0.4+0.18=0.58\), a check on the region sum.

Worked Example: A Region in a Finite Joint Support

Let \(X\) and \(Y\) be independent with \(P(X=0)=0.2\), \(P(X=1)=0.5\), \(P(X=2)=0.3\), and \(P(Y=a)=0.25\), \(P(Y=b)=0.75\). Find the probability of the region \(X+\mathbf{1}_{\{Y=b\}}\geq 2\), where \(\mathbf{1}_{\{Y=b\}}\) is 1 if \(Y=b\) and 0 otherwise.

For \(Y=a\), the indicator is 0, so the condition requires \(X\geq2\). The only qualifying pair in this case is \((2,a)\), with mass \(0.3(0.25)=0.075\).

For \(Y=b\), the indicator is 1, so the condition requires \(X\geq1\). The qualifying pairs are \((1,b)\) and \((2,b)\), with masses \(0.5(0.75)=0.375\) and \(0.3(0.75)=0.225\). Adding the masses of the disjoint qualifying pairs gives

P\bigl(X+\mathbf{1}_{\{Y=b\}}\geq2\bigr) =0.075+0.375+0.225=0.675.

As a check, the three masses outside the region are \(P(0,a)=0.05\), \(P(0,b)=0.15\), and \(P(1,a)=0.125\). They sum to \(0.325\), and \(0.675+0.325=1\).

Worked Example: A Probability Over an Infinite Region

Let \(X\) and \(Y\) be independent geometric random variables on \(\{1,2,\ldots\}\), with \(P(X=k)=0.5(0.5)^{k-1}\) and \(P(Y=j)=0.25(0.75)^{j-1}\). Find \(P(X\leq2,Y\geq3)\), and calculate the joint mass at \((2,3)\).

The event is a rectangle in the joint support: \(X\) can be 1 or 2, while \(Y\) can be any integer at least 3. The product law gives

\begin{aligned} P(X\leq2,Y\geq3) &=\left[P(X=1)+P(X=2)\right]\sum_{j=3}^{\infty}0.25(0.75)^{j-1}\\ &=(0.5+0.25)\left[0.25(0.75)^2\sum_{r=0}^{\infty}(0.75)^r\right]\\ &=0.75\left[0.25(0.75)^2\frac{1}{1-0.75}\right]\\ &=0.75(0.5625)=0.421875. \end{aligned}

The geometric series converges because \(0.75<1\). For the specified pair, \(P(X=2)=0.5(0.5)=0.25\), and \(P(Y=3)=0.25(0.75)^2=0.140625\). Hence

p_{X,Y}(2,3)=0.25(0.140625)=0.03515625.

The pair belongs to the region, and its mass is one of the nonnegative terms included in the region probability.

Common Mistake: Multiplying Marginal Masses Without Independence

The product formula is not a way to infer a joint law from two marginal pmfs in every situation. Its hypothesis is independence. For example, let \(U\) be Bernoulli with success probability \(0.5\), and set \(X=U\) and \(Y=U\). Both marginals assign probability \(0.5\) to 0 and probability \(0.5\) to 1, but \(X\) and \(Y\) always agree. In particular, \(P(X=0,Y=0)=0.5\), not \(P(X=0)P(Y=0)=0.25\). The variables are dependent, so their joint pmf does not factor as a product of their marginal pmfs.

A reliable calculation therefore has two stages: first establish that the variables are independent (or are explicitly specified to be independent), then multiply marginal masses for each pair in the event. For a region, include every qualifying pair and sum its joint mass exactly once.

Key takeaway: For independent discrete \(X\) and \(Y\), the joint mass at \((x,y)\) is \(p_X(x)p_Y(y)\). The probability of any specified region is the sum of these products over its pairs. Without independence, the product rule need not hold.

Check Your Understanding

Use the product-law criterion and show the pairs included in each region before summing.

If independent \(X\) and \(Y\) satisfy \(P(X=0)=0.4\) and \(P(Y=1)=0.7\), what is \(P(X=0,Y=1)\)?
For the Bernoulli variables in the first worked example, which pairs make up the event \(X<Y\), and what is its probability?
Explain why a joint pmf that factors as \(p_X(x)p_Y(y)\) for every pair implies independence.
For the geometric variables in the third worked example, calculate \(P(X=1,Y=1)\).
Can two variables with the same marginal pmfs have different joint pmfs? Give a reason for your answer.

Product Laws for Independent Discrete Variables

What You'll Learn