Marginal PMFs by Summation

Why Summing a Joint PMF Gives a Marginal PMF

A joint PMF assigns probability to pairs of discrete outcomes. To find the probability of one coordinate taking a particular value, we must include every joint outcome with that coordinate value. The other coordinate may take several possible values, so its probabilities are added. On a nonrectangular support, the values to add can depend on the fixed coordinate; identifying those values correctly is the main practical challenge.

The central rule is simple: hold one coordinate fixed and sum the joint PMF over the other coordinate. The sum may be finite or infinite. It must include all positive-probability outcomes in that coordinate’s fiber, and it may also include zero-probability terms without changing the result.

Definition: Let $X$ and $Y$ be discrete random variables with joint PMF $p_{X,Y}(x,y)=P(X=x,Y=y)$, for $x$ in a countable set $A$ and $y$ in a countable set $B$. Their marginal PMFs are $$ p_X(x)=P(X=x)=\sum_{y\in B}p_{X,Y}(x,y),\qquad p_Y(y)=P(Y=y)=\sum_{x\in A}p_{X,Y}(x,y). $$ The positive-probability support is $S=\{(x,y)\in A\times B:p_{X,Y}(x,y)>0\}$. For fixed $x$, its support fiber is $S_x=\{y\in B:(x,y)\in S\}$; for fixed $y$, it is $S^y=\{x\in A:(x,y)\in S\}$. Thus the same marginal sums can be written as $p_X(x)=\sum_{y\in S_x}p_{X,Y}(x,y)$ and $p_Y(y)=\sum_{x\in S^y}p_{X,Y}(x,y)$.

A model may specify a larger set of constraint-compatible pairs than its positive-probability support. A compatible pair can still have zero joint probability. Summing over all compatible values is valid if those zero terms are included, but the compatible values should not be confused with a support fiber. In particular, a zero-mass pair does not belong to $S$.

Marginalization on a Finite Triangular Support

Worked Example: A Constraint With a Zero-Mass Pair

Suppose $X,Y\in\{0,1,2\}$, with constraint-compatible pairs satisfying $x\leq y$. Assign probability $1/5$ to each of $(0,0),(0,1),(1,1),(1,2),(2,2)$, and probability zero to every other pair. The compatible set has six pairs, including $(0,2)$, but the positive-probability support has only the five listed pairs. The PMF is normalized because its five positive masses sum to $5(1/5)=1$.

For $p_X(0)$, the constraint permits $y=0,1,2$, but the support fiber is $S_0=\{0,1\}$: $(0,2)$ is compatible but has zero mass. Therefore

p_X(0)=p_{X,Y}(0,0)+p_{X,Y}(0,1)=\frac15+\frac15=\frac25.

At $x=1$, the support fiber is $S_1=\{1,2\}$, and at $x=2$, it is $S_2=\{2\}$. Hence

p_X(1)=p_{X,Y}(1,1)+p_{X,Y}(1,2)=\frac25,\qquad p_X(2)=p_{X,Y}(2,2)=\frac15.

For the other marginal, the support fibers are $S^0=\{0\}$, $S^1=\{0,1\}$, and $S^2=\{1,2\}$. The constraint permits $x=0,1,2$ when $y=2$, but $x=0$ is not in $S^2$ because $p_{X,Y}(0,2)=0$. Thus

p_Y(0)=\frac15,\qquad p_Y(1)=p_{X,Y}(0,1)+p_{X,Y}(1,1)=\frac25,\qquad p_Y(2)=p_{X,Y}(1,2)+p_{X,Y}(2,2)=\frac25.

Each marginal sums to one: $2/5+2/5+1/5=1$. Notice that including the compatible but zero-mass pair in either sum would leave the numerical answer unchanged, provided its zero probability is explicitly used.

Marginals on a Lattice With Changing Bounds

Worked Example: A Uniform Triangular Lattice

Let $X,Y$ take values in the triangular lattice $C=\{(x,y):x,y\in\{0,1,2,3\},\ 0\leq y\leq x\leq3\}$. Give each compatible pair probability $1/10$, and all other pairs probability zero. There are $1+2+3+4=10$ pairs, so the joint PMF is normalized.

For a fixed $x$, the compatible values are $y=0,\ldots,x$, and all of these pairs have positive mass. Thus

p_X(x)=\sum_{y=0}^{x}\frac1{10}=\frac{x+1}{10},\qquad x=0,1,2,3.

The marginal values are $1/10,2/10,3/10,4/10$, which sum to $10/10=1$. For fixed $y$, the constraint instead requires $x=y,\ldots,3$, so

p_Y(y)=\sum_{x=y}^{3}\frac1{10}=\frac{4-y}{10},\qquad y=0,1,2,3.

These marginal values are $4/10,3/10,2/10,1/10$, also summing to one. The bounds depend on which coordinate is fixed: summing for $X$ uses $y\leq x$, whereas summing for $Y$ uses $x\geq y$. Reusing the first set of bounds for the second marginal would count the wrong pairs.

Infinite Sums and a General Result

With countably infinite values, the same rule applies, but the marginal may involve an infinite series. Since PMF terms are nonnegative, sums can be grouped by one coordinate without cancellation or conditional-convergence issues. The following result records why the formulas produce valid PMFs.

Theorem: If $p_{X,Y}$ is a joint PMF on a countable product $A\times B$, then the functions $p_X(x)=\sum_{y\in B}p_{X,Y}(x,y)$ and $p_Y(y)=\sum_{x\in A}p_{X,Y}(x,y)$ are PMFs. For every subset $D\subseteq A$, $P(X\in D)=\sum_{x\in D}p_X(x)$, and the corresponding statement holds for subsets of $B$.

Proof. Each marginal term is nonnegative because it is a sum of nonnegative joint probabilities. The joint PMF has total mass one. Grouping its nonnegative terms first by $x$ gives

\sum_{x\in A}p_X(x) =\sum_{x\in A}\sum_{y\in B}p_{X,Y}(x,y) =\sum_{(x,y)\in A\times B}p_{X,Y}(x,y) =1.

The regrouping is valid for a countable sum of nonnegative terms: its value is the supremum of the sums over finite subsets, so grouping the terms in a different order does not change that supremum. Grouping first by $y$ proves $\sum_{y\in B}p_Y(y)=1$ by the same argument. Finally, the event $\{X\in D\}$ is the disjoint union of the events $\{X=x,Y=y\}$ for $x\in D$ and $y\in B$. Countable additivity and regrouping nonnegative terms give

P(X\in D)=\sum_{x\in D}\sum_{y\in B}p_{X,Y}(x,y)=\sum_{x\in D}p_X(x).

The proof for $Y$ uses the disjoint union over $y$ and $x$. $\square$

Result: The support of the marginal PMF of $X$ is the projection of the joint support onto the $x$-coordinate: $p_X(x)>0$ if and only if $S_x$ is nonempty. Similarly, $p_Y(y)>0$ if and only if $S^y$ is nonempty.

Proof. If $S_x$ is empty, every term $p_{X,Y}(x,y)$ is zero, so $p_X(x)=0$. If $S_x$ is nonempty, it contains some $y_0$ with $p_{X,Y}(x,y_0)>0$. Since all terms in the marginal sum are nonnegative, $p_X(x)\geq p_{X,Y}(x,y_0)>0$. The argument for $Y$ is identical. $\square$

Worked Example: An Infinite Triangular Support

Fix $q$ with $0<q<1$, and define

p_{X,Y}(i,j)=(1-q)^2q^i,\qquad i,j\in\{0,1,2,\ldots\},\quad 0\leq j\leq i,

with mass zero outside this triangular support. For each $i$, there are $i+1$ possible values of $j$. To verify normalization, first note

\sum_{i=0}^{\infty}(i+1)q^i =\sum_{a=0}^{\infty}\sum_{b=0}^{\infty}q^{a+b} =\left(\sum_{a=0}^{\infty}q^a\right)^2 =\frac{1}{(1-q)^2}.

The first equality groups the nonnegative terms by $i=a+b$; exactly $i+1$ pairs of nonnegative integers have sum $i$. Therefore the total joint mass is $(1-q)^2/(1-q)^2=1$.

For fixed $i$, the support fiber is $j=0,\ldots,i$. Hence

p_X(i)=\sum_{j=0}^{i}(1-q)^2q^i=(i+1)(1-q)^2q^i.

For fixed $j$, the support condition requires $i=j,j+1,\ldots$. Therefore

p_Y(j)=\sum_{i=j}^{\infty}(1-q)^2q^i =(1-q)^2\frac{q^j}{1-q} =(1-q)q^j.

For the concrete value $q=1/2$, $p_X(0)=1/4$, $p_X(1)=1/4$, and $p_X(2)=3/16$; the general formula gives $p_X(i)=(i+1)/2^{i+2}$. Also $p_Y(0)=1/2$, $p_Y(1)=1/4$, and $p_Y(2)=1/8$, following $p_Y(j)=1/2^{j+1}$. The sums are one: $\sum_{i\geq0}(i+1)/2^{i+2}=1$ by the normalization series above, and $\sum_{j\geq0}1/2^{j+1}=1$ by the geometric series.

Common Mistakes and Why Marginalization Matters

A frequent error is to treat a support constraint as if every compatible pair necessarily has positive mass. The correction is to distinguish the constraint-compatible set from $S$, the positive-probability support. When writing a marginal as a sum over the support fiber, include only positive-mass pairs. If instead summing over a larger compatible set, include zero terms wherever the joint PMF assigns zero. Either calculation gives the same value only when those terms are correctly assigned zero.

Another error is to use identical summation limits for both marginals on a nonrectangular support. In a triangle such as $0\leq y\leq x\leq3$, fixing $x$ gives $0\leq y\leq x$, while fixing $y$ gives $y\leq x\leq3$. The fixed coordinate determines which slice of the support is being summed.

Marginalization is essential because it extracts the distribution of one coordinate from a joint model without losing probability accounting. It also gives a direct check on calculations: each resulting marginal must be nonnegative and sum to one, and its positive values must occur exactly at coordinates reached by the joint support. Those checks often reveal a missing endpoint or an incorrectly included support value.

Check Your Understanding

For each question, identify the values that contribute to the marginal before evaluating the sum.

If the positive-probability support is $\{(0,0),(0,1),(1,1)\}$, what are the support fibers for each $x$-value and each $y$-value?
A joint PMF has support $0\leq y\leq x\leq4$. For fixed $x=3$, which $y$-values are summed to obtain $p_X(3)$? For fixed $y=2$, which $x$-values are summed to obtain $p_Y(2)$?
Why does a constraint-compatible pair with zero joint probability not belong to a support fiber?
For the infinite triangular example, explain why the sum for $p_Y(j)$ begins at $i=j$, not $i=0$.
State two checks that can be applied to a computed marginal PMF.

What You'll Learn