Joint PMFs on Countable Supports

From Point Masses to a Joint PMF

For a random pair \((X,Y)\), each possible outcome is a point in the plane. When the pair takes values in a finite or countable set, its entire distribution can be represented by assigning a probability to each point it can occupy. This representation is the joint probability mass function (joint PMF). Once the point masses are specified, the probability of an event is obtained by adding the masses of the points that satisfy it.

The central checks are simple but essential: every assigned mass must be nonnegative, and the masses over the support must sum to one. The support need not be a rectangular grid. It may be finite, infinite, or restricted to a curve or another countable subset of the plane.

A support is the set of points that may receive positive probability, not necessarily the full product of the possible \(X\)-values and \(Y\)-values. For example, if \(Y=2X\) with probability one and \(X\) is integer-valued, the pair lies on the countable set \(\{(k,2k): k\in\mathbb{Z}\}\), not on every pair of integers. It is usually clearest to state the support explicitly before giving a formula for \(p\).

Checking and Using a Joint PMF

For a proposed joint PMF, first identify every point at which the formula is nonzero. Then verify nonnegativity at those points and sum the masses over that support. If the total is one, the function assigns all probability to the possible outcomes. For a set \(A\) of outcomes, the event \(\{(X,Y)\in A\}\) occurs at exactly those support points that lie in \(A\).

Proof. For each \((x,y)\in S\), let \(E_{x,y}=\{X=x,\;Y=y\}\). These events are pairwise disjoint: two different ordered pairs cannot both be the realized value of \((X,Y)\). Their union over \(S\cap A\) is the event that \((X,Y)\in A\cap S\). Since \(P((X,Y)\in S)=1\), the events \(\{(X,Y)\in A\}\) and \(\{(X,Y)\in A\cap S\}\) differ only on an event of probability zero. Countable additivity therefore gives $$ P((X,Y)\in A) =P((X,Y)\in A\cap S) =\sum_{(x,y)\in S\cap A}P(E_{x,y}) =\sum_{(x,y)\in S\cap A}p(x,y). $$ The equality also holds when \(S\) is finite, as a finite sum is a special case of a countable sum. \(\square\)

The same calculation shows why the total mass must be one: take \(A=\mathbb{R}^2\). It also shows that a point’s mass is included once if the point is in the event and not included otherwise. For a countably infinite support, the sum is an infinite series of nonnegative terms. Such a sum is defined as the limit of its finite partial sums; its value does not depend on how the support points are enumerated.

Countably Infinite Supports

With infinitely many support points, the definition and event rule are unchanged; the finite sum is replaced by a series. A useful way to check a proposed series is to simplify it using a known convergent geometric series, while keeping track of all indices and factors.

A countable support can also be much smaller than the lattice on which its coordinates are indexed. In that case, it is important to sum only over points that actually belong to the support. The next example illustrates this directly.

Why the Conditions Are Exactly the Right Ones

Nonnegativity and unit total mass are not merely convenient checks. For a countable set, they are precisely the conditions needed to build a probability law from assigned point masses. This converse confirms that a normalized table or series can stand on its own as a distribution.

Proof. Use \(S\) as the sample space and take its power set as the collection of events. For each subset \(B\subseteq S\), define $$ P(B)=\sum_{s\in B}p(s). $$ The empty set has probability zero, every \(P(B)\) is nonnegative, and \(P(S)=1\) by the assumed normalization. If \(B_1,B_2,\ldots\) are pairwise disjoint subsets of \(S\), their points are divided among those subsets without repetition. Because all terms \(p(s)\) are nonnegative, the sum over their union equals the sum of the separate sums: $$ P\left(\bigcup_{m=1}^{\infty}B_m\right) =\sum_{s\in\cup_m B_m}p(s) =\sum_{m=1}^{\infty}\sum_{s\in B_m}p(s) =\sum_{m=1}^{\infty}P(B_m). $$ Thus \(P\) is a probability measure. Define \(X(s)\) and \(Y(s)\) to be the first and second coordinates of \(s\). At a point \(s=(x,y)\in S\), the event \(\{X=x,Y=y\}\) contains exactly that sample point, so its probability is \(p(s)\). The resulting random vector has the stated joint PMF. \(\square\)

Common Mistake: Summing Outside the Support

A frequent error is to treat a joint PMF as if every combination of coordinate values were possible. In the curve example, one might sum \(p(i,j)\) over all \(i,j\geq 0\) when calculating \(P(X\geq 2)\), even though the formula was defined only at \((k,2k)\). That includes points that cannot occur and can lead to an incorrect total.

The correction is to write the event first and then intersect it with the support: $$ P((X,Y)\in A)=\sum_{(x,y)\in S\cap A}p(x,y). $$ This one rule handles inequalities, equalities, unions of points, and any other event involving the pair. For example, a boundary point is included exactly when it satisfies the event as stated; for \(X\geq 2\), the point with \(X=2\) is included, while for \(X>2\), it is not.