Introduction
Discrete conditioning problems often look different on the surface: one gives a joint pmf table, another describes evidence about an unobserved class, and a third asks for an average within groups. The same organizing principle handles them all: restrict the joint law to the information observed, then divide by the probability of that information. Marginal probabilities provide essential checks on the calculation.
This workshop develops that method through finite examples. The emphasis is on keeping track of which variable is being conditioned on, using only conditioning events with positive probability, and verifying each answer against the relevant marginal distribution.
Reading a Joint pmf Table
Suppose \(X\) and \(Y\) are discrete random variables with joint pmf \(p_{X,Y}(x,y)=P(X=x,Y=y)\). Each cell of a joint table gives the probability of one pair of values. Adding across a row or column gives the corresponding marginal probability. For example, \(P(X=x)=\sum_y p_{X,Y}(x,y)\), and \(P(Y=y)=\sum_x p_{X,Y}(x,y)\).
The denominator is not optional: it makes the probabilities in the restricted row or column sum to one. If the conditioning value has probability zero, this ratio is undefined, so the table does not specify a conditional distribution there.
Worked Example: Conditioning in a Joint pmf Table
Suppose \(X\) takes values \(0,1,2\), and \(Y\) takes values \(A,B\), with joint pmf given below. The entries are probabilities, not counts.
| \(X\) | \(Y=A\) | \(Y=B\) | \(P(X=x)\) |
|---|---|---|---|
| 0 | \(2/20\) | \(3/20\) | \(5/20\) |
| 1 | \(4/20\) | \(2/20\) | \(6/20\) |
| 2 | \(1/20\) | \(8/20\) | \(9/20\) |
| \(P(Y=y)\) | \(7/20\) | \(13/20\) | 1 |
First check the marginals. The row sums are \(5/20,6/20,9/20\), and the column sums are \(7/20,13/20\). Both sets sum to \(20/20=1\), as required.
To find the conditional distribution of \(X\) given \(Y=B\), divide the \(Y=B\) entries by \(P(Y=B)=13/20\):
These conditional probabilities sum to \((3+2+8)/13=1\). As an event restriction, the same calculation gives \(P(X\geq 1\mid Y=B)=(2+8)/13=10/13\). Check this directly using the joint event probability: \(P(X\geq 1,Y=B)=(2+8)/20=10/20\), so dividing by \(P(Y=B)=13/20\) again gives \(10/13\).
Reversing the conditioning illustrates why the denominator must change. In the row \(X=1\), the joint probabilities are \(4/20\) and \(2/20\), and the row marginal is \(6/20\). Thus \(P(Y=A\mid X=1)=(4/20)/(6/20)=2/3\), while \(P(Y=B\mid X=1)=1/3\). The two conditional probabilities sum to one.
Restricting the Joint Law to an Event
Sometimes the information is an event rather than a single value. For a set \(C\) of possible values of \(Y\), conditioning on \(Y\in C\) retains only the columns indexed by \(C\). For a set \(D\) of values of \(X\), the conditional probability of \(X\in D\) is obtained by summing the retained joint probabilities and dividing by \(P(Y\in C)\), provided this denominator is positive.
Proof. By the definition of conditional probability, \(P(X\in D\mid Y\in C)=P(X\in D,Y\in C)/P(Y\in C)\). The events \(\{X=x,Y=y\}\) are disjoint as \((x,y)\) varies. Therefore their probabilities add, giving \(P(X\in D,Y\in C)=\sum_{x\in D,\,y\in C}p_{X,Y}(x,y)\). For the conditional pmf, sum its values over all possible \(x\). The numerator then sums over every \(x\) and every \(y\in C\), which is exactly \(P(Y\in C)\). Dividing by that positive quantity gives 1. Each term is nonnegative, so these values satisfy the requirements for a pmf. \(\square\)
Worked Example: Restricting to a Range of Values
Use the joint pmf table above to find the conditional distribution of \(X\) given \(Y\in\{A,B\}\), and then find \(P(X=2\mid Y\in\{A,B\})\).
Here \(Y\) has only the values \(A\) and \(B\), so the conditioning event is the entire sample space: \(P(Y\in\{A,B\})=7/20+13/20=1\). No probabilities are removed or rescaled. Consequently the conditional distribution is just the marginal distribution of \(X\): \(5/20,6/20,9/20\) at \(0,1,2\), respectively. In particular, \(P(X=2\mid Y\in\{A,B\})=9/20\).
A less trivial restriction is \(X\geq 1\) given \(Y=B\). The event probability in the denominator is \(P(Y=B)=13/20\); the joint probability in the numerator is \(P(X\geq 1,Y=B)=10/20\). Hence the conditional probability is \(10/13\), agreeing with the sum of the two restricted conditional pmf entries from the earlier calculation.
Bayes Updates as Table Calculations
In a finite-class problem, let \(C\) denote a class and \(T\) the observed evidence. Bayes' rule calculates the conditional class probabilities after observing \(T=t\): \(P(C=c\mid T=t)=P(C=c,T=t)/P(T=t)\), when \(P(T=t)>0\). A joint table makes both the evidence probability and the posterior probabilities visible. The posterior entries must sum to one, and summing the joint table over classes must recover the marginal probability of the evidence.
Worked Example: Updating a Class Probability After a Positive Test
Suppose a population has two classes, \(A\) and \(B\), with probabilities \(0.10\) and \(0.90\). A test is positive with probability \(0.80\) in class \(A\) and \(0.20\) in class \(B\). Find the class probabilities after a positive result.
Build the joint probabilities by multiplying each class probability by the relevant conditional test probability. For example, \(P(A,\text{positive})=0.10(0.80)=0.08\). The complete table is:
| Class | Positive | Negative | Class marginal |
|---|---|---|---|
| \(A\) | 0.08 | 0.02 | 0.10 |
| \(B\) | 0.18 | 0.72 | 0.90 |
| Test-result marginal | 0.26 | 0.74 | 1 |
The positive-result marginal is \(0.08+0.18=0.26\). Thus \(P(A\mid\text{positive})=0.08/0.26=4/13\), and \(P(B\mid\text{positive})=0.18/0.26=9/13\). These posterior probabilities sum to \((4+9)/13=1\).
As an additional check, the negative-result posterior for class \(A\) is \(0.02/0.74=1/37\), and for class \(B\) it is \(0.72/0.74=36/37\). The total probability of class \(A\) is recovered by averaging its two conditional probabilities over the test-result marginal: \(0.26(4/13)+0.74(1/37)=0.08+0.02=0.10\). This matches the original class probability.
Iterated Conditioning
Conditioning can also be used in stages. If \(Y\) divides the sample space into finitely many positive-probability groups, a probability or expectation can be calculated within each group and then averaged using the probabilities of those groups. This provides another check: the result must agree with the corresponding marginal calculation.
Proof. For each \(y\) with positive probability, \(E[X\mid Y=y]=\sum_x x\,p_{X,Y}(x,y)/P(Y=y)\). Multiplying by \(P(Y=y)\) and summing over \(y\) gives
All sums are finite, so rearranging their order requires no limiting argument. For the event identity, replace the summand \(x\,p_{X,Y}(x,y)\) by \(1_{\{x\in D\}}p_{X,Y}(x,y)\). Summing over \(x\) and \(y\) gives \(P(X\in D)\), while the conditional sums give \(\sum_yP(X\in D\mid Y=y)P(Y=y)\). Values \(y\) with zero probability contribute zero to the marginal and are omitted. \(\square\)
Worked Example: Checking an Iterated Expectation Against the Marginal
Let \(P(Y=A)=1/3\) and \(P(Y=B)=2/3\). Given \(Y=A\), let \(X\) equal 0 with probability \(3/4\) and 2 with probability \(1/4\). Given \(Y=B\), let \(X\) equal 0, 1, or 2 with probabilities \(1/4,1/2,1/4\), respectively. Find \(E[X]\) by conditioning on \(Y\), then verify it from the marginal pmf of \(X\).
Within group \(A\), \(E[X\mid Y=A]=0(3/4)+2(1/4)=1/2\). Within group \(B\), \(E[X\mid Y=B]=0(1/4)+1(1/2)+2(1/4)=1\). Therefore iterated conditioning gives
Now form the joint probabilities by multiplying each conditional probability by its group probability. For \(Y=A\), the joint probabilities at \(X=0,1,2\) are \(1/4,0,1/12\). For \(Y=B\), they are \(1/6,1/3,1/6\). Adding by \(X\) gives the marginal pmf
The marginal probabilities sum to \(5/12+4/12+3/12=1\). Directly, \(E[X]=0(5/12)+1(1/3)+2(1/4)=1/3+1/2=5/6\), agreeing with the conditional calculation. As a probability check, \(P(X\geq1\mid Y=A)=1/4\) and \(P(X\geq1\mid Y=B)=3/4\). Averaging gives \((1/3)(1/4)+(2/3)(3/4)=1/12+1/2=7/12\). The marginal gives \(P(X\geq1)=1/3+1/4=7/12\), as it must.
Common Mistakes and a Reliable Check
The most common error is using the wrong denominator. For \(P(X=x\mid Y=y)\), the denominator is \(P(Y=y)\), not \(P(X=x)\). Switching the conditioning direction changes both the relevant row or column and its normalizing marginal. Another frequent error is to apply Bayes' rule to a likelihood alone: the joint probability requires multiplying the likelihood by the prior class probability before normalizing.
A practical check is to calculate the marginal before normalizing, then confirm that the conditional probabilities sum to one. For a Bayes update, check both the posterior sum and the evidence marginal. For iterated conditioning, calculate the requested quantity a second way from the marginal distribution whenever the table makes that possible. A mismatch usually signals an omitted cell, a wrong denominator, or a failure to distinguish a joint probability from a conditional one.
Check Your Understanding
For each problem, identify the conditioning event and use the relevant marginal as a check.
- In the first joint pmf table, calculate \(P(X=0\mid Y=A)\) and verify that the conditional probabilities for \(X=0,1,2\) given \(Y=A\) sum to one.
- Using that table, calculate \(P(X=0\text{ or }X=2\mid Y=B)\) by both summing conditional probabilities and dividing a joint event probability by \(P(Y=B)\).
- In the test example, calculate \(P(A\mid\text{negative})\). Why is it incorrect to divide \(P(A,\text{negative})\) by \(P(A)\)?
- In the iterated-conditioning example, calculate \(P(X=1)\) from the two conditional distributions and compare it with the marginal table calculation.
- What happens to the ratio defining \(P(X=x\mid Y=y)\) if \(P(Y=y)=0\), and why can the joint pmf table not supply that conditional distribution by division?