Conditional Probability and Bayes’ Rule for Events

Conditional Probability Starts with an Intersection

Knowing that an event \(B\) occurred changes the relevant sample space: outcomes outside \(B\) are no longer possible. To find the probability that \(A\) also occurred, count the part of \(B\) that lies in \(A\), then compare it with the probability of \(B\). This simple ratio is the basis for conditional probability, the multiplication rule, and Bayes’ rule.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), where \(\Omega\) is the set of outcomes, \(\mathcal{F}\) is a collection of events, and \(\mathbb{P}\) assigns probabilities to events. For events \(A,B\in\mathcal{F}\), the conditional probability of \(A\) given \(B\) is defined only when \(\mathbb{P}(B)>0\). If \(\mathbb{P}(B)=0\), the ratio in the definition is undefined, and this event-based definition does not assign a value to \(\mathbb{P}(A\mid B)\).

The denominator is the probability of the information being conditioned on; the numerator is the probability that this information and the event of interest both occur. In particular, \(\mathbb{P}(B\mid B)=1\), because \(\mathbb{P}(B\cap B)=\mathbb{P}(B)\). Also, \(\mathbb{P}(A\mid B)=0\) if \(A\cap B\) is empty. These facts depend on \(\mathbb{P}(B)>0\).

Proof. Nonnegativity follows because \(\mathbb{P}(A\cap B)\ge0\) and the denominator \(\mathbb{P}(B)\) is positive. For the whole sample space, \(\mathbb{P}(\Omega\mid B)=\mathbb{P}(\Omega\cap B)/\mathbb{P}(B)=\mathbb{P}(B)/\mathbb{P}(B)=1\). If \(A_1,A_2,\ldots\) are pairwise disjoint, then the events \(A_r\cap B\) are pairwise disjoint and their union is \((\bigcup_r A_r)\cap B\). Countable additivity of \(\mathbb{P}\), followed by division by the positive number \(\mathbb{P}(B)\), gives \(\mathbb{P}((\bigcup_r A_r)\mid B)=\sum_r\mathbb{P}(A_r\mid B)\). Finally, multiplying the defining equation by \(\mathbb{P}(B)\) gives the multiplication rule. \(\square\)

The multiplication rule can also be used in the other direction. If \(\mathbb{P}(A)>0\), then \(\mathbb{P}(A\cap B)=\mathbb{P}(A)\mathbb{P}(B\mid A)\). Equating these two expressions for the same intersection gives \(\mathbb{P}(B)\mathbb{P}(A\mid B)=\mathbb{P}(A)\mathbb{P}(B\mid A)\). The two conditional probabilities generally differ: their denominators refer to different events.

Partitions, Total Probability, and Bayes’ Rule

A finite partition \(H_1,\ldots,H_m\) of \(\Omega\) consists of pairwise disjoint events whose union is \(\Omega\). Think of the \(H_i\) as mutually exclusive possibilities that cover every outcome. For the formulas below, write \(I=\{i:\mathbb{P}(H_i)>0\}\), and assume \(\mathbb{P}(E)>0\) for the event \(E\) on which we condition. Conditional probabilities given \(H_i\) are used only for \(i\in I\). Any cell with zero probability has \(\mathbb{P}(E\cap H_i)=0\), so it contributes nothing to the sums.

Proof. Since the partition covers \(\Omega\), the events \(E\cap H_1,\ldots,E\cap H_m\) are disjoint and their union is \(E\). Additivity gives \(\mathbb{P}(E)=\sum_{i=1}^m\mathbb{P}(E\cap H_i)\). A zero-probability cell has \(\mathbb{P}(E\cap H_i)=0\); for every \(i\in I\), the multiplication rule gives \(\mathbb{P}(E\cap H_i)=\mathbb{P}(E\mid H_i)\mathbb{P}(H_i)\). This proves the total probability formula. For \(i\in I\), the definition of conditional probability gives \(\mathbb{P}(H_i\mid E)=\mathbb{P}(H_i\cap E)/\mathbb{P}(E)\). Substitute the multiplication-rule expression for the numerator and the total-probability expression for the denominator to obtain Bayes’ formula. If \(\mathbb{P}(H_i)=0\), then \(\mathbb{P}(H_i\cap E)=0\), and the same definition, with \(\mathbb{P}(E)>0\), gives \(\mathbb{P}(H_i\mid E)=0\). \(\square\)

Total probability combines the contributions from all possible cases. Bayes’ rule then reverses the direction of conditioning: it starts with the probability of observing \(E\) under each case \(H_i\) and returns the probabilities of those cases after \(E\) is observed. The denominator is essential because it normalizes the contributions to sum to \(1\).

Reading Conditional Probabilities from a Table

A finite table is a compact way to keep the intersection and conditioning event distinct. Each cell records a joint probability or a count for two events; row and column totals provide the marginal probabilities. To calculate \(\mathbb{P}(A\mid B)\), divide the cell for \(A\cap B\) by the total for \(B\), not by the total for \(A\). If the table contains counts from equally weighted observations, dividing the relevant counts gives the same conditional probability because the common total cancels.

A Table-Based Bayes Calculation

The same formulas work when a table contains counts rather than probabilities. Suppose a study records whether each person received a treatment and whether the person responded. Let \(T\) mean treated and \(R\) mean response. The four cells below are counts; the row and column totals are included so the conditioning denominator can be read directly.

Common Mistakes and a Reliable Check

A frequent error is to use the probability of the event being asked about as the denominator, regardless of the conditioning event. For example, in the die example, the incorrect calculation \(\mathbb{P}(A\mid B)=\mathbb{P}(A\cap B)/\mathbb{P}(A)=(3/36)/(5/36)=3/5\) actually calculates \(\mathbb{P}(B\mid A)\). The correction is to use \(\mathbb{P}(B)=18/36\) in the denominator for \(\mathbb{P}(A\mid B)\), giving \((3/36)/(18/36)=1/6\).

Before using a conditional probability or Bayes’ rule, check three items: the conditioning event has positive probability; the numerator is the intersection with that event; and the denominator is the probability of that event. For Bayes’ rule, also check that the cases form a partition and that the total-probability denominator is positive. These checks prevent undefined conditionals and help distinguish a likelihood such as \(\mathbb{P}(E\mid H_i)\) from a posterior probability such as \(\mathbb{P}(H_i\mid E)\).