Conditional Expectation · Tutorial 233 of 946

Indicators and Conditional Probabilities

Interpret conditional probabilities as random variables measurable with respect to available information, and use their integral identities to characterize and compute them.

Intermediate 10 min read

What You'll Learn

Define conditional probability of an event given a sigma-algebra
Explain why conditional probability is a measurable random variable
Prove its almost-sure bounds between zero and one
Use the integral identity on every event in the conditioning sigma-algebra
Compute conditional probabilities in finite and continuous examples
Distinguish conditioning on a sigma-algebra from conditioning on a single event

Conditional probability as a random variable

When we condition on a single event of positive probability, conditional probability is a number. When we condition on a sigma-algebra, the information available can vary across outcomes. The resulting conditional probability is therefore generally a random variable: it assigns a probability value according to the information revealed by that sigma-algebra.

The connection is made through indicator functions. The indicator of an event records whether that event occurred, taking value one when it did and zero otherwise. Its conditional expectation records the conditional probability of the event.

Definition: Let $(\Omega,\mathcal F,\mathbb P)$ be a probability space, let $A\in\mathcal F$, and let $\mathcal G\subseteq\mathcal F$ be a sigma-algebra. The conditional probability of $A$ given $\mathcal G$ is the conditional expectation of its indicator: $$ \mathbb P(A\mid\mathcal G)=\mathbb E[\mathbf 1_A\mid\mathcal G]. $$ Precisely, it is any integrable, $\mathcal G$-measurable random variable $Q$ such that, for every $B\in\mathcal G$, $$ \int_B Q\,d\mathbb P=\mathbb P(A\cap B). $$ Any two such versions agree almost surely.

Here $\mathbf 1_A$ is the function equal to one on $A$ and zero on $A^c$. The defining integral identity is the key: for each event $B$ that can be described using the available information $\mathcal G$, the integral of the conditional probability over $B$ equals the probability that both $A$ and $B$ occur. In particular, the conditional probability is not usually a single constant.

Basic bounds and integral identities

Conditional probabilities must lie between zero and one, up to changes on a set of probability zero. This follows directly from the defining identity, without needing a pointwise interpretation of the conditioning information.

Theorem: Let $Q=\mathbb P(A\mid\mathcal G)$. Then $$ 0\le Q\le 1\quad\text{almost surely}, $$ and, for every $B\in\mathcal G$, $$ \int_B Q\,d\mathbb P=\mathbb P(A\cap B). $$ Consequently, $$ \mathbb E[Q]=\mathbb P(A). $$

Proof. The integral identity is the defining property. To prove the lower bound, suppose that $\mathbb P(Q<0)>0$. Since $$ \{Q<0\}=\bigcup_{n\in\mathbb N}\{Q\le -1/n\}, $$ there is some $n\in\mathbb N$ for which $B=\{Q\le-1/n\}$ has positive probability. This event belongs to $\mathcal G$, because $Q$ is $\mathcal G$-measurable. Therefore, $$ \int_B Q\,d\mathbb P\le-\frac{\mathbb P(B)}{n}<0. $$ But the defining identity says that this integral is $\mathbb P(A\cap B)$, which is nonnegative. This contradiction proves $Q\ge0$ almost surely.

For the upper bound, if $\mathbb P(Q>1)>0$, then, since $$ \{Q>1\}=\bigcup_{n\in\mathbb N}\{Q\ge1+1/n\}, $$ there is an $n$ for which $B=\{Q\ge1+1/n\}$ has positive probability. The identity and the event inclusion $A\cap B\subseteq B$ give $$ \int_B Q\,d\mathbb P=\mathbb P(A\cap B)\le\mathbb P(B). $$ On the other hand, the choice of $B$ gives $$ \int_B Q\,d\mathbb P\ge(1+1/n)\mathbb P(B)>\mathbb P(B), $$ a contradiction. Thus $Q\le1$ almost surely. Finally, taking $B=\Omega$ in the defining identity yields $\mathbb E[Q]=\mathbb P(A)$. \p>

The integral identity also explains why an almost-sure bound is the appropriate kind of bound. A conditional expectation is defined only up to almost-sure equality, so changing its values on a null set cannot change any of these integrals or probabilities.

Worked examples

Worked Example: Conditioning on trivial information

Let $\mathcal G=\{\varnothing,\Omega\}$, the trivial sigma-algebra. For any event $A$, define the constant random variable $Q(\omega)=\mathbb P(A)$. It is $\mathcal G$-measurable. The only choices for $B\in\mathcal G$ are $\varnothing$ and $\Omega$. For $B=\varnothing$, both sides of the defining identity are zero. For $B=\Omega$, $$ \int_\Omega Q\,d\mathbb P =\mathbb P(A)\mathbb P(\Omega) =\mathbb P(A) =\mathbb P(A\cap\Omega). $$ Thus $\mathbb P(A\mid\mathcal G)=\mathbb P(A)$ almost surely. With no information beyond the trivial sigma-algebra, the conditional probability is the original probability.

Worked Example: A finite partition

Let $\Omega=\{\omega_1,\omega_2,\omega_3,\omega_4,\omega_5\}$, with probabilities $0.1,0.2,0.3,0.15,0.25$, respectively. Let $\mathcal G$ be generated by the partition $C_1=\{\omega_1,\omega_2,\omega_3\}$ and $C_2=\{\omega_4,\omega_5\}$, and set $A=\{\omega_1,\omega_3,\omega_4\}$. The probabilities of the partition cells and their intersections with $A$ are $$ \mathbb P(C_1)=0.6,\qquad \mathbb P(A\cap C_1)=0.4, $$ $$ \mathbb P(C_2)=0.4,\qquad \mathbb P(A\cap C_2)=0.15. $$ Define $Q$ to be $2/3$ on $C_1$ and $3/8$ on $C_2$. It is $\mathcal G$-measurable. For $B=C_1$, $$ \int_{C_1}Q\,d\mathbb P=\frac23(0.6)=0.4=\mathbb P(A\cap C_1). $$ For $B=C_2$, $$ \int_{C_2}Q\,d\mathbb P=\frac38(0.4)=0.15=\mathbb P(A\cap C_2). $$ The identity holds for $B=\varnothing$ and $B=\Omega$ as well: both sides are zero for the empty set, and for $\Omega$ both sides equal $0.4+0.15=0.55=\mathbb P(A)$. These are all the events in $\mathcal G$, so $Q$ is a version of $\mathbb P(A\mid\mathcal G)$.

Worked Example: The event is already measurable

If $A\in\mathcal G$, then $\mathbf 1_A$ is $\mathcal G$-measurable. For every $B\in\mathcal G$, $$ \int_B\mathbf 1_A\,d\mathbb P =\mathbb P(A\cap B). $$ Thus $\mathbf 1_A$ itself satisfies the defining identity, and uniqueness of conditional expectation gives $$ \mathbb P(A\mid\mathcal G)=\mathbf 1_A\quad\text{almost surely}. $$ Once the conditioning information determines whether $A$ occurred, its conditional probability is one on $A$ and zero off $A$, up to null sets.

A continuous example: information about one coordinate

Worked Example: Probability below the diagonal

Take $\Omega=[0,1]^2$ with two-dimensional Lebesgue probability measure. Write an outcome as $(x,y)$, let $\mathcal G$ be the sigma-algebra generated by the first-coordinate map $(x,y)\mapsto x$, and let $$ A=\{(x,y)\in[0,1]^2:y\le x\}. $$ Define $Q(x,y)=x$. This is $\mathcal G$-measurable. Each $B\in\mathcal G$ has the form $C\times[0,1]$ for a Borel set $C\subseteq[0,1]$. Integrating $Q$ over $B$ gives $$ \int_B Q\,d\mathbb P =\int_C\int_0^1 x\,dy\,dx =\int_C x\,dx. $$ The area of $A\cap B$, computed by integrating first with respect to $y$, is $$ \mathbb P(A\cap B) =\int_C\int_0^x 1\,dy\,dx =\int_C x\,dx. $$ The two quantities agree for every $B\in\mathcal G$. Hence $$ \mathbb P(A\mid\mathcal G)(x,y)=x\quad\text{almost surely}. $$ Knowing the first coordinate $x$ leaves a vertical segment of height $x$ in the event $A$, out of a full vertical segment of height one. The integral calculation verifies that this geometric interpretation has the required conditional-probability identity.

Why the conditioning sigma-algebra matters

A conditional probability given a sigma-algebra is a random variable, not a probability assigned by dividing by the probability of that sigma-algebra. It is characterized by measurability and by its integrals over every event in the sigma-algebra. When the sigma-algebra comes from a finite partition, the value is constant on each partition cell of positive probability and equals the probability of the event within that cell. The finite example illustrates this; the defining identity continues to apply when the information is not a finite partition.

There is a related distinction between conditioning on a sigma-algebra and conditioning on one event. If $C$ has positive probability, conditioning on the two-cell sigma-algebra generated by $C$ gives the value $\mathbb P(A\cap C)/\mathbb P(C)$ on $C$, and the corresponding ratio on $C^c$ when $\mathbb P(C^c)>0$. This familiar ratio is a special case of the sigma-algebra definition. A ratio on a zero-probability event is not supplied by that formula; the integral characterization is what defines conditional expectation in general.

Key takeaway: For an event $A$, its conditional probability given $\mathcal G$ is a $\mathcal G$-measurable version of $\mathbb E[\mathbf 1_A\mid\mathcal G]$. It lies between zero and one almost surely, and its integral over each $B\in\mathcal G$ is exactly $\mathbb P(A\cap B)$.

Check Your Understanding

Use the defining integral identity to answer the following questions.

Why is conditional probability given a sigma-algebra generally a random variable rather than a single number?
For $Q=\mathbb P(A\mid\mathcal G)$, what does $\int_B Q\,d\mathbb P$ equal when $B\in\mathcal G$?
How does the integral identity rule out $Q<0$ on a set of positive probability?
If $A\in\mathcal G$, which random variable is a version of $\mathbb P(A\mid\mathcal G)$, and why?
In the square example, why does the function $Q(x,y)=x$ satisfy the required integral identity?

Back to Tutorials