Covariances of Indicators and Events

From Events to Covariance

Events describe whether outcomes meet specified conditions; covariance describes how two random quantities vary together. An indicator random variable connects these ideas directly: it records whether an event occurred, taking the value 1 when it did and 0 otherwise. The covariance of two such indicators is therefore a numerical measure of how the occurrence of one event is associated with the occurrence of the other.

This connection is useful because it translates a general covariance calculation into a simple comparison of probabilities. The key comparison is between the probability that both events occur and the product of their separate probabilities. If that product is larger or smaller than the joint probability, the indicators have negative or positive covariance, respectively.

Definition: On a probability space $(\Omega,\mathcal F,P)$, the indicator of an event $A\in\mathcal F$ is the random variable $$ I_A(\omega)= \begin{cases} 1, & \omega\in A,\\ 0, & \omega\notin A. \end{cases} $$ It has expectation $E[I_A]=P(A)$. For events $A$ and $B$, their indicator covariance is $\operatorname{Cov}(I_A,I_B)$. Both indicators are bounded, so their expectations, variances, and covariance are finite.

The Joint-Probability Identity

The product of two indicators is itself an indicator: it equals 1 exactly when both events occur. Thus $I_AI_B=I_{A\cap B}$. This pointwise identity is the step that makes the covariance formula immediate.

Theorem: For any events $A,B\in\mathcal F$, $$ \operatorname{Cov}(I_A,I_B)=P(A\cap B)-P(A)P(B). $$ Consequently, the covariance is positive when the joint probability exceeds the product of the marginal probabilities, negative when it is smaller, and zero when the two quantities are equal.

Proof. By the definition of covariance and the expectation of an indicator, $$ \operatorname{Cov}(I_A,I_B) =E[I_AI_B]-E[I_A]E[I_B]. $$ For every outcome $\omega$, $I_A(\omega)I_B(\omega)=1$ if $\omega$ belongs to both $A$ and $B$, and is 0 otherwise. Hence $I_AI_B=I_{A\cap B}$, so $E[I_AI_B]=P(A\cap B)$. Also, $E[I_A]=P(A)$ and $E[I_B]=P(B)$. Substitution gives the stated identity. $\square$

The product $P(A)P(B)$ is the joint probability the events would have if they were independent, given their marginal probabilities. The covariance therefore measures the difference between the observed joint probability and that independence benchmark. It is measured in probability units, and its size depends on the marginal event probabilities as well as on their association.

Worked Example: Positive Association on a Die

Roll a fair six-sided die once. Let $A$ be the event that the result is even, and let $B$ be the event that the result is at least 4. Calculate the covariance of the two event indicators and interpret it.

The outcomes in $A$ are $\{2,4,6\}$, so $P(A)=3/6=1/2$. The outcomes in $B$ are $\{4,5,6\}$, so $P(B)=3/6=1/2$. Their intersection is $\{4,6\}$, giving $P(A\cap B)=2/6=1/3$. Therefore, $$ \operatorname{Cov}(I_A,I_B) =\frac{1}{3}-\left(\frac{1}{2}\right)\left(\frac{1}{2}\right) =\frac{1}{3}-\frac{1}{4} =\frac{1}{12}. $$

The covariance is positive: observing an even result is associated with a greater chance that the result is at least 4. In fact, $P(B\mid A)=2/3$, which exceeds the overall probability $P(B)=1/2$. The event $A$ does not determine the result, but it shifts the probability of $B$ upward.

Reading Covariance Through Conditional Probability

When $P(A)>0$, the joint probability can also be written $P(A\cap B)=P(A)P(B\mid A)$. Substituting this expression gives a conditional-probability interpretation of the covariance.

Result: If $P(A)>0$, then $$ \operatorname{Cov}(I_A,I_B) =P(A)\bigl(P(B\mid A)-P(B)\bigr). $$ Thus the covariance is positive precisely when $P(B\mid A)>P(B)$, and negative precisely when $P(B\mid A)<P(B)$. If $P(A)=0$, then $\operatorname{Cov}(I_A,I_B)=0$; the conditional probability $P(B\mid A)$ is not defined by the usual ratio formula.

Derivation. By the definition of conditional probability, $P(B\mid A)=P(A\cap B)/P(A)$ when $P(A)>0$, so $P(A\cap B)=P(A)P(B\mid A)$. Use the joint-probability identity and factor out $P(A)$: $$ \operatorname{Cov}(I_A,I_B) =P(A)P(B\mid A)-P(A)P(B) =P(A)\bigl(P(B\mid A)-P(B)\bigr). $$ If $P(A)=0$, then $P(A\cap B)=0$, and the joint-probability identity gives covariance $0-P(A)P(B)=0$. $\square$

This formula compares the probability of $B$ among outcomes in $A$ with its overall probability. The factor $P(A)$ matters: the difference in conditional probabilities is weighted by how often $A$ occurs. The formula does not make covariance a causal effect. It describes an association between events under the specified probability model; a causal interpretation requires additional assumptions and a suitable study design.

Worked Example: Association in a Two-by-Two Table

In a group of 1,000 people, 300 have a particular exposure and 200 experience an outcome. Of the 300 exposed people, 120 experience the outcome. Choose one person uniformly at random. Let $A$ be the event that the person is exposed and $B$ the event that the person experiences the outcome. Find the covariance of $I_A$ and $I_B$.

The marginal probabilities are $P(A)=300/1000=0.30$ and $P(B)=200/1000=0.20$. The joint probability is $P(A\cap B)=120/1000=0.12$. Therefore, $$ \operatorname{Cov}(I_A,I_B) =0.12-(0.30)(0.20) =0.12-0.06 =0.06. $$

The positive value means that the joint outcome occurs more often than it would under independence with these same marginal probabilities: the observed joint probability is $0.12$, while the product benchmark is $0.06$. The conditional form gives the same result. Among exposed people, the outcome probability is $120/300=0.40$, so $$ P(A)\bigl(P(B\mid A)-P(B)\bigr) =0.30(0.40-0.20) =0.06. $$

For comparison, among the 700 unexposed people, 80 experience the outcome, so their outcome probability is $80/700=4/35$, about $0.114$. These calculations describe association in this group; by themselves, they do not establish that exposure causes the outcome.

Independence, Complements, and Negative Association

The identity also gives an exact event-level criterion for independence. Events $A$ and $B$ are independent if and only if $P(A\cap B)=P(A)P(B)$. The identity says this holds if and only if their indicator covariance is zero. This criterion concerns indicators of events; zero covariance between arbitrary random variables does not in general imply their independence.

Complements reverse the sign of the association when only one event is complemented. Since $I_{A^c}=1-I_A$, the covariance identity or the definition of covariance gives $$ \operatorname{Cov}(I_{A^c},I_B)=-\operatorname{Cov}(I_A,I_B). $$ Complementing both events leaves the covariance unchanged: $$ \operatorname{Cov}(I_{A^c},I_{B^c})=\operatorname{Cov}(I_A,I_B). $$ These signs match the interpretation: if $A$ tends to occur with $B$, then not-$A$ tends not to occur with $B$, while not-$A$ and not-$B$ retain the same direction of association as $A$ and $B$.

Worked Example: Negative Association for Disjoint Events

Draw one card uniformly from a standard 52-card deck. Let $A$ be the event that the card is a heart and $B$ the event that it is a spade. These events are disjoint, and each has probability $13/52=1/4$. Thus $P(A\cap B)=0$, and $$ \operatorname{Cov}(I_A,I_B) =0-\left(\frac{1}{4}\right)\left(\frac{1}{4}\right) =-\frac{1}{16}. $$

The covariance is negative because once the card is known to be a heart, it cannot be a spade: $P(B\mid A)=0$, less than the overall probability $P(B)=1/4$. In this example the events are not independent, since independence would require their intersection probability to be $1/16$, not 0.

Worked Example: Zero Covariance for Independent Events

Toss two independent fair coins. Let $A$ be the event that the first coin is heads and $B$ the event that the second coin is heads. The four ordered outcomes $HH, HT, TH, TT$ are equally likely. Hence $P(A)=1/2$, $P(B)=1/2$, and only $HH$ belongs to $A\cap B$, so $P(A\cap B)=1/4$. Therefore, $$ \operatorname{Cov}(I_A,I_B) =\frac{1}{4}-\left(\frac{1}{2}\right)\left(\frac{1}{2}\right) =0. $$

Here the zero covariance reflects independence: the second coin's chance of heads remains $1/2$ whether or not the first coin is heads. This differs from settings with general random variables, where zero covariance alone need not imply independence.

Common Mistake: Using the Wrong Probability Product

A frequent error is to treat $P(A\cap B)$ as though it were automatically $P(A)P(B)$. That product is justified when $A$ and $B$ are independent; for arbitrary events, the joint probability must be found from the model or data. For example, in the die calculation, multiplying the marginal probabilities gives $1/4$, but the actual intersection probability is $1/3$. The covariance is their difference, $1/3-1/4=1/12$, not zero.

A second error is to interpret covariance itself as the joint probability. The joint probability $P(A\cap B)$ cannot be negative, but covariance can be negative, as the card example shows. Covariance is a difference between a joint probability and a product benchmark, not a probability of an event.

Key takeaway: For any two events, $\operatorname{Cov}(I_A,I_B)=P(A\cap B)-P(A)P(B)$. The sign records whether the events occur together more or less often than the independence benchmark; zero is exactly the independence condition for the events.

Check Your Understanding

Use the joint-minus-product identity and, where appropriate, its conditional-probability form.

If $P(A)=0.4$, $P(B)=0.3$, and $P(A\cap B)=0.18$, calculate $\operatorname{Cov}(I_A,I_B)$ and state its sign.
If $P(A)>0$ and $\operatorname{Cov}(I_A,I_B)=0$, what does this imply about $P(B\mid A)$ and $P(B)$?
Two events are disjoint and each has probability $0.2$. Find the covariance of their indicators.
Explain why the statement “zero covariance implies independence” is valid for two event indicators but not for arbitrary random variables.
If $\operatorname{Cov}(I_A,I_B)>0$, what is the sign of $\operatorname{Cov}(I_{A^c},I_B)$?