Covariance from Conditional Expectation

Why Conditional Means Can Simplify Covariance

A covariance calculation can be difficult when the joint distribution of $X$ and $Y$ is complicated. It may be much easier to find the conditional mean of $X$ given another variable $Z$. When $Y$ is completely determined by $Z$, that conditional mean contains all the information needed to calculate the covariance with $Y$. This is useful in models where $Z$ describes a group, operating environment, or event count, and the response of interest is summarized by $E[X\mid Z]$.

The key condition is stronger than saying $Y$ is associated with $Z$: there must be a function $g$ such that $Y=g(Z)$ with probability 1. In other words, once $Z$ is known, $Y$ is known as well. The result is aligned with the SOA Exam P (Probability) syllabus.

Definition: For random variables $X$ and $Y$ with finite second moments, their covariance is $$ \operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]. $$ The conditional expectation $E[X\mid Z]$ is itself a random variable: its value depends on the realized value of $Z$.

Theorem: Suppose $X$ and $Y$ have finite second moments, and $Y=g(Z)$ with probability 1 for some measurable function $g$. Then $$ \operatorname{Cov}(X,Y)=\operatorname{Cov}(E[X\mid Z],Y). $$

Why the Identity Holds

The condition that $Y$ is determined by $Z$ lets us treat $Y$ as known when conditioning on $Z$. In particular, the conditional expectation of $XY$ given $Z$ is $Y E[X\mid Z]$. Taking expectations gives the identity.

Start from the definition.
$\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]$.

Condition the product on $Z$.
Since $Y=g(Z)$, it is fixed once $Z$ is known, so $E[XY\mid Z]=Y E[X\mid Z]$.

Take expectations.
The law of total expectation gives $E[XY]=E[Y E[X\mid Z]]$.

Recognize the covariance.
Also $E[E[X\mid Z]]=E[X]$, so $E[Y E[X\mid Z]]-E[X]E[Y]=\operatorname{Cov}(E[X\mid Z],Y)$.

Thus the proof follows directly from the covariance definition, the fact that a function of $Z$ can be taken outside conditional expectation given $Z$, and the law of total expectation. The argument requires the expectations in the calculation to exist; finite second moments are a convenient sufficient condition.

Using the Identity in Discrete Models

When $Z$ takes a small number of values, calculate $E[X\mid Z=z]$ at each value and then average over the distribution of $Z$. If $Y=g(Z)$, the covariance can be found using those conditional means without specifying the full conditional distribution of $X$.

Worked Example: A Hypothetical Service-Tier Model

A hypothetical service provider classifies each account as tier $Z=1,2,$ or $3$, with probabilities $0.20, 0.50,$ and $0.30$, respectively. Let $X$ be the number of service interruptions in a year. The conditional means are $E[X\mid Z=1]=2$, $E[X\mid Z=2]=5$, and $E[X\mid Z=3]=1$. Define a score $Y$ by assigning scores $4,1,$ and $3$ to the three tiers. Find $\operatorname{Cov}(X,Y)$.

Method. The score is determined by the tier, so $Y=g(Z)$. Use the conditional-expectation identity and calculate the moments from the tier probabilities.

First, $$ E[X]=0.20(2)+0.50(5)+0.30(1)=0.40+2.50+0.30=3.20. $$

The mean score is $$ E[Y]=0.20(4)+0.50(1)+0.30(3)=0.80+0.50+0.90=2.20. $$

By the identity, $E[XY]=E[Y E[X\mid Z]]$. The three conditional-mean contributions are $$ E[XY]=0.20(4)(2)+0.50(1)(5)+0.30(3)(1)=1.60+2.50+0.90=5.00. $$ Therefore, $$ \operatorname{Cov}(X,Y)=5.00-(3.20)(2.20)=5.00-7.04=-2.04. $$ The negative covariance is consistent with the tier values: the highest score occurs in tier 1, whose conditional mean interruption count is lower than the mean for tier 2. The final check is that the three probabilities sum to $1$, and the weighted means above use all three tiers.

Common mistake avoided: Do not multiply $E[X]$ by $E[Y]$ and treat that as $E[XY]$. The product moment must account for how the conditional mean of $X$ changes across the tiers.

Using a Continuous Conditioning Variable

The same reasoning applies when $Z$ is continuous. Instead of summing over categories, calculate expectations by integration using the distribution of $Z$. The identity is especially helpful when the conditional mean is a simple function of $Z$, even if the full conditional distribution of $X$ is not needed.

Worked Example: A Hypothetical Sensor Calibration Model

A hypothetical sensor operates at a setting $Z$ that is uniformly distributed on $(0,1)$. Its measured response $X$ satisfies $E[X\mid Z]=1+2Z$. Define a setting score $Y=Z^2$. Find $\operatorname{Cov}(X,Y)$. Assume the variables have finite second moments.

Method. Since $Y=Z^2$, it is determined by $Z$. Hence $\operatorname{Cov}(X,Y)=\operatorname{Cov}(1+2Z,Z^2)$. For $Z\sim\operatorname{Uniform}(0,1)$, $E[Z^k]=1/(k+1)$ for nonnegative integers $k$, as follows by integrating $z^k$ from 0 to 1.

The necessary moments are $$ E[Z]=\frac{1}{2},\qquad E[Z^2]=\frac{1}{3},\qquad E[Z^3]=\frac{1}{4}. $$ Thus $$ E[X]=E[1+2Z]=1+2\left(\frac12\right)=2, \qquad E[Y]=E[Z^2]=\frac13. $$

For the product moment, use the conditional mean: $$ E[XY]=E[Z^2 E[X\mid Z]] =E[Z^2(1+2Z)] =E[Z^2]+2E[Z^3] =\frac13+2\left(\frac14\right)=\frac56. $$ Therefore, $$ \operatorname{Cov}(X,Y)=\frac56-(2)\left(\frac13\right)=\frac56-\frac23=\frac16. $$ The result is positive: the conditional mean response increases with $Z$, and the score $Z^2$ also increases with $Z$. As a check, both use the same $Z$ on $(0,1)$, and the computed product moment $5/6$ exceeds the product of the means $2/3$.

Common mistake avoided: The conditional mean $E[X\mid Z]$ is not a constant equal to $E[X]$. It varies with $Z$, so it must remain inside the expectation involving $Y$.

Thresholds and Count Variables

A useful application occurs when $Z$ is a count and $Y$ indicates whether the count crosses a threshold. The indicator is determined by $Z$, so the identity can replace $X$ by its conditional mean. This can avoid finding the joint distribution of $X$ and the indicator.

Worked Example: A Hypothetical Alert-Count Threshold

In a hypothetical monitoring system, the number of daily alerts $Z$ has a Poisson distribution with parameter $1$. A response workload $X$ satisfies $E[X\mid Z=z]=10+3z$ workload units. Let $Y=1$ if at least two alerts occur and $Y=0$ otherwise. Find $\operatorname{Cov}(X,Y)$.

Method. Here $Y=I(Z\geq2)$, so it is determined by $Z$, and the covariance equals $\operatorname{Cov}(10+3Z,I(Z\geq2))$. For a Poisson random variable with parameter $1$, $P(Z=k)=e^{-1}/k!$ and $E[Z]=1$.

The threshold probability is $$ E[Y]=P(Z\geq2)=1-P(Z=0)-P(Z=1)=1-e^{-1}-e^{-1}=1-\frac{2}{e}. $$

For the product moment, $ZY=Z$ when $Z\geq2$ and is zero otherwise. Consequently, $$ E[ZY]=E[Z]-1\cdot P(Z=1)=1-\frac{1}{e}, $$ because the only positive count excluded from $Z I(Z\geq2)$ is $Z=1$. Also, $$ E[X]=E[10+3Z]=10+3(1)=13. $$ Using the identity, $$ E[XY]=E[(10+3Z)Y]=10E[Y]+3E[ZY] =10\left(1-\frac{2}{e}\right)+3\left(1-\frac1e\right) =13-\frac{23}{e}. $$ Thus $$ \operatorname{Cov}(X,Y) =13-\frac{23}{e}-13\left(1-\frac2e\right) =\frac{3}{e} \approx 1.1036. $$ The probability $P(Z\geq2)$ is approximately $0.2642$, and $E[ZY]$ is approximately $0.6321$; substituting these values also gives a covariance of approximately $1.1036$. The probability is reported to four decimals.

Common mistake avoided: For $E[ZY]$, do not use $E[Z]P(Y=1)$ as if $Z$ and $Y$ were independent. The indicator is defined from $Z$; instead, remove the excluded $Z=1$ contribution from $E[Z]$.

When the Identity Does Not Apply

Knowing that $Y$ is related to $Z$ is not enough. The identity requires that $Y$ be determined by $Z$. If $Y$ retains random variation after $Z$ is known, it cannot generally be taken outside the conditional expectation of $XY$.

Worked Example: Checking the Determination Condition

Let $Z$ be constant, and let $X=Y$ take the values $-1$ and $1$ with probability $1/2$ each. Compare the actual covariance with $\operatorname{Cov}(E[X\mid Z],Y)$.

Method. Because $Z$ is constant, $E[X\mid Z]=E[X]=0$. Therefore, $$ \operatorname{Cov}(E[X\mid Z],Y)=\operatorname{Cov}(0,Y)=0. $$ But $X=Y$, $E[X]=E[Y]=0$, and $E[XY]=E[X^2]=1$, so $$ \operatorname{Cov}(X,Y)=1-(0)(0)=1. $$ The two covariances differ. Here $Y$ is not determined by the constant $Z$, so the theorem's condition is not satisfied. This example checks the condition rather than providing an alternative formula.

Common Exam Traps

Using the identity when $Y$ is only associated with $Z$. Verify that $Y=g(Z)$ with probability 1. A conditional relationship alone is not sufficient.
Replacing $E[XY]$ by $E[X]E[Y]$. These are equal only under additional conditions such as independence; covariance calculations must preserve the dependence.
Dropping the conditioning variable too early. Compute $E[Y E[X\mid Z]]$, not $E[Y]E[X]$, unless a separate justification allows the factorization.
Confusing a conditional mean with a realized value. $E[X\mid Z]$ summarizes the mean of $X$ at each value of $Z$; it does not assert that $X$ equals that mean.

Key takeaway: If $Y$ is a function of $Z$, then $\operatorname{Cov}(X,Y)=\operatorname{Cov}(E[X\mid Z],Y)$. The practical calculation is to find the conditional mean of $X$, multiply it by the $Z$-determined value of $Y$, and take expectations under the distribution of $Z$.

Check Your Understanding

For each problem, first check whether the variable paired with $X$ is determined by the conditioning variable. Answers are included for checking.

A hypothetical portfolio group $Z$ takes values 1, 2, and 3 with probabilities $0.25, 0.50,$ and $0.25$. The conditional means of a claim count $X$ are 1, 4, and 7. A group score $Y$ is 3, 0, and 2, respectively. Find $\operatorname{Cov}(X,Y)$. (Answer: $-0.7500$)
Let $Z\sim\operatorname{Uniform}(0,2)$, $E[X\mid Z]=2+Z$, and $Y=Z$. Find $\operatorname{Cov}(X,Y)$. (Answer: $1/3$)
Let $Z$ be Bernoulli with $P(Z=1)=0.4$, $E[X\mid Z]=3+2Z$, and $Y=1+5Z$. Find $\operatorname{Cov}(X,Y)$. (Answer: $2.4000$)
Let $Z\sim\operatorname{Poisson}(2)$, $E[X\mid Z]=4+Z$, and $Y=I(Z\geq1)$. Find $\operatorname{Cov}(X,Y)$. (Answer: $2e^{-2}\approx0.2707$)
Let $Z$ take values 0 and 1 with equal probabilities. Given $Z=0$, let $E[X\mid Z]=0$ and $Y=1$; given $Z=1$, let $E[X\mid Z]=7$ and $Y=2$. Calculate the covariance using conditional means. (Answer: $1.7500$)
Let $Z$ be constant and $X=Y$ be equally likely to equal $-1$ or $1$. Is $\operatorname{Cov}(X,Y)=\operatorname{Cov}(E[X\mid Z],Y)$? (Answer: No; the two values are $1$ and $0$, respectively.)

What You'll Learn