Conditional Variance of Sums with Dependence

Why Conditional Variance of a Sum Needs a Dependence Term

When two random quantities are added, their variability depends not only on how much each varies but also on whether they tend to move together. Conditioning on a third variable does not remove this issue: within a particular conditioning group, the two quantities may still be dependent. A reliable calculation must therefore include the conditional covariance.

This matters whenever a total is assembled from components and the analysis separates cases by a known characteristic, such as a risk category or operating state. The conditional variances describe variation within each case; the conditional covariance adjusts for how the components co-vary in that case.

Definition: For random variables $X$, $W$, and $Y$ with finite second moments, the conditional variance of $X$ given $Y$ is $$ \operatorname{Var}(X\mid Y)=E\!\left[(X-E[X\mid Y])^2\mid Y\right]. $$ The conditional covariance of $X$ and $W$ given $Y$ is $$ \operatorname{Cov}(X,W\mid Y) =E\!\left[(X-E[X\mid Y])(W-E[W\mid Y])\mid Y\right]. $$ These are random variables whose values depend on $Y$. For a discrete conditioning variable, the formulas apply within each group having positive probability; for continuous $Y$, they are understood as conditional moments given $Y$, defined almost surely.

The Conditional Variance Formula for a Sum

The conditional mean of a sum is the sum of its conditional means: $E[X+W\mid Y]=E[X\mid Y]+E[W\mid Y]$. Subtracting this mean centers each component separately. Expanding the square then reveals the role of conditional covariance.

Theorem: If $X$ and $W$ have finite second moments, then $$ \operatorname{Var}(X+W\mid Y) =\operatorname{Var}(X\mid Y)+\operatorname{Var}(W\mid Y) +2\operatorname{Cov}(X,W\mid Y). $$

To see why the formula holds, write the centered sum as the sum of the separately centered variables. The cross-product terms in its square have conditional expectation equal to the conditional covariance.

\begin{aligned} \operatorname{Var}(X+W\mid Y) &=E\!\left[\left\{(X-E[X\mid Y])+(W-E[W\mid Y])\right\}^2\mid Y\right]\\ &=E[(X-E[X\mid Y])^2\mid Y]\\ &\quad+E[(W-E[W\mid Y])^2\mid Y]\\ &\quad+2E[(X-E[X\mid Y])(W-E[W\mid Y])\mid Y]\\ &=\operatorname{Var}(X\mid Y)+\operatorname{Var}(W\mid Y) +2\operatorname{Cov}(X,W\mid Y). \end{aligned}

The same result can be written using conditional raw moments: $\operatorname{Cov}(X,W\mid Y)=E[XW\mid Y]-E[X\mid Y]E[W\mid Y]$. Thus, when the conditional means and the conditional product moment are available, they can be used to find the covariance term. The formula does not require conditional independence.

If $X$ and $W$ are conditionally independent given $Y$, and their conditional second moments are finite, their conditional covariance is zero. In that special case the conditional variance of the sum is the sum of the two conditional variances. Conditional independence is a sufficient condition for omitting the covariance term; unconditional independence alone does not justify omitting a conditional covariance.

Worked Example: Combining Conditional Variances Across Two Groups

A hypothetical service portfolio has two components, $X$ and $W$, measured in thousands of dollars. A risk class $Y$ has $P(Y=0)=0.4$ and $P(Y=1)=0.6$. In class 0, the conditional means of $X$ and $W$ are 2 and 3, their conditional variances are 1 and 4, and their conditional covariance is $-1$. In class 1, the corresponding means are 5 and 1, the conditional variances are 4 and 1, and the conditional covariance is $0.5$. Find the overall variance of $S=X+W$.

Method. First calculate the variance of the sum within each class, retaining the conditional covariance term. Then combine the within-class variance with the variance of the class-specific conditional means.

In class 0,

E[S\mid Y=0]=2+3=5,\qquad \operatorname{Var}(S\mid Y=0)=1+4+2(-1)=3.

In class 1,

E[S\mid Y=1]=5+1=6,\qquad \operatorname{Var}(S\mid Y=1)=4+1+2(0.5)=6.

The overall mean is

$$ E[S]=0.4(5)+0.6(6)=2+3.6=5.6. $$

The expected conditional variance, representing average variation within a class, is

E[\operatorname{Var}(S\mid Y)] =0.4(3)+0.6(6)=1.2+3.6=4.8.

The variance of the two conditional means is

\begin{aligned} \operatorname{Var}(E[S\mid Y]) &=0.4(5-5.6)^2+0.6(6-5.6)^2\\ &=0.4(0.36)+0.6(0.16)\\ &=0.144+0.096=0.24. \end{aligned}

By the law of total variance, the overall variance is $4.8+0.24=5.04$, in squared thousands of dollars. The calculation checks: the within-class contribution is 4.8 and the between-class contribution is 0.24, so their sum is 5.04.

Common mistake avoided: Dropping the conditional covariance would give class variances $1+4=5$ and $4+1=5$, leading to an incorrect total of $5+0.24=5.24$. The covariance is negative in class 0 and positive in class 1; neither can be ignored simply because the groups are being analyzed separately.

Conditional Dependence Can Differ Across Groups

A covariance term is specific to the conditioning information. It may be positive in one group, negative in another, or zero in one group and nonzero in another. Even if an unconditional covariance is zero, conditional covariances need not be zero group by group. Conversely, a conditional covariance that is zero in every group does not follow just from knowing that the unconditional covariance is zero.

Worked Example: Dependence Within Groups Despite Overall Independence

A hypothetical device is in state $Y=A$ or $Y=B$, each with probability $0.5$. Conditional on state A, $(X,W)$ is $(0,0)$ or $(2,2)$, each with probability $0.5$. Conditional on state B, it is $(0,2)$ or $(2,0)$, each with probability $0.5$. Find $\operatorname{Var}(X+W\mid Y)$ in each state and the overall variance of $X+W$.

In state A. Both components have conditional mean 1 and conditional variance 1. Their conditional product has mean $E[XW\mid Y=A]=0.5(0)+0.5(4)=2$, so $\operatorname{Cov}(X,W\mid Y=A)=2-(1)(1)=1$. Therefore,

\operatorname{Var}(X+W\mid Y=A)=1+1+2(1)=4.

In state B. Again, each component has conditional mean 1 and variance 1. Here $XW=0$ for both possible outcomes, so the conditional covariance is $0-(1)(1)=-1$. Thus,

\operatorname{Var}(X+W\mid Y=B)=1+1+2(-1)=0.

The conditional sum equals 0 or 4 in state A, and equals 2 in state B. Its conditional mean is 2 in both states, so its overall mean is 2. The expected conditional variance is $0.5(4)+0.5(0)=2$, and the variance of the conditional means is 0. Hence the overall variance is 2.

As a check, the four unconditional pairs $(0,0),(2,2),(0,2),(2,0)$ each have probability $0.25$. The corresponding sums are $0,4,2,2$, with mean 2 and variance $0.25(4+4+0+0)=2$. The components are independent overall because all four pairs are equally likely, yet they are dependent within each state. This example shows why overall independence does not remove the need to inspect conditional dependence.

Using a Shared Random Driver

A common source of dependence is a variable that shifts both components. Once that shared driver is conditioned on, the remaining fluctuations may have a different, simpler dependence structure. The following calculation separates the contribution of the shared driver from the conditional variation.

Worked Example: A Shared State and Correlated Residuals

Let $Y$ be 0 or 2 with probability $0.5$ each. Let $Z$ and $E$ be independent of $Y$ and of each other, with means 0 and variances $\operatorname{Var}(Z)=1$ and $\operatorname{Var}(E)=3$. Define $X=Y+Z$ and $W=2Y-Z+E$. Find the conditional variance of $X+W$ given $Y$, and its overall variance.

Given $Y$, the conditional means are $E[X\mid Y]=Y$ and $E[W\mid Y]=2Y$. The centered parts are $Z$ and $-Z+E$. Their conditional variances and covariance are

\begin{aligned} \operatorname{Var}(X\mid Y)&=\operatorname{Var}(Z)=1,\\ \operatorname{Var}(W\mid Y)&=\operatorname{Var}(-Z+E)=1+3=4,\\ \operatorname{Cov}(X,W\mid Y)&=\operatorname{Cov}(Z,-Z+E)=-1+0=-1. \end{aligned}

The conditional variance of the sum is therefore

\operatorname{Var}(X+W\mid Y)=1+4+2(-1)=3.

Also, $X+W=3Y+E$, so its conditional mean is $3Y$. Since $Y$ takes 0 and 2 equally often, $E[Y]=1$ and $\operatorname{Var}(Y)=0.5(0-1)^2+0.5(2-1)^2=1$. The variance of the conditional mean is therefore $\operatorname{Var}(3Y)=9$. Total variance gives

\operatorname{Var}(X+W)=E[\operatorname{Var}(X+W\mid Y)] +\operatorname{Var}(E[X+W\mid Y])=3+9=12.

This also checks directly from $X+W=3Y+E$: $Y$ and $E$ are independent, so $\operatorname{Var}(3Y+E)=9(1)+3=12$. The negative conditional covariance reduces the within-state variance from $1+4=5$ to 3.

A Reliable Calculation Process

For problems with several conditioning groups, organize the work so that conditional moments are not confused with unconditional ones.

Specify the conditioning information.
List the possible values or groups of $Y$, and use only groups with positive probability.

Find each group’s conditional moments.
For every group, identify the two conditional variances and the conditional covariance. Keep the group label attached to each quantity.

Calculate the conditional variance of the sum.
Use $\operatorname{Var}(X+W\mid Y)=\operatorname{Var}(X\mid Y)+\operatorname{Var}(W\mid Y)+2\operatorname{Cov}(X,W\mid Y)$ separately in each group.

Combine groups only if an overall variance is requested.
Average the conditional variances, then add the variance of the conditional means using the law of total variance.

A useful check is to ask whether the covariance sign makes sense. Positive conditional covariance increases the variance of the sum; negative conditional covariance decreases it. The result must still be nonnegative, as every variance is. If a calculation produces a negative conditional variance, recheck the moments, the covariance sign, or the factor of 2.

The key distinction is between within-group variation and variation between conditional means. The conditional covariance belongs in the within-group variance of the sum. The law of total variance then adds the between-group variation; it does not replace or absorb the conditional covariance calculation.

Key takeaway: For a dependent sum, calculate the conditional covariance in the same conditioning group as the conditional variances. Add twice that covariance to obtain the conditional variance of the sum; add between-group variation only when moving from conditional to overall variance.

Check Your Understanding

For each problem, include the conditional covariance term and distinguish conditional variance from overall variance when needed.

Given a value of $Y$, suppose $\operatorname{Var}(X\mid Y)=7$, $\operatorname{Var}(W\mid Y)=3$, and $\operatorname{Cov}(X,W\mid Y)=-2$. Find $\operatorname{Var}(X+W\mid Y)$. (Answer: 6)
A two-group portfolio has $P(Y=A)=0.25$ and $P(Y=B)=0.75$. The conditional means of $X+W$ are 4 and 7. In group A, the component variances are 2 and 5 and their covariance is 1. In group B, the variances are 1 and 4 and their covariance is $-0.5$. Find the overall variance of $X+W$. (Answer: 6.9375)
Two groups have probabilities 0.4 and 0.6. In the first, the conditional mean of $X+W$ is 2, the component variances are 1 and 4, and the covariance is 0.5. In the second, the conditional mean is 5, the component variances are 4 and 1, and the covariance is $-1$. Find the overall variance of $X+W$. (Answer: 6.36)
Let $Y$ take values 0 and 2 with equal probabilities. Let $U,V$ be independent of $Y$, with zero means, variances 1 and 4, and covariance $-1$. Set $X=Y+U$ and $W=2Y+V$. Find $\operatorname{Var}(X+W)$. (Answer: 12)
Two groups are equally likely. In group A, $E[X+W\mid Y=A]=3$, the component variances are 2 and 1, and the covariance is 0.5. In group B, $E[X+W\mid Y=B]=5$, the component variances are 1 and 4, and the covariance is $-1$. Find the overall variance of $X+W$. (Answer: 4.5)

What You'll Learn