Choosing Conditioning Variables for Variance

Why the Choice of Conditioning Variable Matters

A variance calculation can be difficult when the unconditional distribution of a random variable combines several groups, regimes, or stages. If a variable \(Z\) identifies those groups, the distribution of \(X\) given \(Z\) may be much simpler than the overall distribution of \(X\). Conditioning then separates variation within each group from variation in the conditional means between groups.

The choice of \(Z\) is a modeling decision. A useful conditioning variable should be defined by the problem, make the conditional law of \(X\) tractable, or reveal an important group structure. It must not be chosen merely because it produces a convenient-looking formula: the conditional distributions and probabilities must accurately represent the stated model.

Theorem: If \(X\) has a finite second moment and \(Z\) is a conditioning variable, then \(\operatorname{Var}(X)=E[\operatorname{Var}(X\mid Z)]+\operatorname{Var}(E[X\mid Z])\). The first term is the expected conditional variance; the second is the variance of the conditional mean. Both terms are nonnegative.

For a discrete \(Z\), the first term is a weighted average of the within-group variances, and the second is the variance of the group means using the probabilities of the groups. For continuous \(Z\), the expectations and variances are computed using the distribution of \(Z\). The same identity applies in either setting, provided the required moments are finite.

Why the Decomposition Holds

Set \(M=E[X\mid Z]\). Add and subtract \(M\) inside the centered variable: \(X-E[X]=(X-M)+(M-E[X])\). The two terms represent the deviation from the conditional mean and the deviation of the conditional mean from the overall mean. When the squared expression is expanded, the cross term has expectation zero.

\begin{aligned} \operatorname{Var}(X) &=E[(X-E[X])^2]\\ &=E[(X-M)^2]+E[(M-E[X])^2] +2E[(X-M)(M-E[X])]. \end{aligned}

To verify the last term vanishes, condition on \(Z\). Since \(M\) is determined by \(Z\), the factor \(M-E[X]\) is fixed under this conditioning, while \(E[X-M\mid Z]=E[X\mid Z]-M=0\). Therefore,

E[(X-M)(M-E[X])] =E\!\left[(M-E[X])E[X-M\mid Z]\right]=0.

Also, \(E[(X-M)^2]=E[\operatorname{Var}(X\mid Z)]\), and \(E[(M-E[X])^2]=\operatorname{Var}(M)\) because \(E[M]=E[X]\). Substitution gives the stated identity. This argument explains why a total-variance calculation needs both components, even if only one seems to describe the variability of immediate interest.

Worked Examples

Worked Example: Conditioning on the Operating Regime

A hypothetical monitoring system records whether a unit produces a disruption during a shift. Let \(R\) be the operating regime, calm or stressed, with \(P(R=\text{calm})=0.8\) and \(P(R=\text{stressed})=0.2\). Given a calm regime, the disruption indicator \(X\) is Bernoulli with parameter \(0.1\); given a stressed regime, it is Bernoulli with parameter \(0.6\). Find \(\operatorname{Var}(X)\) by conditioning on \(R\), then check directly. There is no interest-rate or mortality assumption in this probability model.

Method and choice of conditioning variable. The regime is useful because the problem specifies a simple conditional distribution in each regime. For a Bernoulli variable with parameter \(p\), the mean is \(p\) and the variance is \(p(1-p)\).

In the calm regime, \(E[X\mid R=\text{calm}]=0.1\) and \(\operatorname{Var}(X\mid R=\text{calm})=0.1(0.9)=0.09\). In the stressed regime, the corresponding values are \(0.6\) and \(0.6(0.4)=0.24\). Thus,

E[\operatorname{Var}(X\mid R)] =0.8(0.09)+0.2(0.24)=0.072+0.048=0.12.

The overall mean is \(E[X]=0.8(0.1)+0.2(0.6)=0.08+0.12=0.20\). The variance of the conditional means is

\operatorname{Var}(E[X\mid R]) =0.8(0.1-0.2)^2+0.2(0.6-0.2)^2 =0.008+0.032=0.04.

Therefore, total variance gives \(\operatorname{Var}(X)=0.12+0.04=0.16\).

Direct check. Because \(X\) is an indicator, its unconditional success probability is \(P(X=1)=0.8(0.1)+0.2(0.6)=0.20\). Hence \(X\) is marginally Bernoulli with parameter \(0.20\), and \(\operatorname{Var}(X)=0.20(0.80)=0.16\), as before. The common mistake is to report only \(0.12\), the average conditional variance, and omit the \(0.04\) variation among regime-specific means.

Worked Example: Grouping a Discrete Distribution by Source

A hypothetical sensor shipment comes from source A with probability \(1/4\) and source B with probability \(3/4\). Let \(X\) be a sensor's inspection cost, measured in dollars. Conditional on source A, \(X\) equals \(0\) or \(2\), each with probability \(1/2\). Conditional on source B, \(X\) equals \(2\) or \(6\), each with probability \(1/2\). Find the mean and variance using the source as the conditioning variable, then verify the variance from the marginal distribution. No interest-rate or mortality assumption is involved.

Method and choice of conditioning variable. The source identifies two simple conditional distributions. For source A, \(E[X\mid A]=(0+2)/2=1\), and \(E[X^2\mid A]=(0^2+2^2)/2=2\). Thus \(\operatorname{Var}(X\mid A)=2-1^2=1\). For source B, \(E[X\mid B]=(2+6)/2=4\), and \(E[X^2\mid B]=(2^2+6^2)/2=20\). Thus \(\operatorname{Var}(X\mid B)=20-4^2=4\).

The overall mean and expected conditional variance are

E[X]=\frac14(1)+\frac34(4)=\frac{13}{4}, \qquad E[\operatorname{Var}(X\mid \text{source})] =\frac14(1)+\frac34(4)=\frac{13}{4}.

The variance of the two conditional means is

\begin{aligned} \operatorname{Var}(E[X\mid \text{source}]) &=\frac14\left(1-\frac{13}{4}\right)^2 +\frac34\left(4-\frac{13}{4}\right)^2\\ &=\frac14\left(\frac{81}{16}\right) +\frac34\left(\frac{9}{16}\right) =\frac{27}{16}. \end{aligned}

Therefore, the total variance is \(\frac{13}{4}+\frac{27}{16}=\frac{52}{16}+\frac{27}{16}=\frac{79}{16}=4.9375\) dollars squared.

Direct check. The marginal probabilities are \(P(X=0)=\frac14\cdot\frac12=\frac18\), \(P(X=2)=\frac14\cdot\frac12+\frac34\cdot\frac12=\frac12\), and \(P(X=6)=\frac34\cdot\frac12=\frac38\). They sum to \(1/8+1/2+3/8=1\). Then \(E[X]=0(1/8)+2(1/2)+6(3/8)=13/4\), and \(E[X^2]=0^2(1/8)+2^2(1/2)+6^2(3/8)=31/2\). Consequently,

\operatorname{Var}(X)=E[X^2]-(E[X])^2 =\frac{31}{2}-\left(\frac{13}{4}\right)^2 =\frac{248-169}{16}=\frac{79}{16}.

The check agrees. A useful trap to avoid is merging the two sources without adding the probabilities for their shared outcome \(X=2\); both sources can produce that cost.

Worked Example: Conditioning on a Continuous Variable

Let \(Z\) be uniformly distributed on \((0,2)\). Given \(Z=z\), let \(X\) be uniformly distributed on \((0,z)\). Find \(\operatorname{Var}(X)\) using \(Z\), then verify it from the marginal density of \(X\). This is a continuous probability model; no interest-rate or mortality assumption is relevant.

Method and choice of conditioning variable. Given \(Z=z\), the conditional uniform distribution has mean \(z/2\) and variance \(z^2/12\). Since \(E[Z]=1\), \(\operatorname{Var}(Z)=1/3\), and \(E[Z^2]=4/3\), the two terms are

E[\operatorname{Var}(X\mid Z)] =E\left[\frac{Z^2}{12}\right]=\frac{4/3}{12}=\frac19, \qquad \operatorname{Var}(E[X\mid Z]) =\operatorname{Var}\left(\frac Z2\right)=\frac14\left(\frac13\right)=\frac1{12}.

Thus \(\operatorname{Var}(X)=1/9+1/12=4/36+3/36=7/36\).

Direct check from the marginal density. The density of \(Z\) is \(1/2\) on \((0,2)\), and the conditional density of \(X\) given \(Z=z\) is \(1/z\) for \(0<x<z\). For \(0<x<2\), integrate over \(x<z<2\):

f_X(x)=\int_x^2 \frac{1}{z}\frac12\,dz =\frac12\ln\left(\frac2x\right).

For \(k>-1\), integration by parts gives \(\int_0^2 x^k\ln(2/x)\,dx=2^{k+1}/(k+1)^2\). The boundary term is zero because \(x^{k+1}\ln(2/x)\) tends to zero as \(x\) decreases to zero. Applying this identity,

E[X]=\frac12\int_0^2 x\ln(2/x)\,dx =\frac12\left(\frac{2^2}{2^2}\right)=\frac12, \qquad E[X^2]=\frac12\int_0^2 x^2\ln(2/x)\,dx =\frac12\left(\frac{2^3}{3^2}\right)=\frac49.

Therefore, \(\operatorname{Var}(X)=4/9-(1/2)^2=16/36-9/36=7/36\), agreeing with conditioning. A common error is to use the conditional variance \(z^2/12\) as the unconditional variance. It must be averaged over \(Z\), and the variance of the conditional mean must also be added.

How to Select and Use a Conditioning Variable

When several variables could be used, select one that makes the conditional model easiest to state and calculate. In practice, look for a variable that does at least one of the following:

Identifies a group for which the conditional distribution of \(X\) is already specified.
Reveals a regime, stage, or source that explains why the unconditional distribution is a mixture of simpler cases.
Makes the conditional mean and variance easy to compute, even if the full marginal distribution is awkward.

A conditioning variable does not change the marginal distribution or variance of \(X\). Different valid choices must lead to the same answer when all terms are computed correctly. However, one choice may make the conditional distributions more complicated than another. In a calculation, define the conditioning variable precisely, list its possible values or distribution, and compute both components using that same variable.

The identity is particularly useful when a direct marginal probability table or density would require many cases. It also provides a diagnostic: the expected conditional variance cannot exceed the total variance, because the omitted between-group term is nonnegative. If your calculation gives a negative term or a sum inconsistent with a direct check, revisit the conditional probabilities, means, and variances.

Key takeaway: Choose a conditioning variable that exposes simple conditional behavior, but always include both \(E[\operatorname{Var}(X\mid Z)]\) and \(\operatorname{Var}(E[X\mid Z])\). When practical, a direct calculation is an effective check.

Check Your Understanding

For each problem, identify a useful conditioning variable and calculate as requested.

A device operates in mode A with probability \(0.3\) and mode B with probability \(0.7\). Given A, a failure indicator \(X\) is Bernoulli with parameter \(0.2\); given B, it is Bernoulli with parameter \(0.7\). Find the overall mean and variance. (Answer: \(E[X]=0.55\), \(\operatorname{Var}(X)=0.2475\))
A randomly selected parcel comes from route 1 or route 2, each with probability \(1/2\). Given route 1, its inspection score \(X\) is 1 or 3 with equal probabilities; given route 2, \(X=4\) with probability 1. Find the mean and variance. (Answer: \(E[X]=3\), \(\operatorname{Var}(X)=1.5\))
Let \(Z\) be uniform on \((0,1)\), and given \(Z=z\), let \(X\) be uniform on \((0,2z)\). Find the two terms in the total-variance identity and hence \(\operatorname{Var}(X)\). (Answer: \(E[\operatorname{Var}(X\mid Z)]=1/9\), \(\operatorname{Var}(E[X\mid Z])=1/12\), \(\operatorname{Var}(X)=7/36\))
A hypothetical processing cost \(X\), in dollars, has a conditional normal distribution given facility \(F\). Facility A is selected with probability \(1/2\), with \(X\mid A\sim N(10,4)\); facility B is selected with probability \(1/2\), with \(X\mid B\sim N(14,9)\). (The second normal parameter is the variance.) Find the overall mean and variance. (Answer: \(E[X]=12\), \(\operatorname{Var}(X)=10.5\) dollars squared)

What You'll Learn