Conditioning to Compute Unconditional Moments

Why Condition on a Simpler Quantity?

An unconditional moment can be difficult to calculate directly when the distribution of a random variable is a mixture of simpler cases. Conditioning separates those cases: calculate the moment within each case, then average the conditional moments using the distribution of what you conditioned on. This approach applies to means and to higher raw moments, not just to probabilities.

For example, a total might be a sum whose second term has a simple distribution once the first term is known. Or two variables might be independent after conditioning on a shared latent variable, even though they are dependent without that conditioning. In both situations, the tower property turns conditional calculations into unconditional ones.

Definition: For an integer $r\geq 1$, the $r$th raw moment of $X$, when $E[|X|^r]<\infty$, is $E[X^r]$. Given a random variable $Y$, the corresponding conditional raw moment is $E[X^r\mid Y]$, a random variable that is a function of $Y$ up to almost-sure equality. Its expectation is the unconditional raw moment: $$ E[X^r]=E\bigl[E[X^r\mid Y]\bigr]. $$

The integrability condition ensures that both sides are finite and that the conditional expectation is well-defined. More generally, for any nonnegative measurable function $g$, the identity $E[g(X)]=E[E[g(X)\mid Y]]$ holds in the extended nonnegative sense, including when the expectation is infinite. For signed functions, a standard sufficient condition is $E[|g(X)|]<\infty$.

When a conditional distribution is available, the inner calculation is ordinary moment calculation under that distribution. For instance, if $X\mid Y=y$ has conditional law $F_{X\mid Y=y}$, then its conditional $r$th raw moment is obtained by integrating $x^r$ against that law. Averaging those conditional moments over the distribution of $Y$ gives $E[X^r]$. For discrete $Y$, this is a weighted sum; for continuous $Y$, it is an integral with respect to the distribution of $Y$.

Conditional Moments of a Sum

Suppose $A$ is known once we condition on a random variable $Y$, and we want a moment of $A+X$. The binomial theorem gives a useful formula: expand the power first, then take conditional expectations term by term. Each power of $A$ can be treated as known under the conditioning.

Theorem: Let $m\geq 1$ be an integer, let $A$ be measurable with respect to $\sigma(Y)$, and suppose $E[|A|^m]+E[|X|^m]<\infty$. Then $$ E[(A+X)^m\mid Y] = \sum_{j=0}^{m}\binom{m}{j}A^{m-j}E[X^j\mid Y], $$ where $E[X^0\mid Y]=1$.

Proof. Pointwise, the binomial theorem gives $(A+X)^m=\sum_{j=0}^{m}\binom{m}{j}A^{m-j}X^j$. Each term is integrable: for $0\leq j\leq m$, Hölder's inequality and the assumed finite $m$th absolute moments imply $E[|A|^{m-j}|X|^j]<\infty$. Conditional expectation is linear for a finite sum, and the pull-out property applies because $A^{m-j}$ is measurable with respect to $\sigma(Y)$. Thus the conditional expectation of each term is $ \binom{m}{j}A^{m-j}E[X^j\mid Y]$, giving the stated formula. $\square$

For the first two moments, the formula reads $$ E[A+X\mid Y]=A+E[X\mid Y], \qquad E[(A+X)^2\mid Y]=A^2+2A E[X\mid Y]+E[X^2\mid Y]. $$ The second identity shows why a conditional mean alone may not be enough to calculate a second moment: the conditional second raw moment is also needed.

Worked Example: Moments of a Sum with a Conditional Bernoulli Term

Let $X$ be Bernoulli with $P(X=0)=P(X=1)=1/2$. Given $X=0$, let $Y$ be Bernoulli with success probability $0.2$; given $X=1$, let $Y$ be Bernoulli with success probability $0.8$. Find the first and second raw moments of $S=X+Y$.

For a Bernoulli variable with success probability $p$, $E[Y\mid X]=p$ and $E[Y^2\mid X]=p$, since $Y^2=Y$. For the mean, $$ E[S\mid X]=X+E[Y\mid X] = \begin{cases} 0.2,&X=0,\\ 1.8,&X=1. \end{cases} $$ Averaging over the two equally likely values of $X$, $$ E[S]=\frac12(0.2)+\frac12(1.8)=1. $$

For the second raw moment, expand the square: $$ E[S^2\mid X]=X^2+2X E[Y\mid X]+E[Y^2\mid X]. $$ If $X=0$, this is $0+0+0.2=0.2$. If $X=1$, it is $1+2(0.8)+0.8=3.4$. Therefore $$ E[S^2]=\frac12(0.2)+\frac12(3.4)=1.8. $$ As a direct check, $S$ can be $0,1,$ or $2$, with probabilities $0.4,0.2,$ and $0.4$, respectively. Thus $E[S]=0(0.4)+1(0.2)+2(0.4)=1$, and $E[S^2]=0^2(0.4)+1^2(0.2)+2^2(0.4)=1.8$.

Product Moments and Conditional Independence

For products, conditioning can be useful when it makes the factors independent. Conditional independence means that, given the conditioning variable, the joint conditional law factors into the product of the two conditional laws. In particular, when the relevant expectations exist, it implies that the conditional expectation of a product is the product of the conditional expectations. Without conditional independence, that step is generally invalid.

Result: If $A$ and $B$ are conditionally independent given $Z$, and $E[|AB|]<\infty$, then $$ E[AB]=E\bigl[E[A\mid Z]E[B\mid Z]\bigr], $$ provided the conditional product is integrable. Indeed, conditional independence gives $E[AB\mid Z]=E[A\mid Z]E[B\mid Z]$ almost surely, and the tower property gives the displayed identity.

Worked Example: A Product Moment in a Two-Group Mixture

Let $Z$ take values 0 and 1 with probability $1/2$ each. Given $Z=z$, let $A$ and $B$ be conditionally independent Bernoulli variables. Set $P(A=1\mid Z=0)=0.2$, $P(A=1\mid Z=1)=0.8$, and $P(B=1\mid Z=0)=0.5$, $P(B=1\mid Z=1)=0.25$. Calculate $E[AB]$ and compare it with $E[A]E[B]$.

Since $A$ and $B$ are indicators, $AB=1$ exactly when both equal 1. Conditional independence gives $$ E[AB\mid Z=0]=(0.2)(0.5)=0.10, \qquad E[AB\mid Z=1]=(0.8)(0.25)=0.20. $$ Averaging over $Z$, $$ E[AB]=\frac12(0.10)+\frac12(0.20)=0.15. $$

The separate means are $$ E[A]=\frac12(0.2)+\frac12(0.8)=0.5, \qquad E[B]=\frac12(0.5)+\frac12(0.25)=0.375. $$ Their product is $0.5(0.375)=0.1875$, which is not $E[AB]=0.15$. Conditional independence within each group does not imply unconditional independence after the groups are mixed.

Moments with a Random Parameter

A random parameter is a random variable that indexes a conditional distribution. Once its value is given, a familiar distribution may make conditional moments easy to calculate. Unconditional moments then follow by averaging over the parameter's own distribution. This is a general mixture calculation: it does not require the parameter to be continuous or to have a density.

Moment averaging rule: If $E[|X|^r]<\infty$ and $\Theta$ is a random parameter, then $$ E[X^r]=E\bigl[E[X^r\mid \Theta]\bigr]. $$ If $\Theta$ has a discrete distribution with values $\theta_i$ and probabilities $p_i$, this becomes $E[X^r]=\sum_i p_i E[X^r\mid\Theta=\theta_i]$. If $\Theta$ has density $f_\Theta$, it becomes $E[X^r]=\int E[X^r\mid\Theta=\theta] f_\Theta(\theta)\,d\theta$, whenever the integral is finite.

Worked Example: First Three Moments of a Poisson Mixture

Let $\Theta$ equal 1 or 3 with probability $1/2$ each, and suppose $X\mid\Theta=\theta\sim\operatorname{Poisson}(\theta)$. Calculate $E[X]$, $E[X^2]$, and $E[X^3]$.

For a Poisson variable with mean $\theta$, the first three factorial moments are $E[X\mid\Theta=\theta]=\theta$, $E[X(X-1)\mid\Theta=\theta]=\theta^2$, and $E[X(X-1)(X-2)\mid\Theta=\theta]=\theta^3$. The identities $X^2=X(X-1)+X$ and $X^3=X(X-1)(X-2)+3X(X-1)+X$ therefore yield $$ E[X^2\mid\Theta=\theta]=\theta^2+\theta, \qquad E[X^3\mid\Theta=\theta]=\theta^3+3\theta^2+\theta. $$

First, $E[\Theta]=(1+3)/2=2$, so $E[X]=E[\Theta]=2$. Next, $E[\Theta^2]=(1^2+3^2)/2=5$, giving $$ E[X^2]=E[\Theta^2+\Theta]=5+2=7. $$ Finally, $E[\Theta^3]=(1^3+3^3)/2=14$, and hence $$ E[X^3]=E[\Theta^3+3\Theta^2+\Theta]=14+3(5)+2=31. $$ The nonlinear terms must be averaged over the distribution of $\Theta$; they cannot in general be evaluated by simply substituting $E[\Theta]$ for $\Theta$.

Common Pitfalls and a Reliable Calculation

The central discipline is to keep the conditioning step explicit. First calculate the conditional moment using the conditional distribution; next average that result over the conditioning variable. The following shortcuts can lead to incorrect answers:

Replacing a random parameter by its mean. In the Poisson mixture, $E[X^2]=E[\Theta^2+\Theta]=7$, not $E[\Theta]^2+E[\Theta]=2^2+2=6$. The two differ because $E[\Theta^2]$ is not generally $E[\Theta]^2$.
Multiplying conditional means without conditional independence. In general, $E[AB\mid Z]$ need not equal $E[A\mid Z]E[B\mid Z]$. The product-moment shortcut above requires conditional independence, not merely that both conditional means are known.
Stopping at a conditional moment. $E[X^r\mid Y]$ can still depend on $Y$; it is not yet the unconditional number $E[X^r]$. Average it over the distribution of $Y$.
Using a higher-moment formula without checking integrability. To apply the stated raw-moment identities as finite quantities, verify the relevant absolute moment exists.

For a sum, write out the algebra before conditioning. For a product, check what independence is available after conditioning. For a mixture, calculate the moment at each parameter value and average the results. These steps make clear which conditional information is being used and where the unconditional distribution enters.

Check Your Understanding

Use conditional moments first, then average over the conditioning variable.

If $E[X\mid Y]=Y$ and $Y$ takes values 1 and 3 with equal probabilities, what is $E[X]$? What additional conditional information would you need to find $E[X^2]$?
Write the conditional expansion of $E[(A+X)^3\mid Y]$ when $A$ is determined by $Y$.
In the product example, why does conditional independence given $Z$ not justify concluding that $E[AB]=E[A]E[B]$?
If $X\mid\Theta=\theta$ has conditional second moment $\theta^2+2\theta$, express $E[X^2]$ in terms of the first two raw moments of $\Theta$.
State an integrability condition that makes the identity $E[X^r]=E[E[X^r\mid Y]]$ a finite-moment identity.