Factorization of Expectations Under Independence

Why Independence Simplifies Expectations

When two random variables are independent, information about one does not change the distribution of the other. A useful consequence is that certain expectations involving both variables can be computed from their separate expectations. For example, under suitable integrability conditions, \(\mathbb{E}[XY]=\mathbb{E}[X]\mathbb{E}[Y]\). This identity is not a rule for arbitrary pairs of variables: independence is essential, and the expectations must exist in a form that makes the product well-defined.

Throughout, let \(X\) and \(Y\) be random variables on a common probability space \((\Omega,\mathcal{F},\mathbb{P})\). They are independent if, for every pair of Borel sets \(A,B\subseteq\mathbb{R}\), \(\mathbb{P}(X\in A,Y\in B)=\mathbb{P}(X\in A)\mathbb{P}(Y\in B)\). In particular, their joint law is the product of their marginal laws. We first establish the expectation rule for countable discrete laws and then for variables with densities.

Proof. Independence gives \(\mathbb{P}(X=x_i,Y=y_j)=p_iq_j\). The absolute sum of the terms for the joint expectation is \(\sum_i\sum_j |g(x_i)h(y_j)|p_iq_j\). Because all terms in this absolute sum are nonnegative, the countable-sum version of Tonelli’s theorem permits the iterated sums to be evaluated in either order. Thus the sum equals \((\sum_i |g(x_i)|p_i)(\sum_j |h(y_j)|q_j)=\mathbb{E}|g(X)|\mathbb{E}|h(Y)|<\infty\). The joint product is therefore integrable, and its signed expectation can be rearranged. Using the joint probabilities again gives \(\mathbb{E}[g(X)h(Y)]=\sum_i\sum_j g(x_i)h(y_j)p_iq_j=(\sum_i g(x_i)p_i)(\sum_j h(y_j)q_j)=\mathbb{E}[g(X)]\mathbb{E}[h(Y)]\). \(\square\)

This proof also explains why the integrability hypotheses matter: they ensure that rearranging the signed double sum is valid. For the common choice \(g(x)=x\) and \(h(y)=y\), finite absolute first moments are sufficient. For powers, the same theorem applies when the corresponding absolute moments are finite.

Independent Variables with Densities

For continuous random variables, expectations are integrals rather than sums. Suppose \(X\) and \(Y\) have densities \(f_X\) and \(f_Y\), and their joint law has density \(f_{X,Y}\). Independence means that this joint law is the product of the marginal laws, so \(f_{X,Y}(x,y)=f_X(x)f_Y(y)\) for almost every \((x,y)\). Equality almost everywhere is sufficient: changing a density on a set of area zero does not change its integrals.

Proof. By independence, the joint density is \(f_X(x)f_Y(y)\) almost everywhere. Tonelli’s theorem for nonnegative integrands gives \(\int_{\mathbb{R}}\int_{\mathbb{R}} |g(x)h(y)|f_X(x)f_Y(y)\,dy\,dx=(\int_{\mathbb{R}}|g(x)|f_X(x)\,dx)(\int_{\mathbb{R}}|h(y)|f_Y(y)\,dy)<\infty\). Thus the joint integrand is absolutely integrable. Fubini’s theorem therefore allows the signed double integral to be evaluated as an iterated integral, giving \(\mathbb{E}[g(X)h(Y)]=\int_{\mathbb{R}}\int_{\mathbb{R}}g(x)h(y)f_X(x)f_Y(y)\,dy\,dx=(\int_{\mathbb{R}}g(x)f_X(x)\,dx)(\int_{\mathbb{R}}h(y)f_Y(y)\,dy)\). The last expression is \(\mathbb{E}[g(X)]\mathbb{E}[h(Y)]\), which proves the result. \(\square\)

The proof uses two standard integration results: Tonelli’s theorem to evaluate a nonnegative double integral, and Fubini’s theorem to change the order of integration for an absolutely integrable function. The assumptions in the theorem ensure that the absolute double integral is finite, so the latter use is justified.

Consequences for Covariance and Moments

For integrable random variables \(U,V\) whose product is integrable, covariance is \(\operatorname{Cov}(U,V)=\mathbb{E}[UV]-\mathbb{E}[U]\mathbb{E}[V]\). If \(X,Y\) are independent and the expectations in this expression are finite, the factorization theorem applied with \(g(x)=x\) and \(h(y)=y\) gives \(\operatorname{Cov}(X,Y)=0\). More generally, functions of independent variables factor: if \(g(X)\) and \(h(Y)\) meet the theorem’s integrability conditions, then their covariance is zero too.

Proof. Finite second moments imply finite first absolute moments, and independence with the factorization theorem gives \(\mathbb{E}[XY]=\mathbb{E}[X]\mathbb{E}[Y]\). Expanding the square, \(\mathbb{E}[(X+Y)^2]=\mathbb{E}[X^2]+2\mathbb{E}[XY]+\mathbb{E}[Y^2]\), while \(\mathbb{E}[X+Y]^2=(\mathbb{E}[X]+\mathbb{E}[Y])^2\). Subtracting the second quantity from the first cancels the cross terms because \(\mathbb{E}[XY]=\mathbb{E}[X]\mathbb{E}[Y]\), leaving \((\mathbb{E}[X^2]-\mathbb{E}[X]^2)+(\mathbb{E}[Y^2]-\mathbb{E}[Y]^2)\). These two terms are the variances of \(X\) and \(Y\), proving the formula. \(\square\)

When Factorization Fails

Zero covariance follows from independence, but zero covariance alone does not establish independence. Nor may the expectation product rule be used just because two variables are being considered together. The rule relies on their joint law being a product of their marginal laws.

Common mistake. Let \(X=Y\), where \(X\) is Bernoulli with \(\mathbb{P}(X=1)=\mathbb{P}(X=0)=\tfrac12\). A mistaken calculation treats \(X\) and \(Y\) as independent and concludes \(\mathbb{E}[XY]=\mathbb{E}[X]\mathbb{E}[Y]=\tfrac12\cdot\tfrac12=\tfrac14\). But \(Y=X\), so \(XY=X^2=X\), and the actual value is \(\mathbb{E}[XY]=\tfrac12\). The discrepancy arises because these variables are dependent: for example, \(\mathbb{P}(X=1,Y=1)=\tfrac12\), whereas \(\mathbb{P}(X=1)\mathbb{P}(Y=1)=\tfrac14\). Here \(\operatorname{Cov}(X,Y)=\tfrac12-\tfrac14=\tfrac14\), not zero.

Before factoring an expectation, check three things: the variables (or the functions of them) are independent; the separate absolute expectations are finite, or another applicable integrability argument is available; and the expression being factored is indeed a product of a function of \(X\) and a function of \(Y\). Under those conditions, mixed moments become products of simpler moments, and covariance calculations often reduce to a direct expansion.