Linearity and Monotonicity of Expectation

Adding Expectations Without an Independence Assumption

Expectations turn random quantities into numerical summaries, and many calculations require the expectation of a sum. The key fact is that expectation is linear: under appropriate finiteness conditions, the expectation of a sum is the sum of the expectations. No independence assumption is needed. A second basic fact is monotonicity: if one random variable is no larger than another, except possibly on an event of probability zero, then its expectation is no larger either. These facts are useful in probability calculations because they let us work with sums and inequalities without first finding the full distribution of every expression.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). A real-valued random variable \(X\) is integrable if \(\mathbb{E}|X|<\infty\). For a discrete random variable with possible values \(x\) and probability mass function \(p_X(x)=\mathbb{P}(X=x)\), this means \(\sum_x |x|p_X(x)<\infty\), and its expectation is \(\mathbb{E}X=\sum_x xp_X(x)\). For a variable with density \(f_X\), integrability means \(\int_{\mathbb{R}}|x|f_X(x)\,dx<\infty\), and \(\mathbb{E}X=\int_{\mathbb{R}}xf_X(x)\,dx\).

In statements involving two random variables, the probability mass function or density of the pair describes their joint distribution. This matters because the variables may be dependent. The marginal expectations are calculated from their individual distributions, while a direct calculation of a function of both variables uses the joint distribution.

Proof. First suppose \((X,Y)\) has a discrete joint probability mass function \(p(x,y)=\mathbb{P}(X=x,Y=y)\). The inequality \(|aX+bY+c|\le |a||X|+|b||Y|+|c|\) shows that \(aX+bY+c\) is integrable: the expectation of the right side is \(|a|\mathbb{E}|X|+|b|\mathbb{E}|Y|+|c|<\infty\). Absolute convergence therefore permits summing over the joint support and regrouping terms. Since \(\sum_{x,y}p(x,y)=1\), and the marginal masses are obtained by summing the joint masses, we have $$ \begin{aligned} \mathbb{E}[aX+bY+c] &=\sum_{x,y}(ax+by+c)p(x,y)\\ &=a\sum_{x,y}x p(x,y)+b\sum_{x,y}y p(x,y)+c\sum_{x,y}p(x,y)\\ &=a\mathbb{E}X+b\mathbb{E}Y+c. \end{aligned} $$ For a pair with joint density \(f(x,y)\), the same integrability bound makes the integrands absolutely integrable. Linearity of integration then gives \(\int_{\mathbb{R}^2}(ax+by+c)f(x,y)\,dx\,dy=a\int_{\mathbb{R}^2}xf(x,y)\,dx\,dy+b\int_{\mathbb{R}^2}yf(x,y)\,dx\,dy+c\int_{\mathbb{R}^2}f(x,y)\,dx\,dy\). Integrating the joint density over the other coordinate gives the marginal densities, and the joint density integrates to \(1\). The expression is thus \(a\mathbb{E}X+b\mathbb{E}Y+c\), as claimed. \(\square\)

For a finite sum of integrable random variables, apply the two-variable result repeatedly. Thus \(\mathbb{E}[\sum_{k=1}^n a_kX_k]=\sum_{k=1}^n a_k\mathbb{E}X_k\) for finite \(n\) and finite constants \(a_k\). The statement concerns expectation of a sum; it does not say that the distribution of the sum can be found by adding marginal distributions.

Linearity with Joint Densities

For continuous random variables, a joint density provides a direct route to the same identity. If \(f(x,y)\) is the joint density of \((X,Y)\), then \(\mathbb{E}[aX+bY]\) is the integral of \(ax+by\) against \(f(x,y)\). When \(X\) and \(Y\) are integrable, this integral is absolutely convergent, so its terms can be separated and evaluated using the marginal densities. The joint density need not factor into a product: that product form would be an extra condition, not a requirement for linearity.

Monotonicity: Preserving Almost-Sure Order

For random variables, \(X\le Y\) almost surely means \(\mathbb{P}(X\le Y)=1\); the inequality is allowed to fail on a set of probability zero. Expectations are insensitive to changes on such sets. Monotonicity says that, when both variables are integrable, this almost-sure order is preserved after taking expectations.

Proof. Define \(D=Y-X\). Since \(|D|\le |Y|+|X|\), \(D\) is integrable. The assumption gives \(D\ge0\) almost surely. For a discrete joint distribution, \(\mathbb{E}D=\sum_{x,y}(y-x)p(x,y)\); every term with positive probability is nonnegative, so the sum is nonnegative. For a joint density, \(\mathbb{E}D=\int_{\mathbb{R}^2}(y-x)f(x,y)\,dx\,dy\). The integrand is nonnegative almost everywhere under the assumption, so its integral is nonnegative. In either setting, linearity gives \(\mathbb{E}Y-\mathbb{E}X=\mathbb{E}(Y-X)\ge0\), which proves the claim. \(\square\)

Why the Conditions Matter

Monotonicity is useful for bounding expectations: if a random quantity can be bounded above or below by a simpler integrable variable, its expectation inherits that bound. Linearity can simplify a sum even when the variables are dependent. The joint distribution may affect other features of a sum, but it does not change this expectation identity.

Common mistake: requiring independence for linearity. In the first worked example, \(X\) and \(Y\) are dependent because the three listed pairs constrain which values occur together. It would be incorrect to say that \(\mathbb{E}[2X-3Y+1]\) cannot be split because of that dependence. The correct calculation uses linearity and gives \(2(1)-3(\tfrac54)+1=-\tfrac34\), which the direct joint-support calculation verifies.

Common mistake: using linearity when expectations are undefined. Integrability is not a technical decoration. For example, let a discrete variable \(X\) have probabilities \(\mathbb{P}(X=n)=\mathbb{P}(X=-n)=c/n^2\) for \(n\ge1\), where \(c\) is chosen to make the total probability one. Then the positive and negative parts of \(X\) both have infinite expectation, so \(\mathbb{E}X\) is undefined. Although \(X+(-X)=0\), writing \(\mathbb{E}[X+(-X)]=\mathbb{E}X+\mathbb{E}[-X]\) is not a valid numerical calculation: the right side is undefined. The theorem applies when the variables being combined are integrable.

For order comparisons, check the direction before taking expectations: \(X\le Y\) implies \(\mathbb{E}X\le\mathbb{E}Y\), not the reverse. The proof reduces the comparison to the nonnegativity of \(Y-X\). In a discrete model, check the difference at every joint outcome with positive probability; in a density model, check it almost everywhere under the joint density. Neither condition should be replaced by a comparison of unrelated marginal support endpoints.